diff --git a/404.html b/404.html
index 5db2c22..c21d9f3 100644
--- a/404.html
+++ b/404.html
@@ -6,7 +6,7 @@
   <meta http-equiv="X-UA-Compatible" content="IE=edge" />
   <title>Page not found | Applied Statistics with R</title>
   <meta name="description" content="Page not found | Applied Statistics with R" />
-  <meta name="generator" content="bookdown 0.37 and GitBook 2.6.7" />
+  <meta name="generator" content="bookdown 0.40 and GitBook 2.6.7" />
 
   <meta property="og:title" content="Page not found | Applied Statistics with R" />
   <meta property="og:type" content="book" />
@@ -55,7 +55,7 @@
 
 <style type="text/css">
 pre > code.sourceCode { white-space: pre; position: relative; }
-pre > code.sourceCode > span { display: inline-block; line-height: 1.25; }
+pre > code.sourceCode > span { line-height: 1.25; }
 pre > code.sourceCode > span:empty { height: 1.2em; }
 .sourceCode { overflow: visible; }
 code.sourceCode > span { color: inherit; text-decoration: inherit; }
diff --git a/analysis-of-variance.html b/analysis-of-variance.html
index 1599592..37656d2 100644
--- a/analysis-of-variance.html
+++ b/analysis-of-variance.html
@@ -6,7 +6,7 @@
   <meta http-equiv="X-UA-Compatible" content="IE=edge" />
   <title>Chapter 12 Analysis of Variance | Applied Statistics with R</title>
   <meta name="description" content="Chapter 12 Analysis of Variance | Applied Statistics with R" />
-  <meta name="generator" content="bookdown 0.37 and GitBook 2.6.7" />
+  <meta name="generator" content="bookdown 0.40 and GitBook 2.6.7" />
 
   <meta property="og:title" content="Chapter 12 Analysis of Variance | Applied Statistics with R" />
   <meta property="og:type" content="book" />
@@ -55,7 +55,7 @@
 
 <style type="text/css">
 pre > code.sourceCode { white-space: pre; position: relative; }
-pre > code.sourceCode > span { display: inline-block; line-height: 1.25; }
+pre > code.sourceCode > span { line-height: 1.25; }
 pre > code.sourceCode > span:empty { height: 1.2em; }
 .sourceCode { overflow: visible; }
 code.sourceCode > span { color: inherit; text-decoration: inherit; }
@@ -524,7 +524,7 @@ <h2><span class="header-section-number">12.2</span> Two-Sample t-Test<a href="an
 \]</span></p>
 <p>For the stated model and assuming the null hypothesis is true, the <span class="math inline">\(t\)</span> test statistic would follow a <span class="math inline">\(t\)</span> distribution with degrees of freedom <span class="math inline">\(n_1 + n_2 - 2\)</span>.</p>
 <p>As an example, suppose we are interested in the effect of <a href="https://en.wikipedia.org/wiki/Melatonin" target="_blank">melatonin</a> on sleep duration. A researcher obtains a random sample of 20 adult males. Of these subjects, 10 are randomly chosen for the control group, which will receive a placebo. The remaining 10 will be given 5mg of melatonin before bed. The sleep duration in hours of each subject is then measured. The researcher chooses a significance level of <span class="math inline">\(\alpha = 0.10\)</span>. Was sleep duration affected by the melatonin?</p>
-<div class="sourceCode" id="cb807"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb807-1"><a href="analysis-of-variance.html#cb807-1" aria-hidden="true" tabindex="-1"></a>melatonin</span></code></pre></div>
+<div class="sourceCode" id="cb807"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb807-1"><a href="analysis-of-variance.html#cb807-1" tabindex="-1"></a>melatonin</span></code></pre></div>
 <pre><code>##        sleep     group
 ## 1   8.145150   control
 ## 2   7.522362 treatment
@@ -551,7 +551,7 @@ <h2><span class="header-section-number">12.2</span> Two-Sample t-Test<a href="an
 H_0: \mu_C = \mu_T \quad \text{vs} \quad H_1: \mu_C \neq \mu_T
 \]</span></p>
 <p>To do so in <code>R</code>, we use the <code>t.test()</code> function, with the <code>var.equal</code> argument set to <code>TRUE</code>.</p>
-<div class="sourceCode" id="cb809"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb809-1"><a href="analysis-of-variance.html#cb809-1" aria-hidden="true" tabindex="-1"></a><span class="fu">t.test</span>(sleep <span class="sc">~</span> group, <span class="at">data =</span> melatonin, <span class="at">var.equal =</span> <span class="cn">TRUE</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb809"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb809-1"><a href="analysis-of-variance.html#cb809-1" tabindex="-1"></a><span class="fu">t.test</span>(sleep <span class="sc">~</span> group, <span class="at">data =</span> melatonin, <span class="at">var.equal =</span> <span class="cn">TRUE</span>)</span></code></pre></div>
 <pre><code>## 
 ##  Two Sample t-test
 ## 
@@ -564,12 +564,12 @@ <h2><span class="header-section-number">12.2</span> Two-Sample t-Test<a href="an
 ##   mean in group control mean in group treatment 
 ##                6.827152                8.333456</code></pre>
 <p>At a significance level of <span class="math inline">\(\alpha = 0.10\)</span>, we reject the null hypothesis. It seems that the melatonin had a <strong>statistically significant</strong> effect. Be aware that statistical significance is not always the same as scientific or <strong>practical significance</strong>. To determine practical significance, we need to investigate the <strong>effect size</strong> in the context of the situation. Here the effect size is the difference of the sample means.</p>
-<div class="sourceCode" id="cb811"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb811-1"><a href="analysis-of-variance.html#cb811-1" aria-hidden="true" tabindex="-1"></a><span class="fu">t.test</span>(sleep <span class="sc">~</span> group, <span class="at">data =</span> melatonin, <span class="at">var.equal =</span> <span class="cn">TRUE</span>)<span class="sc">$</span>estimate</span></code></pre></div>
+<div class="sourceCode" id="cb811"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb811-1"><a href="analysis-of-variance.html#cb811-1" tabindex="-1"></a><span class="fu">t.test</span>(sleep <span class="sc">~</span> group, <span class="at">data =</span> melatonin, <span class="at">var.equal =</span> <span class="cn">TRUE</span>)<span class="sc">$</span>estimate</span></code></pre></div>
 <pre><code>##   mean in group control mean in group treatment 
 ##                6.827152                8.333456</code></pre>
 <p>Here we see that the subjects in the melatonin group sleep an average of about 1.5 hours longer than the control group. An hour and a half of sleep is certainly important!</p>
 <p>With a big enough sample size, we could make an effect size of say, four minutes statistically significant. Is it worth taking a pill every night to get an extra four minutes of sleep? (Probably not.)</p>
-<div class="sourceCode" id="cb813"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb813-1"><a href="analysis-of-variance.html#cb813-1" aria-hidden="true" tabindex="-1"></a><span class="fu">boxplot</span>(sleep <span class="sc">~</span> group, <span class="at">data =</span> melatonin, <span class="at">col =</span> <span class="dv">5</span><span class="sc">:</span><span class="dv">6</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb813"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb813-1"><a href="analysis-of-variance.html#cb813-1" tabindex="-1"></a><span class="fu">boxplot</span>(sleep <span class="sc">~</span> group, <span class="at">data =</span> melatonin, <span class="at">col =</span> <span class="dv">5</span><span class="sc">:</span><span class="dv">6</span>)</span></code></pre></div>
 <p><img src="anova_files/figure-html/unnamed-chunk-6-1.png" width="672" style="display: block; margin: auto;" /></p>
 <!-- TODO: other parametarization, explain identify -->
 </div>
@@ -723,14 +723,14 @@ <h2><span class="header-section-number">12.3</span> One-Way ANOVA<a href="analys
 H_0: \mu_A = \mu_B = \mu_C = \mu_D
 \]</span></p>
 <p>where, for example, <span class="math inline">\(\mu_A\)</span> is the mean blood coagulation time for an animal that ate diet <code>A</code>.</p>
-<div class="sourceCode" id="cb814"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb814-1"><a href="analysis-of-variance.html#cb814-1" aria-hidden="true" tabindex="-1"></a><span class="fu">library</span>(faraway)</span>
-<span id="cb814-2"><a href="analysis-of-variance.html#cb814-2" aria-hidden="true" tabindex="-1"></a><span class="fu">names</span>(coagulation)</span></code></pre></div>
+<div class="sourceCode" id="cb814"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb814-1"><a href="analysis-of-variance.html#cb814-1" tabindex="-1"></a><span class="fu">library</span>(faraway)</span>
+<span id="cb814-2"><a href="analysis-of-variance.html#cb814-2" tabindex="-1"></a><span class="fu">names</span>(coagulation)</span></code></pre></div>
 <pre><code>## [1] &quot;coag&quot; &quot;diet&quot;</code></pre>
-<div class="sourceCode" id="cb816"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb816-1"><a href="analysis-of-variance.html#cb816-1" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(coag <span class="sc">~</span> diet, <span class="at">data =</span> coagulation, <span class="at">col =</span> <span class="dv">2</span><span class="sc">:</span><span class="dv">5</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb816"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb816-1"><a href="analysis-of-variance.html#cb816-1" tabindex="-1"></a><span class="fu">plot</span>(coag <span class="sc">~</span> diet, <span class="at">data =</span> coagulation, <span class="at">col =</span> <span class="dv">2</span><span class="sc">:</span><span class="dv">5</span>)</span></code></pre></div>
 <p><img src="anova_files/figure-html/unnamed-chunk-13-1.png" width="672" style="display: block; margin: auto;" /></p>
 <p>We first load the data and create the relevant boxplot. The plot alone suggests a difference of means. The <code>aov()</code> function is used to obtain the relevant sums of squares. Using the <code>summary()</code> function on the output from <code>aov()</code> creates the desired ANOVA table (without the unneeded row for total).</p>
-<div class="sourceCode" id="cb817"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb817-1"><a href="analysis-of-variance.html#cb817-1" aria-hidden="true" tabindex="-1"></a>coag_aov <span class="ot">=</span> <span class="fu">aov</span>(coag <span class="sc">~</span> diet, <span class="at">data =</span> coagulation)</span>
-<span id="cb817-2"><a href="analysis-of-variance.html#cb817-2" aria-hidden="true" tabindex="-1"></a>coag_aov</span></code></pre></div>
+<div class="sourceCode" id="cb817"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb817-1"><a href="analysis-of-variance.html#cb817-1" tabindex="-1"></a>coag_aov <span class="ot">=</span> <span class="fu">aov</span>(coag <span class="sc">~</span> diet, <span class="at">data =</span> coagulation)</span>
+<span id="cb817-2"><a href="analysis-of-variance.html#cb817-2" tabindex="-1"></a>coag_aov</span></code></pre></div>
 <pre><code>## Call:
 ##    aov(formula = coag ~ diet, data = coagulation)
 ## 
@@ -741,15 +741,15 @@ <h2><span class="header-section-number">12.3</span> One-Way ANOVA<a href="analys
 ## 
 ## Residual standard error: 2.366432
 ## Estimated effects may be unbalanced</code></pre>
-<div class="sourceCode" id="cb819"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb819-1"><a href="analysis-of-variance.html#cb819-1" aria-hidden="true" tabindex="-1"></a><span class="fu">summary</span>(coag_aov)</span></code></pre></div>
+<div class="sourceCode" id="cb819"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb819-1"><a href="analysis-of-variance.html#cb819-1" tabindex="-1"></a><span class="fu">summary</span>(coag_aov)</span></code></pre></div>
 <pre><code>##             Df Sum Sq Mean Sq F value   Pr(&gt;F)    
 ## diet         3    228    76.0   13.57 4.66e-05 ***
 ## Residuals   20    112     5.6                     
 ## ---
 ## Signif. codes:  0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1</code></pre>
 <p>Were we to run this experiment, we would have pre-specified a significance level. However, notice that the p-value of this test is incredibly low, so using any reasonable significance level we would reject the null hypothesis. Thus we believe the diets had an effect on blood coagulation time.</p>
-<div class="sourceCode" id="cb821"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb821-1"><a href="analysis-of-variance.html#cb821-1" aria-hidden="true" tabindex="-1"></a>diets <span class="ot">=</span> <span class="fu">data.frame</span>(<span class="at">diet =</span> <span class="fu">unique</span>(coagulation<span class="sc">$</span>diet))</span>
-<span id="cb821-2"><a href="analysis-of-variance.html#cb821-2" aria-hidden="true" tabindex="-1"></a><span class="fu">data.frame</span>(diets, <span class="at">coag =</span> <span class="fu">predict</span>(coag_aov, diets))</span></code></pre></div>
+<div class="sourceCode" id="cb821"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb821-1"><a href="analysis-of-variance.html#cb821-1" tabindex="-1"></a>diets <span class="ot">=</span> <span class="fu">data.frame</span>(<span class="at">diet =</span> <span class="fu">unique</span>(coagulation<span class="sc">$</span>diet))</span>
+<span id="cb821-2"><a href="analysis-of-variance.html#cb821-2" tabindex="-1"></a><span class="fu">data.frame</span>(diets, <span class="at">coag =</span> <span class="fu">predict</span>(coag_aov, diets))</span></code></pre></div>
 <pre><code>##   diet coag
 ## 1    A   61
 ## 2    B   66
@@ -759,56 +759,56 @@ <h2><span class="header-section-number">12.3</span> One-Way ANOVA<a href="analys
 <div id="factor-variables-1" class="section level3 hasAnchor" number="12.3.1">
 <h3><span class="header-section-number">12.3.1</span> Factor Variables<a href="analysis-of-variance.html#factor-variables-1" class="anchor-section" aria-label="Anchor link to header"></a></h3>
 <p>When performing ANOVA in <code>R</code>, be sure the grouping variable is a factor variable. If it is not, your result might not be ANOVA, but instead a linear regression with the predictor variable considered numeric.</p>
-<div class="sourceCode" id="cb823"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb823-1"><a href="analysis-of-variance.html#cb823-1" aria-hidden="true" tabindex="-1"></a><span class="fu">set.seed</span>(<span class="dv">42</span>)</span>
-<span id="cb823-2"><a href="analysis-of-variance.html#cb823-2" aria-hidden="true" tabindex="-1"></a>response <span class="ot">=</span> <span class="fu">rnorm</span>(<span class="dv">15</span>)</span>
-<span id="cb823-3"><a href="analysis-of-variance.html#cb823-3" aria-hidden="true" tabindex="-1"></a>group    <span class="ot">=</span> <span class="fu">c</span>(<span class="fu">rep</span>(<span class="dv">1</span>, <span class="dv">5</span>), <span class="fu">rep</span>(<span class="dv">2</span>, <span class="dv">5</span>), <span class="fu">rep</span>(<span class="dv">3</span>, <span class="dv">5</span>))</span>
-<span id="cb823-4"><a href="analysis-of-variance.html#cb823-4" aria-hidden="true" tabindex="-1"></a>bad <span class="ot">=</span> <span class="fu">data.frame</span>(response, group)</span>
-<span id="cb823-5"><a href="analysis-of-variance.html#cb823-5" aria-hidden="true" tabindex="-1"></a><span class="fu">summary</span>(<span class="fu">aov</span>(response <span class="sc">~</span> group, <span class="at">data =</span> bad)) <span class="co"># wrong DF!</span></span></code></pre></div>
+<div class="sourceCode" id="cb823"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb823-1"><a href="analysis-of-variance.html#cb823-1" tabindex="-1"></a><span class="fu">set.seed</span>(<span class="dv">42</span>)</span>
+<span id="cb823-2"><a href="analysis-of-variance.html#cb823-2" tabindex="-1"></a>response <span class="ot">=</span> <span class="fu">rnorm</span>(<span class="dv">15</span>)</span>
+<span id="cb823-3"><a href="analysis-of-variance.html#cb823-3" tabindex="-1"></a>group    <span class="ot">=</span> <span class="fu">c</span>(<span class="fu">rep</span>(<span class="dv">1</span>, <span class="dv">5</span>), <span class="fu">rep</span>(<span class="dv">2</span>, <span class="dv">5</span>), <span class="fu">rep</span>(<span class="dv">3</span>, <span class="dv">5</span>))</span>
+<span id="cb823-4"><a href="analysis-of-variance.html#cb823-4" tabindex="-1"></a>bad <span class="ot">=</span> <span class="fu">data.frame</span>(response, group)</span>
+<span id="cb823-5"><a href="analysis-of-variance.html#cb823-5" tabindex="-1"></a><span class="fu">summary</span>(<span class="fu">aov</span>(response <span class="sc">~</span> group, <span class="at">data =</span> bad)) <span class="co"># wrong DF!</span></span></code></pre></div>
 <pre><code>##             Df Sum Sq Mean Sq F value Pr(&gt;F)
 ## group        1  0.017  0.0173   0.015  0.903
 ## Residuals   13 14.698  1.1306</code></pre>
-<div class="sourceCode" id="cb825"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb825-1"><a href="analysis-of-variance.html#cb825-1" aria-hidden="true" tabindex="-1"></a>good <span class="ot">=</span> <span class="fu">data.frame</span>(response, <span class="at">group =</span> <span class="fu">as.factor</span>(group))</span>
-<span id="cb825-2"><a href="analysis-of-variance.html#cb825-2" aria-hidden="true" tabindex="-1"></a><span class="fu">summary</span>(<span class="fu">aov</span>(response <span class="sc">~</span> group, <span class="at">data =</span> good))</span></code></pre></div>
+<div class="sourceCode" id="cb825"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb825-1"><a href="analysis-of-variance.html#cb825-1" tabindex="-1"></a>good <span class="ot">=</span> <span class="fu">data.frame</span>(response, <span class="at">group =</span> <span class="fu">as.factor</span>(group))</span>
+<span id="cb825-2"><a href="analysis-of-variance.html#cb825-2" tabindex="-1"></a><span class="fu">summary</span>(<span class="fu">aov</span>(response <span class="sc">~</span> group, <span class="at">data =</span> good))</span></code></pre></div>
 <pre><code>##             Df Sum Sq Mean Sq F value Pr(&gt;F)
 ## group        2  0.232  0.1158   0.096  0.909
 ## Residuals   12 14.484  1.2070</code></pre>
-<div class="sourceCode" id="cb827"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb827-1"><a href="analysis-of-variance.html#cb827-1" aria-hidden="true" tabindex="-1"></a><span class="fu">is.factor</span>(bad<span class="sc">$</span>group)  <span class="co"># 1, 2, and 3 are numbers.</span></span></code></pre></div>
+<div class="sourceCode" id="cb827"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb827-1"><a href="analysis-of-variance.html#cb827-1" tabindex="-1"></a><span class="fu">is.factor</span>(bad<span class="sc">$</span>group)  <span class="co"># 1, 2, and 3 are numbers.</span></span></code></pre></div>
 <pre><code>## [1] FALSE</code></pre>
-<div class="sourceCode" id="cb829"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb829-1"><a href="analysis-of-variance.html#cb829-1" aria-hidden="true" tabindex="-1"></a><span class="fu">is.factor</span>(good<span class="sc">$</span>group) <span class="co"># 1, 2, and 3 are labels.</span></span></code></pre></div>
+<div class="sourceCode" id="cb829"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb829-1"><a href="analysis-of-variance.html#cb829-1" tabindex="-1"></a><span class="fu">is.factor</span>(good<span class="sc">$</span>group) <span class="co"># 1, 2, and 3 are labels.</span></span></code></pre></div>
 <pre><code>## [1] TRUE</code></pre>
 </div>
 <div id="some-simulation" class="section level3 hasAnchor" number="12.3.2">
 <h3><span class="header-section-number">12.3.2</span> Some Simulation<a href="analysis-of-variance.html#some-simulation" class="anchor-section" aria-label="Anchor link to header"></a></h3>
 <p>Here we verify the distribution of the test statistic under the null hypothesis. We simulate from a null model (equal variance) to obtain an empirical distribution of the <span class="math inline">\(F\)</span> statistic. We add the curve for the expected distribution.</p>
-<div class="sourceCode" id="cb831"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb831-1"><a href="analysis-of-variance.html#cb831-1" aria-hidden="true" tabindex="-1"></a><span class="fu">library</span>(broom)</span>
-<span id="cb831-2"><a href="analysis-of-variance.html#cb831-2" aria-hidden="true" tabindex="-1"></a></span>
-<span id="cb831-3"><a href="analysis-of-variance.html#cb831-3" aria-hidden="true" tabindex="-1"></a>sim_anova <span class="ot">=</span> <span class="cf">function</span>(<span class="at">n =</span> <span class="dv">10</span>, <span class="at">mu_a =</span> <span class="dv">0</span>, <span class="at">mu_b =</span> <span class="dv">0</span>, <span class="at">mu_c =</span> <span class="dv">0</span>, <span class="at">mu_d =</span> <span class="dv">0</span>, <span class="at">sigma =</span> <span class="dv">1</span>, <span class="at">stat =</span> <span class="cn">TRUE</span>) {</span>
-<span id="cb831-4"><a href="analysis-of-variance.html#cb831-4" aria-hidden="true" tabindex="-1"></a>  </span>
-<span id="cb831-5"><a href="analysis-of-variance.html#cb831-5" aria-hidden="true" tabindex="-1"></a>  <span class="co"># create data from one-way ANOVA model with four groups of equal size</span></span>
-<span id="cb831-6"><a href="analysis-of-variance.html#cb831-6" aria-hidden="true" tabindex="-1"></a>  <span class="co"># response simulated from normal with group mean, shared variance</span></span>
-<span id="cb831-7"><a href="analysis-of-variance.html#cb831-7" aria-hidden="true" tabindex="-1"></a>  <span class="co"># group variable indicates group A, B, C or D</span></span>
-<span id="cb831-8"><a href="analysis-of-variance.html#cb831-8" aria-hidden="true" tabindex="-1"></a>  sim_data <span class="ot">=</span> <span class="fu">data.frame</span>(</span>
-<span id="cb831-9"><a href="analysis-of-variance.html#cb831-9" aria-hidden="true" tabindex="-1"></a>    <span class="at">response =</span> <span class="fu">c</span>(<span class="fu">rnorm</span>(<span class="at">n =</span> n, <span class="at">mean =</span> mu_a, <span class="at">sd =</span> sigma),</span>
-<span id="cb831-10"><a href="analysis-of-variance.html#cb831-10" aria-hidden="true" tabindex="-1"></a>                 <span class="fu">rnorm</span>(<span class="at">n =</span> n, <span class="at">mean =</span> mu_b, <span class="at">sd =</span> sigma),</span>
-<span id="cb831-11"><a href="analysis-of-variance.html#cb831-11" aria-hidden="true" tabindex="-1"></a>                 <span class="fu">rnorm</span>(<span class="at">n =</span> n, <span class="at">mean =</span> mu_c, <span class="at">sd =</span> sigma),</span>
-<span id="cb831-12"><a href="analysis-of-variance.html#cb831-12" aria-hidden="true" tabindex="-1"></a>                 <span class="fu">rnorm</span>(<span class="at">n =</span> n, <span class="at">mean =</span> mu_d, <span class="at">sd =</span> sigma)),</span>
-<span id="cb831-13"><a href="analysis-of-variance.html#cb831-13" aria-hidden="true" tabindex="-1"></a>    <span class="at">group =</span> <span class="fu">c</span>(<span class="fu">rep</span>(<span class="st">&quot;A&quot;</span>, <span class="at">times =</span> n), <span class="fu">rep</span>(<span class="st">&quot;B&quot;</span>, <span class="at">times =</span> n), </span>
-<span id="cb831-14"><a href="analysis-of-variance.html#cb831-14" aria-hidden="true" tabindex="-1"></a>              <span class="fu">rep</span>(<span class="st">&quot;C&quot;</span>, <span class="at">times =</span> n), <span class="fu">rep</span>(<span class="st">&quot;D&quot;</span>, <span class="at">times =</span> n))</span>
-<span id="cb831-15"><a href="analysis-of-variance.html#cb831-15" aria-hidden="true" tabindex="-1"></a>  )</span>
-<span id="cb831-16"><a href="analysis-of-variance.html#cb831-16" aria-hidden="true" tabindex="-1"></a>  </span>
-<span id="cb831-17"><a href="analysis-of-variance.html#cb831-17" aria-hidden="true" tabindex="-1"></a>  <span class="co"># obtain F-statistic and p-value for testing difference of means</span></span>
-<span id="cb831-18"><a href="analysis-of-variance.html#cb831-18" aria-hidden="true" tabindex="-1"></a>  <span class="co"># use lm instead of aov for better result formatting with glance</span></span>
-<span id="cb831-19"><a href="analysis-of-variance.html#cb831-19" aria-hidden="true" tabindex="-1"></a>  aov_results <span class="ot">=</span> <span class="fu">lm</span>(response <span class="sc">~</span> group, <span class="at">data =</span> sim_data)</span>
-<span id="cb831-20"><a href="analysis-of-variance.html#cb831-20" aria-hidden="true" tabindex="-1"></a>  f_stat <span class="ot">=</span> <span class="fu">glance</span>(aov_results)<span class="sc">$</span>statistic</span>
-<span id="cb831-21"><a href="analysis-of-variance.html#cb831-21" aria-hidden="true" tabindex="-1"></a>  p_val  <span class="ot">=</span> <span class="fu">glance</span>(aov_results)<span class="sc">$</span>p.value</span>
-<span id="cb831-22"><a href="analysis-of-variance.html#cb831-22" aria-hidden="true" tabindex="-1"></a>  </span>
-<span id="cb831-23"><a href="analysis-of-variance.html#cb831-23" aria-hidden="true" tabindex="-1"></a>  <span class="co"># return f_stat if stat = TRUE, otherwise, p-value</span></span>
-<span id="cb831-24"><a href="analysis-of-variance.html#cb831-24" aria-hidden="true" tabindex="-1"></a>  <span class="fu">ifelse</span>(stat, f_stat, p_val)</span>
-<span id="cb831-25"><a href="analysis-of-variance.html#cb831-25" aria-hidden="true" tabindex="-1"></a>  </span>
-<span id="cb831-26"><a href="analysis-of-variance.html#cb831-26" aria-hidden="true" tabindex="-1"></a>}</span></code></pre></div>
-<div class="sourceCode" id="cb832"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb832-1"><a href="analysis-of-variance.html#cb832-1" aria-hidden="true" tabindex="-1"></a>f_stats <span class="ot">=</span> <span class="fu">replicate</span>(<span class="at">n =</span> <span class="dv">5000</span>, <span class="fu">sim_anova</span>(<span class="at">stat =</span> <span class="cn">TRUE</span>))</span></code></pre></div>
-<div class="sourceCode" id="cb833"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb833-1"><a href="analysis-of-variance.html#cb833-1" aria-hidden="true" tabindex="-1"></a><span class="fu">hist</span>(f_stats, <span class="at">breaks =</span> <span class="dv">100</span>, <span class="at">prob =</span> <span class="cn">TRUE</span>, <span class="at">border =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">main =</span> <span class="st">&quot;Empirical Distribution of F&quot;</span>)</span>
-<span id="cb833-2"><a href="analysis-of-variance.html#cb833-2" aria-hidden="true" tabindex="-1"></a><span class="fu">curve</span>(<span class="fu">df</span>(x, <span class="at">df1 =</span> <span class="dv">4</span> <span class="sc">-</span> <span class="dv">1</span>, <span class="at">df2 =</span> <span class="dv">40</span> <span class="sc">-</span> <span class="dv">4</span>), <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">add =</span> <span class="cn">TRUE</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb831"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb831-1"><a href="analysis-of-variance.html#cb831-1" tabindex="-1"></a><span class="fu">library</span>(broom)</span>
+<span id="cb831-2"><a href="analysis-of-variance.html#cb831-2" tabindex="-1"></a></span>
+<span id="cb831-3"><a href="analysis-of-variance.html#cb831-3" tabindex="-1"></a>sim_anova <span class="ot">=</span> <span class="cf">function</span>(<span class="at">n =</span> <span class="dv">10</span>, <span class="at">mu_a =</span> <span class="dv">0</span>, <span class="at">mu_b =</span> <span class="dv">0</span>, <span class="at">mu_c =</span> <span class="dv">0</span>, <span class="at">mu_d =</span> <span class="dv">0</span>, <span class="at">sigma =</span> <span class="dv">1</span>, <span class="at">stat =</span> <span class="cn">TRUE</span>) {</span>
+<span id="cb831-4"><a href="analysis-of-variance.html#cb831-4" tabindex="-1"></a>  </span>
+<span id="cb831-5"><a href="analysis-of-variance.html#cb831-5" tabindex="-1"></a>  <span class="co"># create data from one-way ANOVA model with four groups of equal size</span></span>
+<span id="cb831-6"><a href="analysis-of-variance.html#cb831-6" tabindex="-1"></a>  <span class="co"># response simulated from normal with group mean, shared variance</span></span>
+<span id="cb831-7"><a href="analysis-of-variance.html#cb831-7" tabindex="-1"></a>  <span class="co"># group variable indicates group A, B, C or D</span></span>
+<span id="cb831-8"><a href="analysis-of-variance.html#cb831-8" tabindex="-1"></a>  sim_data <span class="ot">=</span> <span class="fu">data.frame</span>(</span>
+<span id="cb831-9"><a href="analysis-of-variance.html#cb831-9" tabindex="-1"></a>    <span class="at">response =</span> <span class="fu">c</span>(<span class="fu">rnorm</span>(<span class="at">n =</span> n, <span class="at">mean =</span> mu_a, <span class="at">sd =</span> sigma),</span>
+<span id="cb831-10"><a href="analysis-of-variance.html#cb831-10" tabindex="-1"></a>                 <span class="fu">rnorm</span>(<span class="at">n =</span> n, <span class="at">mean =</span> mu_b, <span class="at">sd =</span> sigma),</span>
+<span id="cb831-11"><a href="analysis-of-variance.html#cb831-11" tabindex="-1"></a>                 <span class="fu">rnorm</span>(<span class="at">n =</span> n, <span class="at">mean =</span> mu_c, <span class="at">sd =</span> sigma),</span>
+<span id="cb831-12"><a href="analysis-of-variance.html#cb831-12" tabindex="-1"></a>                 <span class="fu">rnorm</span>(<span class="at">n =</span> n, <span class="at">mean =</span> mu_d, <span class="at">sd =</span> sigma)),</span>
+<span id="cb831-13"><a href="analysis-of-variance.html#cb831-13" tabindex="-1"></a>    <span class="at">group =</span> <span class="fu">c</span>(<span class="fu">rep</span>(<span class="st">&quot;A&quot;</span>, <span class="at">times =</span> n), <span class="fu">rep</span>(<span class="st">&quot;B&quot;</span>, <span class="at">times =</span> n), </span>
+<span id="cb831-14"><a href="analysis-of-variance.html#cb831-14" tabindex="-1"></a>              <span class="fu">rep</span>(<span class="st">&quot;C&quot;</span>, <span class="at">times =</span> n), <span class="fu">rep</span>(<span class="st">&quot;D&quot;</span>, <span class="at">times =</span> n))</span>
+<span id="cb831-15"><a href="analysis-of-variance.html#cb831-15" tabindex="-1"></a>  )</span>
+<span id="cb831-16"><a href="analysis-of-variance.html#cb831-16" tabindex="-1"></a>  </span>
+<span id="cb831-17"><a href="analysis-of-variance.html#cb831-17" tabindex="-1"></a>  <span class="co"># obtain F-statistic and p-value for testing difference of means</span></span>
+<span id="cb831-18"><a href="analysis-of-variance.html#cb831-18" tabindex="-1"></a>  <span class="co"># use lm instead of aov for better result formatting with glance</span></span>
+<span id="cb831-19"><a href="analysis-of-variance.html#cb831-19" tabindex="-1"></a>  aov_results <span class="ot">=</span> <span class="fu">lm</span>(response <span class="sc">~</span> group, <span class="at">data =</span> sim_data)</span>
+<span id="cb831-20"><a href="analysis-of-variance.html#cb831-20" tabindex="-1"></a>  f_stat <span class="ot">=</span> <span class="fu">glance</span>(aov_results)<span class="sc">$</span>statistic</span>
+<span id="cb831-21"><a href="analysis-of-variance.html#cb831-21" tabindex="-1"></a>  p_val  <span class="ot">=</span> <span class="fu">glance</span>(aov_results)<span class="sc">$</span>p.value</span>
+<span id="cb831-22"><a href="analysis-of-variance.html#cb831-22" tabindex="-1"></a>  </span>
+<span id="cb831-23"><a href="analysis-of-variance.html#cb831-23" tabindex="-1"></a>  <span class="co"># return f_stat if stat = TRUE, otherwise, p-value</span></span>
+<span id="cb831-24"><a href="analysis-of-variance.html#cb831-24" tabindex="-1"></a>  <span class="fu">ifelse</span>(stat, f_stat, p_val)</span>
+<span id="cb831-25"><a href="analysis-of-variance.html#cb831-25" tabindex="-1"></a>  </span>
+<span id="cb831-26"><a href="analysis-of-variance.html#cb831-26" tabindex="-1"></a>}</span></code></pre></div>
+<div class="sourceCode" id="cb832"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb832-1"><a href="analysis-of-variance.html#cb832-1" tabindex="-1"></a>f_stats <span class="ot">=</span> <span class="fu">replicate</span>(<span class="at">n =</span> <span class="dv">5000</span>, <span class="fu">sim_anova</span>(<span class="at">stat =</span> <span class="cn">TRUE</span>))</span></code></pre></div>
+<div class="sourceCode" id="cb833"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb833-1"><a href="analysis-of-variance.html#cb833-1" tabindex="-1"></a><span class="fu">hist</span>(f_stats, <span class="at">breaks =</span> <span class="dv">100</span>, <span class="at">prob =</span> <span class="cn">TRUE</span>, <span class="at">border =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">main =</span> <span class="st">&quot;Empirical Distribution of F&quot;</span>)</span>
+<span id="cb833-2"><a href="analysis-of-variance.html#cb833-2" tabindex="-1"></a><span class="fu">curve</span>(<span class="fu">df</span>(x, <span class="at">df1 =</span> <span class="dv">4</span> <span class="sc">-</span> <span class="dv">1</span>, <span class="at">df2 =</span> <span class="dv">40</span> <span class="sc">-</span> <span class="dv">4</span>), <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">add =</span> <span class="cn">TRUE</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
 <p><img src="anova_files/figure-html/sim-anova-graphs-1.png" width="672" style="display: block; margin: auto;" /></p>
 </div>
 <div id="power" class="section level3 hasAnchor" number="12.3.3">
@@ -829,30 +829,30 @@ <h3><span class="header-section-number">12.3.3</span> Power<a href="analysis-of-
 <li><strong>Balanced design</strong>. An equal number of observations per group leads to higher power.</li>
 </ul>
 <p>The following simulations look at the effect of significance level, effect size, and noise level on the power of an ANOVA <span class="math inline">\(F\)</span>-test. Homework will look into sample size and balance.</p>
-<div class="sourceCode" id="cb834"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb834-1"><a href="analysis-of-variance.html#cb834-1" aria-hidden="true" tabindex="-1"></a>p_vals <span class="ot">=</span> <span class="fu">replicate</span>(<span class="at">n =</span> <span class="dv">1000</span>, <span class="fu">sim_anova</span>(<span class="at">mu_a =</span> <span class="sc">-</span><span class="dv">1</span>, <span class="at">mu_b =</span> <span class="dv">0</span>, <span class="at">mu_c =</span> <span class="dv">0</span>, <span class="at">mu_d =</span> <span class="dv">1</span>,</span>
-<span id="cb834-2"><a href="analysis-of-variance.html#cb834-2" aria-hidden="true" tabindex="-1"></a>                                     <span class="at">sigma =</span> <span class="fl">1.5</span>, <span class="at">stat =</span> <span class="cn">FALSE</span>))</span>
-<span id="cb834-3"><a href="analysis-of-variance.html#cb834-3" aria-hidden="true" tabindex="-1"></a><span class="fu">mean</span>(p_vals <span class="sc">&lt;</span> <span class="fl">0.05</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb834"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb834-1"><a href="analysis-of-variance.html#cb834-1" tabindex="-1"></a>p_vals <span class="ot">=</span> <span class="fu">replicate</span>(<span class="at">n =</span> <span class="dv">1000</span>, <span class="fu">sim_anova</span>(<span class="at">mu_a =</span> <span class="sc">-</span><span class="dv">1</span>, <span class="at">mu_b =</span> <span class="dv">0</span>, <span class="at">mu_c =</span> <span class="dv">0</span>, <span class="at">mu_d =</span> <span class="dv">1</span>,</span>
+<span id="cb834-2"><a href="analysis-of-variance.html#cb834-2" tabindex="-1"></a>                                     <span class="at">sigma =</span> <span class="fl">1.5</span>, <span class="at">stat =</span> <span class="cn">FALSE</span>))</span>
+<span id="cb834-3"><a href="analysis-of-variance.html#cb834-3" tabindex="-1"></a><span class="fu">mean</span>(p_vals <span class="sc">&lt;</span> <span class="fl">0.05</span>)</span></code></pre></div>
 <pre><code>## [1] 0.661</code></pre>
-<div class="sourceCode" id="cb836"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb836-1"><a href="analysis-of-variance.html#cb836-1" aria-hidden="true" tabindex="-1"></a><span class="fu">mean</span>(p_vals <span class="sc">&lt;</span> <span class="fl">0.01</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb836"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb836-1"><a href="analysis-of-variance.html#cb836-1" tabindex="-1"></a><span class="fu">mean</span>(p_vals <span class="sc">&lt;</span> <span class="fl">0.01</span>)</span></code></pre></div>
 <pre><code>## [1] 0.384</code></pre>
-<div class="sourceCode" id="cb838"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb838-1"><a href="analysis-of-variance.html#cb838-1" aria-hidden="true" tabindex="-1"></a>p_vals <span class="ot">=</span> <span class="fu">replicate</span>(<span class="at">n =</span> <span class="dv">1000</span>, <span class="fu">sim_anova</span>(<span class="at">mu_a =</span> <span class="sc">-</span><span class="dv">1</span>, <span class="at">mu_b =</span> <span class="dv">0</span>, <span class="at">mu_c =</span> <span class="dv">0</span>, <span class="at">mu_d =</span> <span class="dv">1</span>,</span>
-<span id="cb838-2"><a href="analysis-of-variance.html#cb838-2" aria-hidden="true" tabindex="-1"></a>                                     <span class="at">sigma =</span> <span class="fl">2.0</span>, <span class="at">stat =</span> <span class="cn">FALSE</span>))</span>
-<span id="cb838-3"><a href="analysis-of-variance.html#cb838-3" aria-hidden="true" tabindex="-1"></a><span class="fu">mean</span>(p_vals <span class="sc">&lt;</span> <span class="fl">0.05</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb838"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb838-1"><a href="analysis-of-variance.html#cb838-1" tabindex="-1"></a>p_vals <span class="ot">=</span> <span class="fu">replicate</span>(<span class="at">n =</span> <span class="dv">1000</span>, <span class="fu">sim_anova</span>(<span class="at">mu_a =</span> <span class="sc">-</span><span class="dv">1</span>, <span class="at">mu_b =</span> <span class="dv">0</span>, <span class="at">mu_c =</span> <span class="dv">0</span>, <span class="at">mu_d =</span> <span class="dv">1</span>,</span>
+<span id="cb838-2"><a href="analysis-of-variance.html#cb838-2" tabindex="-1"></a>                                     <span class="at">sigma =</span> <span class="fl">2.0</span>, <span class="at">stat =</span> <span class="cn">FALSE</span>))</span>
+<span id="cb838-3"><a href="analysis-of-variance.html#cb838-3" tabindex="-1"></a><span class="fu">mean</span>(p_vals <span class="sc">&lt;</span> <span class="fl">0.05</span>)</span></code></pre></div>
 <pre><code>## [1] 0.425</code></pre>
-<div class="sourceCode" id="cb840"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb840-1"><a href="analysis-of-variance.html#cb840-1" aria-hidden="true" tabindex="-1"></a><span class="fu">mean</span>(p_vals <span class="sc">&lt;</span> <span class="fl">0.01</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb840"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb840-1"><a href="analysis-of-variance.html#cb840-1" tabindex="-1"></a><span class="fu">mean</span>(p_vals <span class="sc">&lt;</span> <span class="fl">0.01</span>)</span></code></pre></div>
 <pre><code>## [1] 0.19</code></pre>
-<div class="sourceCode" id="cb842"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb842-1"><a href="analysis-of-variance.html#cb842-1" aria-hidden="true" tabindex="-1"></a>p_vals <span class="ot">=</span> <span class="fu">replicate</span>(<span class="at">n =</span> <span class="dv">1000</span>, <span class="fu">sim_anova</span>(<span class="at">mu_a =</span> <span class="sc">-</span><span class="dv">2</span>, <span class="at">mu_b =</span> <span class="dv">0</span>, <span class="at">mu_c =</span> <span class="dv">0</span>, <span class="at">mu_d =</span> <span class="dv">2</span>,</span>
-<span id="cb842-2"><a href="analysis-of-variance.html#cb842-2" aria-hidden="true" tabindex="-1"></a>                                     <span class="at">sigma =</span> <span class="fl">2.0</span>, <span class="at">stat =</span> <span class="cn">FALSE</span>))</span>
-<span id="cb842-3"><a href="analysis-of-variance.html#cb842-3" aria-hidden="true" tabindex="-1"></a><span class="fu">mean</span>(p_vals <span class="sc">&lt;</span> <span class="fl">0.05</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb842"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb842-1"><a href="analysis-of-variance.html#cb842-1" tabindex="-1"></a>p_vals <span class="ot">=</span> <span class="fu">replicate</span>(<span class="at">n =</span> <span class="dv">1000</span>, <span class="fu">sim_anova</span>(<span class="at">mu_a =</span> <span class="sc">-</span><span class="dv">2</span>, <span class="at">mu_b =</span> <span class="dv">0</span>, <span class="at">mu_c =</span> <span class="dv">0</span>, <span class="at">mu_d =</span> <span class="dv">2</span>,</span>
+<span id="cb842-2"><a href="analysis-of-variance.html#cb842-2" tabindex="-1"></a>                                     <span class="at">sigma =</span> <span class="fl">2.0</span>, <span class="at">stat =</span> <span class="cn">FALSE</span>))</span>
+<span id="cb842-3"><a href="analysis-of-variance.html#cb842-3" tabindex="-1"></a><span class="fu">mean</span>(p_vals <span class="sc">&lt;</span> <span class="fl">0.05</span>)</span></code></pre></div>
 <pre><code>## [1] 0.956</code></pre>
-<div class="sourceCode" id="cb844"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb844-1"><a href="analysis-of-variance.html#cb844-1" aria-hidden="true" tabindex="-1"></a><span class="fu">mean</span>(p_vals <span class="sc">&lt;</span> <span class="fl">0.01</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb844"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb844-1"><a href="analysis-of-variance.html#cb844-1" tabindex="-1"></a><span class="fu">mean</span>(p_vals <span class="sc">&lt;</span> <span class="fl">0.01</span>)</span></code></pre></div>
 <pre><code>## [1] 0.837</code></pre>
 </div>
 </div>
 <div id="post-hoc-testing" class="section level2 hasAnchor" number="12.4">
 <h2><span class="header-section-number">12.4</span> Post Hoc Testing<a href="analysis-of-variance.html#post-hoc-testing" class="anchor-section" aria-label="Anchor link to header"></a></h2>
 <p>Suppose we reject the null hypothesis from the ANOVA test for equal means. That tells us that the means are different. But which means? All of them? Some of them? The obvious strategy is to test all possible comparisons of two means. We can do this easily in <code>R</code>.</p>
-<div class="sourceCode" id="cb846"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb846-1"><a href="analysis-of-variance.html#cb846-1" aria-hidden="true" tabindex="-1"></a><span class="fu">with</span>(coagulation, <span class="fu">pairwise.t.test</span>(coag, diet, <span class="at">p.adj =</span> <span class="st">&quot;none&quot;</span>))</span></code></pre></div>
+<div class="sourceCode" id="cb846"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb846-1"><a href="analysis-of-variance.html#cb846-1" tabindex="-1"></a><span class="fu">with</span>(coagulation, <span class="fu">pairwise.t.test</span>(coag, diet, <span class="at">p.adj =</span> <span class="st">&quot;none&quot;</span>))</span></code></pre></div>
 <pre><code>## 
 ##  Pairwise comparisons using t tests with pooled SD 
 ## 
@@ -864,7 +864,7 @@ <h2><span class="header-section-number">12.4</span> Post Hoc Testing<a href="ana
 ## D 1.00000 0.00086 2.3e-05
 ## 
 ## P value adjustment method: none</code></pre>
-<div class="sourceCode" id="cb848"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb848-1"><a href="analysis-of-variance.html#cb848-1" aria-hidden="true" tabindex="-1"></a><span class="co"># pairwise.t.test(coagulation$coag, coagulation$diet, p.adj = &quot;none&quot;)</span></span></code></pre></div>
+<div class="sourceCode" id="cb848"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb848-1"><a href="analysis-of-variance.html#cb848-1" tabindex="-1"></a><span class="co"># pairwise.t.test(coagulation$coag, coagulation$diet, p.adj = &quot;none&quot;)</span></span></code></pre></div>
 <p>Notice the <code>pairwise.t.test()</code> function does not have a <code>data</code> argument. To avoid using <code>attach()</code> or the <code>$</code> operator, we introduce the <code>with()</code> function. The commented line would perform the same operation.</p>
 <p>Also note that we are using the argument <code>p.adj = "none"</code>. What is this? An adjustment (in this case not an adjustment) to the p-value of each test. Why would we need to do this?</p>
 <p>The adjustment is an attempt to correct for the <a href="https://en.wikipedia.org/wiki/Multiple_comparisons_problem" target="_blank">multiple testing problem</a>. (See also: <a href="https://xkcd.com/882/" target="_blank">Relevant XKCD</a>. ) Imagine that you knew ahead of time that you were going to perform 100 <span class="math inline">\(t\)</span>-tests. Suppose you wish to do this with a false positive rate of <span class="math inline">\(\alpha = 0.05\)</span>. If we use this significance level for each test, for 100 tests, we then expect 5 false positives. That means, with 100 tests, we’re almost guaranteed to have at least one error.</p>
@@ -873,7 +873,7 @@ <h2><span class="header-section-number">12.4</span> Post Hoc Testing<a href="ana
 <p><span class="math display">\[
 \text{p-value-bonf} = \min(1, n_{tests} \cdot \text{p-value})
 \]</span></p>
-<div class="sourceCode" id="cb849"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb849-1"><a href="analysis-of-variance.html#cb849-1" aria-hidden="true" tabindex="-1"></a><span class="fu">with</span>(coagulation, <span class="fu">pairwise.t.test</span>(coag, diet, <span class="at">p.adj =</span> <span class="st">&quot;bonferroni&quot;</span>))</span></code></pre></div>
+<div class="sourceCode" id="cb849"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb849-1"><a href="analysis-of-variance.html#cb849-1" tabindex="-1"></a><span class="fu">with</span>(coagulation, <span class="fu">pairwise.t.test</span>(coag, diet, <span class="at">p.adj =</span> <span class="st">&quot;bonferroni&quot;</span>))</span></code></pre></div>
 <pre><code>## 
 ##  Pairwise comparisons using t tests with pooled SD 
 ## 
@@ -887,31 +887,31 @@ <h2><span class="header-section-number">12.4</span> Post Hoc Testing<a href="ana
 ## P value adjustment method: bonferroni</code></pre>
 <p>We see that these p-values are much higher than the unadjusted p-values, thus, we are less likely to reject each test. As a result, the FWER is 0.05, instead of an error rate of 0.05 for each test.</p>
 <p>We can simulate the 100 test scenarios to illustrate this point.</p>
-<div class="sourceCode" id="cb851"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb851-1"><a href="analysis-of-variance.html#cb851-1" aria-hidden="true" tabindex="-1"></a>get_p_val <span class="ot">=</span> <span class="cf">function</span>() {</span>
-<span id="cb851-2"><a href="analysis-of-variance.html#cb851-2" aria-hidden="true" tabindex="-1"></a>  </span>
-<span id="cb851-3"><a href="analysis-of-variance.html#cb851-3" aria-hidden="true" tabindex="-1"></a>  <span class="co"># create data for two groups, equal mean</span></span>
-<span id="cb851-4"><a href="analysis-of-variance.html#cb851-4" aria-hidden="true" tabindex="-1"></a>  y <span class="ot">=</span> <span class="fu">rnorm</span>(<span class="dv">20</span>, <span class="at">mean =</span> <span class="dv">0</span>, <span class="at">sd =</span> <span class="dv">1</span>)</span>
-<span id="cb851-5"><a href="analysis-of-variance.html#cb851-5" aria-hidden="true" tabindex="-1"></a>  g <span class="ot">=</span> <span class="fu">c</span>(<span class="fu">rep</span>(<span class="st">&quot;A&quot;</span>, <span class="dv">10</span>), <span class="fu">rep</span>(<span class="st">&quot;B&quot;</span>, <span class="dv">10</span>))</span>
-<span id="cb851-6"><a href="analysis-of-variance.html#cb851-6" aria-hidden="true" tabindex="-1"></a>  </span>
-<span id="cb851-7"><a href="analysis-of-variance.html#cb851-7" aria-hidden="true" tabindex="-1"></a>  <span class="co"># p-value of t-test when null is true</span></span>
-<span id="cb851-8"><a href="analysis-of-variance.html#cb851-8" aria-hidden="true" tabindex="-1"></a>  <span class="fu">glance</span>(<span class="fu">t.test</span>(y <span class="sc">~</span> g, <span class="at">var.equal =</span> <span class="cn">TRUE</span>))<span class="sc">$</span>p.value</span>
-<span id="cb851-9"><a href="analysis-of-variance.html#cb851-9" aria-hidden="true" tabindex="-1"></a>  </span>
-<span id="cb851-10"><a href="analysis-of-variance.html#cb851-10" aria-hidden="true" tabindex="-1"></a>}</span></code></pre></div>
-<div class="sourceCode" id="cb852"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb852-1"><a href="analysis-of-variance.html#cb852-1" aria-hidden="true" tabindex="-1"></a><span class="fu">set.seed</span>(<span class="dv">1337</span>)</span>
-<span id="cb852-2"><a href="analysis-of-variance.html#cb852-2" aria-hidden="true" tabindex="-1"></a></span>
-<span id="cb852-3"><a href="analysis-of-variance.html#cb852-3" aria-hidden="true" tabindex="-1"></a><span class="co"># FWER with 100 tests</span></span>
-<span id="cb852-4"><a href="analysis-of-variance.html#cb852-4" aria-hidden="true" tabindex="-1"></a><span class="co"># desired rate = 0.05</span></span>
-<span id="cb852-5"><a href="analysis-of-variance.html#cb852-5" aria-hidden="true" tabindex="-1"></a><span class="co"># no adjustment</span></span>
-<span id="cb852-6"><a href="analysis-of-variance.html#cb852-6" aria-hidden="true" tabindex="-1"></a><span class="fu">mean</span>(<span class="fu">replicate</span>(<span class="dv">1000</span>, <span class="fu">any</span>(<span class="fu">replicate</span>(<span class="dv">100</span>, <span class="fu">get_p_val</span>()) <span class="sc">&lt;</span> <span class="fl">0.05</span>)))</span></code></pre></div>
+<div class="sourceCode" id="cb851"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb851-1"><a href="analysis-of-variance.html#cb851-1" tabindex="-1"></a>get_p_val <span class="ot">=</span> <span class="cf">function</span>() {</span>
+<span id="cb851-2"><a href="analysis-of-variance.html#cb851-2" tabindex="-1"></a>  </span>
+<span id="cb851-3"><a href="analysis-of-variance.html#cb851-3" tabindex="-1"></a>  <span class="co"># create data for two groups, equal mean</span></span>
+<span id="cb851-4"><a href="analysis-of-variance.html#cb851-4" tabindex="-1"></a>  y <span class="ot">=</span> <span class="fu">rnorm</span>(<span class="dv">20</span>, <span class="at">mean =</span> <span class="dv">0</span>, <span class="at">sd =</span> <span class="dv">1</span>)</span>
+<span id="cb851-5"><a href="analysis-of-variance.html#cb851-5" tabindex="-1"></a>  g <span class="ot">=</span> <span class="fu">c</span>(<span class="fu">rep</span>(<span class="st">&quot;A&quot;</span>, <span class="dv">10</span>), <span class="fu">rep</span>(<span class="st">&quot;B&quot;</span>, <span class="dv">10</span>))</span>
+<span id="cb851-6"><a href="analysis-of-variance.html#cb851-6" tabindex="-1"></a>  </span>
+<span id="cb851-7"><a href="analysis-of-variance.html#cb851-7" tabindex="-1"></a>  <span class="co"># p-value of t-test when null is true</span></span>
+<span id="cb851-8"><a href="analysis-of-variance.html#cb851-8" tabindex="-1"></a>  <span class="fu">glance</span>(<span class="fu">t.test</span>(y <span class="sc">~</span> g, <span class="at">var.equal =</span> <span class="cn">TRUE</span>))<span class="sc">$</span>p.value</span>
+<span id="cb851-9"><a href="analysis-of-variance.html#cb851-9" tabindex="-1"></a>  </span>
+<span id="cb851-10"><a href="analysis-of-variance.html#cb851-10" tabindex="-1"></a>}</span></code></pre></div>
+<div class="sourceCode" id="cb852"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb852-1"><a href="analysis-of-variance.html#cb852-1" tabindex="-1"></a><span class="fu">set.seed</span>(<span class="dv">1337</span>)</span>
+<span id="cb852-2"><a href="analysis-of-variance.html#cb852-2" tabindex="-1"></a></span>
+<span id="cb852-3"><a href="analysis-of-variance.html#cb852-3" tabindex="-1"></a><span class="co"># FWER with 100 tests</span></span>
+<span id="cb852-4"><a href="analysis-of-variance.html#cb852-4" tabindex="-1"></a><span class="co"># desired rate = 0.05</span></span>
+<span id="cb852-5"><a href="analysis-of-variance.html#cb852-5" tabindex="-1"></a><span class="co"># no adjustment</span></span>
+<span id="cb852-6"><a href="analysis-of-variance.html#cb852-6" tabindex="-1"></a><span class="fu">mean</span>(<span class="fu">replicate</span>(<span class="dv">1000</span>, <span class="fu">any</span>(<span class="fu">replicate</span>(<span class="dv">100</span>, <span class="fu">get_p_val</span>()) <span class="sc">&lt;</span> <span class="fl">0.05</span>)))</span></code></pre></div>
 <pre><code>## [1] 0.994</code></pre>
-<div class="sourceCode" id="cb854"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb854-1"><a href="analysis-of-variance.html#cb854-1" aria-hidden="true" tabindex="-1"></a><span class="co"># FWER with 100 tests</span></span>
-<span id="cb854-2"><a href="analysis-of-variance.html#cb854-2" aria-hidden="true" tabindex="-1"></a><span class="co"># desired rate = 0.05</span></span>
-<span id="cb854-3"><a href="analysis-of-variance.html#cb854-3" aria-hidden="true" tabindex="-1"></a><span class="co"># bonferroni adjustment</span></span>
-<span id="cb854-4"><a href="analysis-of-variance.html#cb854-4" aria-hidden="true" tabindex="-1"></a><span class="fu">mean</span>(<span class="fu">replicate</span>(<span class="dv">1000</span>, <span class="fu">any</span>(<span class="fu">p.adjust</span>(<span class="fu">replicate</span>(<span class="dv">100</span>, <span class="fu">get_p_val</span>()), <span class="st">&quot;bonferroni&quot;</span>) <span class="sc">&lt;</span> <span class="fl">0.05</span>)))</span></code></pre></div>
+<div class="sourceCode" id="cb854"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb854-1"><a href="analysis-of-variance.html#cb854-1" tabindex="-1"></a><span class="co"># FWER with 100 tests</span></span>
+<span id="cb854-2"><a href="analysis-of-variance.html#cb854-2" tabindex="-1"></a><span class="co"># desired rate = 0.05</span></span>
+<span id="cb854-3"><a href="analysis-of-variance.html#cb854-3" tabindex="-1"></a><span class="co"># bonferroni adjustment</span></span>
+<span id="cb854-4"><a href="analysis-of-variance.html#cb854-4" tabindex="-1"></a><span class="fu">mean</span>(<span class="fu">replicate</span>(<span class="dv">1000</span>, <span class="fu">any</span>(<span class="fu">p.adjust</span>(<span class="fu">replicate</span>(<span class="dv">100</span>, <span class="fu">get_p_val</span>()), <span class="st">&quot;bonferroni&quot;</span>) <span class="sc">&lt;</span> <span class="fl">0.05</span>)))</span></code></pre></div>
 <pre><code>## [1] 0.058</code></pre>
 <p>For the specific case of testing all two-way mean differences after an ANOVA test, there are <a href="https://en.wikipedia.org/wiki/Post_hoc_analysis" target="_blank">a number of potential methods</a> for making an adjustment of this type. The pros and cons of the potential methods are beyond the scope of this course. We choose a method for its ease of use, and to a lesser extent, its developer.</p>
 <p>Tukey’s Honest Significance difference can be applied directly to an object which was created using <code>aov()</code>. It will adjust the p-values of the pairwise comparisons of the means to control the FWER, in this case, for 0.05. Notice it also gives confidence intervals for the difference of the means.</p>
-<div class="sourceCode" id="cb856"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb856-1"><a href="analysis-of-variance.html#cb856-1" aria-hidden="true" tabindex="-1"></a><span class="fu">TukeyHSD</span>(coag_aov, <span class="at">conf.level =</span> <span class="fl">0.95</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb856"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb856-1"><a href="analysis-of-variance.html#cb856-1" tabindex="-1"></a><span class="fu">TukeyHSD</span>(coag_aov, <span class="at">conf.level =</span> <span class="fl">0.95</span>)</span></code></pre></div>
 <pre><code>##   Tukey multiple comparisons of means
 ##     95% family-wise confidence level
 ## 
@@ -927,7 +927,7 @@ <h2><span class="header-section-number">12.4</span> Post Hoc Testing<a href="ana
 ## D-C   -7 -10.5770944 -3.422906 0.0001268</code></pre>
 <p>Based on these results, we see no difference between <code>A</code> and <code>D</code> as well as <code>B</code> and <code>C</code>. All other pairwise comparisons are significant. If you return to the original boxplot, these results should not be surprising.</p>
 <p>Also, nicely, we can easily produce a plot of these confidence intervals.</p>
-<div class="sourceCode" id="cb858"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb858-1"><a href="analysis-of-variance.html#cb858-1" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(<span class="fu">TukeyHSD</span>(coag_aov, <span class="at">conf.level =</span> <span class="fl">0.95</span>))</span></code></pre></div>
+<div class="sourceCode" id="cb858"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb858-1"><a href="analysis-of-variance.html#cb858-1" tabindex="-1"></a><span class="fu">plot</span>(<span class="fu">TukeyHSD</span>(coag_aov, <span class="at">conf.level =</span> <span class="fl">0.95</span>))</span></code></pre></div>
 <p><img src="anova_files/figure-html/unnamed-chunk-25-1.png" width="672" style="display: block; margin: auto;" /></p>
 <p>The creator of this method, <a href="https://en.wikipedia.org/wiki/John_Tukey" target="_blank">John Tukey</a>, is an important figure in the history of data science. He essentially <a href="https://projecteuclid.org/euclid.aoms/1177704711" target="_blank">predicted the rise of data science over 50 years ago</a>. For some retrospective thoughts on those 50 years, see <a href="http://courses.csail.mit.edu/18.337/2015/docs/50YearsDataScience.pdf" target="_blank">this paper from David Donoho</a>.</p>
 </div>
@@ -1084,27 +1084,27 @@ <h2><span class="header-section-number">12.5</span> Two-Way ANOVA<a href="analys
 </table>
 <p>The question then, is which of these models should we use if we have two factors? The most important question to consider is whether or not we should model the <strong>interaction</strong>. Is the effect of Factor A the <em>same</em> for all levels of Factor B? In the additive model, yes. In the interaction model, no. Both models would use a different mean for each group, but in a very specific way in both cases.</p>
 <p>Let’s discuss these comparisons by looking at some examples. We’ll first look at the <code>rats</code> data from the <code>faraway</code> package. There are two factors here: <code>poison</code> and <code>treat</code>. We use the <code>levels()</code> function to extract the levels of a factor variable.</p>
-<div class="sourceCode" id="cb859"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb859-1"><a href="analysis-of-variance.html#cb859-1" aria-hidden="true" tabindex="-1"></a><span class="fu">levels</span>(rats<span class="sc">$</span>poison)</span></code></pre></div>
+<div class="sourceCode" id="cb859"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb859-1"><a href="analysis-of-variance.html#cb859-1" tabindex="-1"></a><span class="fu">levels</span>(rats<span class="sc">$</span>poison)</span></code></pre></div>
 <pre><code>## [1] &quot;I&quot;   &quot;II&quot;  &quot;III&quot;</code></pre>
-<div class="sourceCode" id="cb861"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb861-1"><a href="analysis-of-variance.html#cb861-1" aria-hidden="true" tabindex="-1"></a><span class="fu">levels</span>(rats<span class="sc">$</span>treat)</span></code></pre></div>
+<div class="sourceCode" id="cb861"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb861-1"><a href="analysis-of-variance.html#cb861-1" tabindex="-1"></a><span class="fu">levels</span>(rats<span class="sc">$</span>treat)</span></code></pre></div>
 <pre><code>## [1] &quot;A&quot; &quot;B&quot; &quot;C&quot; &quot;D&quot;</code></pre>
 <p>Here, 48 rats were randomly assigned both one of three poisons and one of four possible treatments. The experimenters then measured their survival time in tens of hours. A total of 12 groups, each with 4 replicates.</p>
 <p>Before running any tests, we should first look at the data. We will create <strong>interaction plots</strong>, which will help us visualize the effect of one factor, as we move through the levels of another factor.</p>
-<div class="sourceCode" id="cb863"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb863-1"><a href="analysis-of-variance.html#cb863-1" aria-hidden="true" tabindex="-1"></a><span class="fu">par</span>(<span class="at">mfrow =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>))</span>
-<span id="cb863-2"><a href="analysis-of-variance.html#cb863-2" aria-hidden="true" tabindex="-1"></a><span class="fu">with</span>(rats, <span class="fu">interaction.plot</span>(poison, treat, time, <span class="at">lwd =</span> <span class="dv">2</span>, <span class="at">col =</span> <span class="dv">1</span><span class="sc">:</span><span class="dv">4</span>))</span>
-<span id="cb863-3"><a href="analysis-of-variance.html#cb863-3" aria-hidden="true" tabindex="-1"></a><span class="fu">with</span>(rats, <span class="fu">interaction.plot</span>(treat, poison, time, <span class="at">lwd =</span> <span class="dv">2</span>, <span class="at">col =</span> <span class="dv">1</span><span class="sc">:</span><span class="dv">3</span>))</span></code></pre></div>
+<div class="sourceCode" id="cb863"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb863-1"><a href="analysis-of-variance.html#cb863-1" tabindex="-1"></a><span class="fu">par</span>(<span class="at">mfrow =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>))</span>
+<span id="cb863-2"><a href="analysis-of-variance.html#cb863-2" tabindex="-1"></a><span class="fu">with</span>(rats, <span class="fu">interaction.plot</span>(poison, treat, time, <span class="at">lwd =</span> <span class="dv">2</span>, <span class="at">col =</span> <span class="dv">1</span><span class="sc">:</span><span class="dv">4</span>))</span>
+<span id="cb863-3"><a href="analysis-of-variance.html#cb863-3" tabindex="-1"></a><span class="fu">with</span>(rats, <span class="fu">interaction.plot</span>(treat, poison, time, <span class="at">lwd =</span> <span class="dv">2</span>, <span class="at">col =</span> <span class="dv">1</span><span class="sc">:</span><span class="dv">3</span>))</span></code></pre></div>
 <p><img src="anova_files/figure-html/unnamed-chunk-27-1.png" width="1440" style="display: block; margin: auto;" /></p>
 <p>If there is not interaction, thus an additive model, we would expect to see parallel lines. That would mean, when we change the level of one factor, there can be an effect on the response. However, the difference between the levels of the other factor should still be the same.</p>
 <p>The obvious indication of interaction would be lines that cross while heading in different directions. Here we don’t see that, but the lines aren’t strictly parallel, and there is some overlap on the right panel. However, is this interaction effect significant?</p>
 <p>Let’s fit each of the possible models, then investigate their estimates for each of the group means.</p>
-<div class="sourceCode" id="cb864"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb864-1"><a href="analysis-of-variance.html#cb864-1" aria-hidden="true" tabindex="-1"></a>rats_int   <span class="ot">=</span> <span class="fu">aov</span>(time <span class="sc">~</span> poison <span class="sc">*</span> treat, <span class="at">data =</span> rats) <span class="co"># interaction model</span></span>
-<span id="cb864-2"><a href="analysis-of-variance.html#cb864-2" aria-hidden="true" tabindex="-1"></a>rats_add   <span class="ot">=</span> <span class="fu">aov</span>(time <span class="sc">~</span> poison <span class="sc">+</span> treat, <span class="at">data =</span> rats) <span class="co"># additive model</span></span>
-<span id="cb864-3"><a href="analysis-of-variance.html#cb864-3" aria-hidden="true" tabindex="-1"></a>rats_pois  <span class="ot">=</span> <span class="fu">aov</span>(time <span class="sc">~</span> poison , <span class="at">data =</span> rats)        <span class="co"># single factor model   </span></span>
-<span id="cb864-4"><a href="analysis-of-variance.html#cb864-4" aria-hidden="true" tabindex="-1"></a>rats_treat <span class="ot">=</span> <span class="fu">aov</span>(time <span class="sc">~</span> treat, <span class="at">data =</span> rats)          <span class="co"># single factor model</span></span>
-<span id="cb864-5"><a href="analysis-of-variance.html#cb864-5" aria-hidden="true" tabindex="-1"></a>rats_null  <span class="ot">=</span> <span class="fu">aov</span>(time <span class="sc">~</span> <span class="dv">1</span>, <span class="at">data =</span> rats)              <span class="co"># null model</span></span></code></pre></div>
+<div class="sourceCode" id="cb864"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb864-1"><a href="analysis-of-variance.html#cb864-1" tabindex="-1"></a>rats_int   <span class="ot">=</span> <span class="fu">aov</span>(time <span class="sc">~</span> poison <span class="sc">*</span> treat, <span class="at">data =</span> rats) <span class="co"># interaction model</span></span>
+<span id="cb864-2"><a href="analysis-of-variance.html#cb864-2" tabindex="-1"></a>rats_add   <span class="ot">=</span> <span class="fu">aov</span>(time <span class="sc">~</span> poison <span class="sc">+</span> treat, <span class="at">data =</span> rats) <span class="co"># additive model</span></span>
+<span id="cb864-3"><a href="analysis-of-variance.html#cb864-3" tabindex="-1"></a>rats_pois  <span class="ot">=</span> <span class="fu">aov</span>(time <span class="sc">~</span> poison , <span class="at">data =</span> rats)        <span class="co"># single factor model   </span></span>
+<span id="cb864-4"><a href="analysis-of-variance.html#cb864-4" tabindex="-1"></a>rats_treat <span class="ot">=</span> <span class="fu">aov</span>(time <span class="sc">~</span> treat, <span class="at">data =</span> rats)          <span class="co"># single factor model</span></span>
+<span id="cb864-5"><a href="analysis-of-variance.html#cb864-5" tabindex="-1"></a>rats_null  <span class="ot">=</span> <span class="fu">aov</span>(time <span class="sc">~</span> <span class="dv">1</span>, <span class="at">data =</span> rats)              <span class="co"># null model</span></span></code></pre></div>
 <p>To get the estimates, we’ll create a table which we will predict on.</p>
-<div class="sourceCode" id="cb865"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb865-1"><a href="analysis-of-variance.html#cb865-1" aria-hidden="true" tabindex="-1"></a>rats_table <span class="ot">=</span> <span class="fu">expand.grid</span>(<span class="at">poison =</span> <span class="fu">unique</span>(rats<span class="sc">$</span>poison), <span class="at">treat =</span> <span class="fu">unique</span>(rats<span class="sc">$</span>treat))</span>
-<span id="cb865-2"><a href="analysis-of-variance.html#cb865-2" aria-hidden="true" tabindex="-1"></a>rats_table</span></code></pre></div>
+<div class="sourceCode" id="cb865"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb865-1"><a href="analysis-of-variance.html#cb865-1" tabindex="-1"></a>rats_table <span class="ot">=</span> <span class="fu">expand.grid</span>(<span class="at">poison =</span> <span class="fu">unique</span>(rats<span class="sc">$</span>poison), <span class="at">treat =</span> <span class="fu">unique</span>(rats<span class="sc">$</span>treat))</span>
+<span id="cb865-2"><a href="analysis-of-variance.html#cb865-2" tabindex="-1"></a>rats_table</span></code></pre></div>
 <pre><code>##    poison treat
 ## 1       I     A
 ## 2      II     A
@@ -1118,21 +1118,21 @@ <h2><span class="header-section-number">12.5</span> Two-Way ANOVA<a href="analys
 ## 10      I     D
 ## 11     II     D
 ## 12    III     D</code></pre>
-<div class="sourceCode" id="cb867"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb867-1"><a href="analysis-of-variance.html#cb867-1" aria-hidden="true" tabindex="-1"></a><span class="fu">matrix</span>(<span class="fu">paste0</span>(rats_table<span class="sc">$</span>poison, <span class="st">&quot;-&quot;</span>, rats_table<span class="sc">$</span>treat) , <span class="dv">4</span>, <span class="dv">3</span>, <span class="at">byrow =</span> <span class="cn">TRUE</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb867"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb867-1"><a href="analysis-of-variance.html#cb867-1" tabindex="-1"></a><span class="fu">matrix</span>(<span class="fu">paste0</span>(rats_table<span class="sc">$</span>poison, <span class="st">&quot;-&quot;</span>, rats_table<span class="sc">$</span>treat) , <span class="dv">4</span>, <span class="dv">3</span>, <span class="at">byrow =</span> <span class="cn">TRUE</span>)</span></code></pre></div>
 <pre><code>##      [,1]  [,2]   [,3]   
 ## [1,] &quot;I-A&quot; &quot;II-A&quot; &quot;III-A&quot;
 ## [2,] &quot;I-B&quot; &quot;II-B&quot; &quot;III-B&quot;
 ## [3,] &quot;I-C&quot; &quot;II-C&quot; &quot;III-C&quot;
 ## [4,] &quot;I-D&quot; &quot;II-D&quot; &quot;III-D&quot;</code></pre>
 <p>Since we’ll be repeating ourselves a number of times, we write a function to perform the prediction. Some housekeeping is done to keep the estimates in order, and provide row and column names. Above, we’ve shown where each of the estimates will be placed in the resulting matrix.</p>
-<div class="sourceCode" id="cb869"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb869-1"><a href="analysis-of-variance.html#cb869-1" aria-hidden="true" tabindex="-1"></a>get_est_means <span class="ot">=</span> <span class="cf">function</span>(model, table) {</span>
-<span id="cb869-2"><a href="analysis-of-variance.html#cb869-2" aria-hidden="true" tabindex="-1"></a>  mat <span class="ot">=</span> <span class="fu">matrix</span>(<span class="fu">predict</span>(model, table), <span class="at">nrow =</span> <span class="dv">4</span>, <span class="at">ncol =</span> <span class="dv">3</span>, <span class="at">byrow =</span> <span class="cn">TRUE</span>)</span>
-<span id="cb869-3"><a href="analysis-of-variance.html#cb869-3" aria-hidden="true" tabindex="-1"></a>  <span class="fu">colnames</span>(mat) <span class="ot">=</span> <span class="fu">c</span>(<span class="st">&quot;I&quot;</span>, <span class="st">&quot;II&quot;</span>, <span class="st">&quot;III&quot;</span>)</span>
-<span id="cb869-4"><a href="analysis-of-variance.html#cb869-4" aria-hidden="true" tabindex="-1"></a>  <span class="fu">rownames</span>(mat) <span class="ot">=</span> <span class="fu">c</span>(<span class="st">&quot;A&quot;</span>, <span class="st">&quot;B&quot;</span>, <span class="st">&quot;C&quot;</span>, <span class="st">&quot;D&quot;</span>)</span>
-<span id="cb869-5"><a href="analysis-of-variance.html#cb869-5" aria-hidden="true" tabindex="-1"></a>  mat</span>
-<span id="cb869-6"><a href="analysis-of-variance.html#cb869-6" aria-hidden="true" tabindex="-1"></a>}</span></code></pre></div>
+<div class="sourceCode" id="cb869"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb869-1"><a href="analysis-of-variance.html#cb869-1" tabindex="-1"></a>get_est_means <span class="ot">=</span> <span class="cf">function</span>(model, table) {</span>
+<span id="cb869-2"><a href="analysis-of-variance.html#cb869-2" tabindex="-1"></a>  mat <span class="ot">=</span> <span class="fu">matrix</span>(<span class="fu">predict</span>(model, table), <span class="at">nrow =</span> <span class="dv">4</span>, <span class="at">ncol =</span> <span class="dv">3</span>, <span class="at">byrow =</span> <span class="cn">TRUE</span>)</span>
+<span id="cb869-3"><a href="analysis-of-variance.html#cb869-3" tabindex="-1"></a>  <span class="fu">colnames</span>(mat) <span class="ot">=</span> <span class="fu">c</span>(<span class="st">&quot;I&quot;</span>, <span class="st">&quot;II&quot;</span>, <span class="st">&quot;III&quot;</span>)</span>
+<span id="cb869-4"><a href="analysis-of-variance.html#cb869-4" tabindex="-1"></a>  <span class="fu">rownames</span>(mat) <span class="ot">=</span> <span class="fu">c</span>(<span class="st">&quot;A&quot;</span>, <span class="st">&quot;B&quot;</span>, <span class="st">&quot;C&quot;</span>, <span class="st">&quot;D&quot;</span>)</span>
+<span id="cb869-5"><a href="analysis-of-variance.html#cb869-5" tabindex="-1"></a>  mat</span>
+<span id="cb869-6"><a href="analysis-of-variance.html#cb869-6" tabindex="-1"></a>}</span></code></pre></div>
 <p>First, we obtain the estimates from the <strong>interaction</strong> model. Note that each cell has a different value.</p>
-<div class="sourceCode" id="cb870"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb870-1"><a href="analysis-of-variance.html#cb870-1" aria-hidden="true" tabindex="-1"></a>knitr<span class="sc">::</span><span class="fu">kable</span>(<span class="fu">get_est_means</span>(<span class="at">model =</span> rats_int, <span class="at">table =</span> rats_table))</span></code></pre></div>
+<div class="sourceCode" id="cb870"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb870-1"><a href="analysis-of-variance.html#cb870-1" tabindex="-1"></a>knitr<span class="sc">::</span><span class="fu">kable</span>(<span class="fu">get_est_means</span>(<span class="at">model =</span> rats_int, <span class="at">table =</span> rats_table))</span></code></pre></div>
 <table>
 <thead>
 <tr class="header">
@@ -1170,7 +1170,7 @@ <h2><span class="header-section-number">12.5</span> Two-Way ANOVA<a href="analys
 </tbody>
 </table>
 <p>Next, we obtain the estimates from the <strong>additive</strong> model. Again, each cell has a different value. We also see that these estimates are somewhat close to those from the interaction model.</p>
-<div class="sourceCode" id="cb871"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb871-1"><a href="analysis-of-variance.html#cb871-1" aria-hidden="true" tabindex="-1"></a>knitr<span class="sc">::</span><span class="fu">kable</span>(<span class="fu">get_est_means</span>(<span class="at">model =</span> rats_add, <span class="at">table =</span> rats_table))</span></code></pre></div>
+<div class="sourceCode" id="cb871"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb871-1"><a href="analysis-of-variance.html#cb871-1" tabindex="-1"></a>knitr<span class="sc">::</span><span class="fu">kable</span>(<span class="fu">get_est_means</span>(<span class="at">model =</span> rats_add, <span class="at">table =</span> rats_table))</span></code></pre></div>
 <table>
 <thead>
 <tr class="header">
@@ -1208,17 +1208,17 @@ <h2><span class="header-section-number">12.5</span> Two-Way ANOVA<a href="analys
 </tbody>
 </table>
 <p>To understand the difference, let’s consider the effect of the treatments.</p>
-<div class="sourceCode" id="cb872"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb872-1"><a href="analysis-of-variance.html#cb872-1" aria-hidden="true" tabindex="-1"></a>additive_means <span class="ot">=</span> <span class="fu">get_est_means</span>(<span class="at">model =</span> rats_add, <span class="at">table =</span> rats_table)</span>
-<span id="cb872-2"><a href="analysis-of-variance.html#cb872-2" aria-hidden="true" tabindex="-1"></a>additive_means[<span class="st">&quot;A&quot;</span>,] <span class="sc">-</span> additive_means[<span class="st">&quot;B&quot;</span>,]</span></code></pre></div>
+<div class="sourceCode" id="cb872"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb872-1"><a href="analysis-of-variance.html#cb872-1" tabindex="-1"></a>additive_means <span class="ot">=</span> <span class="fu">get_est_means</span>(<span class="at">model =</span> rats_add, <span class="at">table =</span> rats_table)</span>
+<span id="cb872-2"><a href="analysis-of-variance.html#cb872-2" tabindex="-1"></a>additive_means[<span class="st">&quot;A&quot;</span>,] <span class="sc">-</span> additive_means[<span class="st">&quot;B&quot;</span>,]</span></code></pre></div>
 <pre><code>##       I      II     III 
 ## -0.3625 -0.3625 -0.3625</code></pre>
-<div class="sourceCode" id="cb874"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb874-1"><a href="analysis-of-variance.html#cb874-1" aria-hidden="true" tabindex="-1"></a>interaction_means <span class="ot">=</span> <span class="fu">get_est_means</span>(<span class="at">model =</span> rats_int, <span class="at">table =</span> rats_table)</span>
-<span id="cb874-2"><a href="analysis-of-variance.html#cb874-2" aria-hidden="true" tabindex="-1"></a>interaction_means[<span class="st">&quot;A&quot;</span>,] <span class="sc">-</span> interaction_means[<span class="st">&quot;B&quot;</span>,]</span></code></pre></div>
+<div class="sourceCode" id="cb874"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb874-1"><a href="analysis-of-variance.html#cb874-1" tabindex="-1"></a>interaction_means <span class="ot">=</span> <span class="fu">get_est_means</span>(<span class="at">model =</span> rats_int, <span class="at">table =</span> rats_table)</span>
+<span id="cb874-2"><a href="analysis-of-variance.html#cb874-2" tabindex="-1"></a>interaction_means[<span class="st">&quot;A&quot;</span>,] <span class="sc">-</span> interaction_means[<span class="st">&quot;B&quot;</span>,]</span></code></pre></div>
 <pre><code>##       I      II     III 
 ## -0.4675 -0.4950 -0.1250</code></pre>
 <p>This is the key difference between the interaction and additive models. The difference between the effect of treatments <code>A</code> and <code>B</code> is the <strong>same</strong> for each poison in the additive model. They are <strong>different</strong> in the interaction model.</p>
 <p>The remaining three models are much simpler, having either only row or only column effects. Or no effects in the case of the null model.</p>
-<div class="sourceCode" id="cb876"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb876-1"><a href="analysis-of-variance.html#cb876-1" aria-hidden="true" tabindex="-1"></a>knitr<span class="sc">::</span><span class="fu">kable</span>(<span class="fu">get_est_means</span>(<span class="at">model =</span> rats_pois, <span class="at">table =</span> rats_table))</span></code></pre></div>
+<div class="sourceCode" id="cb876"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb876-1"><a href="analysis-of-variance.html#cb876-1" tabindex="-1"></a>knitr<span class="sc">::</span><span class="fu">kable</span>(<span class="fu">get_est_means</span>(<span class="at">model =</span> rats_pois, <span class="at">table =</span> rats_table))</span></code></pre></div>
 <table>
 <thead>
 <tr class="header">
@@ -1255,7 +1255,7 @@ <h2><span class="header-section-number">12.5</span> Two-Way ANOVA<a href="analys
 </tr>
 </tbody>
 </table>
-<div class="sourceCode" id="cb877"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb877-1"><a href="analysis-of-variance.html#cb877-1" aria-hidden="true" tabindex="-1"></a>knitr<span class="sc">::</span><span class="fu">kable</span>(<span class="fu">get_est_means</span>(<span class="at">model =</span> rats_treat, <span class="at">table =</span> rats_table))</span></code></pre></div>
+<div class="sourceCode" id="cb877"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb877-1"><a href="analysis-of-variance.html#cb877-1" tabindex="-1"></a>knitr<span class="sc">::</span><span class="fu">kable</span>(<span class="fu">get_est_means</span>(<span class="at">model =</span> rats_treat, <span class="at">table =</span> rats_table))</span></code></pre></div>
 <table>
 <thead>
 <tr class="header">
@@ -1292,7 +1292,7 @@ <h2><span class="header-section-number">12.5</span> Two-Way ANOVA<a href="analys
 </tr>
 </tbody>
 </table>
-<div class="sourceCode" id="cb878"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb878-1"><a href="analysis-of-variance.html#cb878-1" aria-hidden="true" tabindex="-1"></a>knitr<span class="sc">::</span><span class="fu">kable</span>(<span class="fu">get_est_means</span>(<span class="at">model =</span> rats_null, <span class="at">table =</span> rats_table))</span></code></pre></div>
+<div class="sourceCode" id="cb878"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb878-1"><a href="analysis-of-variance.html#cb878-1" tabindex="-1"></a>knitr<span class="sc">::</span><span class="fu">kable</span>(<span class="fu">get_est_means</span>(<span class="at">model =</span> rats_null, <span class="at">table =</span> rats_table))</span></code></pre></div>
 <table>
 <thead>
 <tr class="header">
@@ -1413,11 +1413,11 @@ <h2><span class="header-section-number">12.5</span> Two-Way ANOVA<a href="analys
 </ul>
 <p>We reject the null when the <span class="math inline">\(F\)</span> statistic is large. Under the null hypothesis, the distribution of the test statistic is <span class="math inline">\(F\)</span> with degrees of freedom <span class="math inline">\(I - 1\)</span> and <span class="math inline">\(IJ(K - 1)\)</span>.</p>
 <p>These tests should be performed according to the model <strong>hierarchy</strong>. First consider the test of interaction. If it is significant, we select the interaction model and perform no further testing. If interaction is not significant, we then consider the necessity of the individual factors of the additive model.</p>
-<div class="figure">
-<img src="images/hierarchy.png" alt="" />
-<p class="caption">Model Hierarchy</p>
+<div class="float">
+<img src="images/hierarchy.png" alt="Model Hierarchy" />
+<div class="figcaption">Model Hierarchy</div>
 </div>
-<div class="sourceCode" id="cb879"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb879-1"><a href="analysis-of-variance.html#cb879-1" aria-hidden="true" tabindex="-1"></a><span class="fu">summary</span>(<span class="fu">aov</span>(time <span class="sc">~</span> poison <span class="sc">*</span> treat, <span class="at">data =</span> rats))</span></code></pre></div>
+<div class="sourceCode" id="cb879"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb879-1"><a href="analysis-of-variance.html#cb879-1" tabindex="-1"></a><span class="fu">summary</span>(<span class="fu">aov</span>(time <span class="sc">~</span> poison <span class="sc">*</span> treat, <span class="at">data =</span> rats))</span></code></pre></div>
 <pre><code>##              Df Sum Sq Mean Sq F value   Pr(&gt;F)    
 ## poison        2 1.0330  0.5165  23.222 3.33e-07 ***
 ## treat         3 0.9212  0.3071  13.806 3.78e-06 ***
@@ -1430,7 +1430,7 @@ <h2><span class="header-section-number">12.5</span> Two-Way ANOVA<a href="analys
 <!-- - TODO: correct if int is not significant? -->
 <!-- - TODO: here we pick the full additive model -->
 <p>Within the additive model, we could do further testing about the main effects.</p>
-<div class="sourceCode" id="cb881"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb881-1"><a href="analysis-of-variance.html#cb881-1" aria-hidden="true" tabindex="-1"></a><span class="fu">TukeyHSD</span>(<span class="fu">aov</span>(time <span class="sc">~</span> poison <span class="sc">+</span> treat, <span class="at">data =</span> rats))</span></code></pre></div>
+<div class="sourceCode" id="cb881"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb881-1"><a href="analysis-of-variance.html#cb881-1" tabindex="-1"></a><span class="fu">TukeyHSD</span>(<span class="fu">aov</span>(time <span class="sc">~</span> poison <span class="sc">+</span> treat, <span class="at">data =</span> rats))</span></code></pre></div>
 <pre><code>##   Tukey multiple comparisons of means
 ##     95% family-wise confidence level
 ## 
@@ -1451,12 +1451,12 @@ <h2><span class="header-section-number">12.5</span> Two-Way ANOVA<a href="analys
 ## D-B -0.14250000 -0.31523865  0.03023865 0.1380432
 ## D-C  0.14166667 -0.03107198  0.31440532 0.1416151</code></pre>
 <p>For an example <strong>with</strong> interaction, we investigate the <code>warpbreaks</code> dataset, a default dataset in <code>R</code>.</p>
-<div class="sourceCode" id="cb883"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb883-1"><a href="analysis-of-variance.html#cb883-1" aria-hidden="true" tabindex="-1"></a><span class="fu">par</span>(<span class="at">mfrow =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>))</span>
-<span id="cb883-2"><a href="analysis-of-variance.html#cb883-2" aria-hidden="true" tabindex="-1"></a><span class="fu">with</span>(warpbreaks, <span class="fu">interaction.plot</span>(wool, tension, breaks, <span class="at">lwd =</span> <span class="dv">2</span>, <span class="at">col =</span> <span class="dv">2</span><span class="sc">:</span><span class="dv">4</span>))</span>
-<span id="cb883-3"><a href="analysis-of-variance.html#cb883-3" aria-hidden="true" tabindex="-1"></a><span class="fu">with</span>(warpbreaks, <span class="fu">interaction.plot</span>(tension, wool, breaks, <span class="at">lwd =</span> <span class="dv">2</span>, <span class="at">col =</span> <span class="dv">2</span><span class="sc">:</span><span class="dv">3</span>))</span></code></pre></div>
+<div class="sourceCode" id="cb883"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb883-1"><a href="analysis-of-variance.html#cb883-1" tabindex="-1"></a><span class="fu">par</span>(<span class="at">mfrow =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>))</span>
+<span id="cb883-2"><a href="analysis-of-variance.html#cb883-2" tabindex="-1"></a><span class="fu">with</span>(warpbreaks, <span class="fu">interaction.plot</span>(wool, tension, breaks, <span class="at">lwd =</span> <span class="dv">2</span>, <span class="at">col =</span> <span class="dv">2</span><span class="sc">:</span><span class="dv">4</span>))</span>
+<span id="cb883-3"><a href="analysis-of-variance.html#cb883-3" tabindex="-1"></a><span class="fu">with</span>(warpbreaks, <span class="fu">interaction.plot</span>(tension, wool, breaks, <span class="at">lwd =</span> <span class="dv">2</span>, <span class="at">col =</span> <span class="dv">2</span><span class="sc">:</span><span class="dv">3</span>))</span></code></pre></div>
 <p><img src="anova_files/figure-html/unnamed-chunk-42-1.png" width="1440" style="display: block; margin: auto;" /></p>
 <p>Either plot makes it rather clear that the <code>wool</code> and <code>tensions</code> factors interact.</p>
-<div class="sourceCode" id="cb884"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb884-1"><a href="analysis-of-variance.html#cb884-1" aria-hidden="true" tabindex="-1"></a><span class="fu">summary</span>(<span class="fu">aov</span>(breaks <span class="sc">~</span> wool <span class="sc">*</span> tension, <span class="at">data =</span> warpbreaks))</span></code></pre></div>
+<div class="sourceCode" id="cb884"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb884-1"><a href="analysis-of-variance.html#cb884-1" tabindex="-1"></a><span class="fu">summary</span>(<span class="fu">aov</span>(breaks <span class="sc">~</span> wool <span class="sc">*</span> tension, <span class="at">data =</span> warpbreaks))</span></code></pre></div>
 <pre><code>##              Df Sum Sq Mean Sq F value   Pr(&gt;F)    
 ## wool          1    451   450.7   3.765 0.058213 .  
 ## tension       2   2034  1017.1   8.498 0.000693 ***
@@ -1473,7 +1473,7 @@ <h2><span class="header-section-number">12.6</span> <code>R</code> Markdown<a hr
 <ul>
 <li><a href="anova.Rmd" target="_blank"><code>anova.Rmd</code></a></li>
 </ul>
-<p>The file was created using <code>R</code> version <code>4.3.2</code>.</p>
+<p>The file was created using <code>R</code> version <code>4.4.1</code>.</p>
 
 </div>
 </div>
diff --git a/anova.md b/anova.md
index be19487..6b8ac1e 100644
--- a/anova.md
+++ b/anova.md
@@ -84,7 +84,7 @@ For the stated model and assuming the null hypothesis is true, the $t$ test stat
 As an example, suppose we are interested in the effect of [melatonin](https://en.wikipedia.org/wiki/Melatonin){target="_blank"} on sleep duration. A researcher obtains a random sample of 20 adult males. Of these subjects, 10 are randomly chosen for the control group, which will receive a placebo. The remaining 10 will be given 5mg of melatonin before bed. The sleep duration in hours of each subject is then measured. The researcher chooses a significance level of $\alpha = 0.10$. Was sleep duration affected by the melatonin?
 
 
-```r
+``` r
 melatonin
 ```
 
@@ -121,7 +121,7 @@ H_0: \mu_C = \mu_T \quad \text{vs} \quad H_1: \mu_C \neq \mu_T
 To do so in `R`, we use the `t.test()` function, with the `var.equal` argument set to `TRUE`.
 
 
-```r
+``` r
 t.test(sleep ~ group, data = melatonin, var.equal = TRUE)
 ```
 
@@ -142,7 +142,7 @@ t.test(sleep ~ group, data = melatonin, var.equal = TRUE)
 At a significance level of $\alpha = 0.10$, we reject the null hypothesis. It seems that the melatonin had a **statistically significant** effect. Be aware that statistical significance is not always the same as scientific or **practical significance**. To determine practical significance, we need to investigate the **effect size** in the context of the situation. Here the effect size is the difference of the sample means.
 
 
-```r
+``` r
 t.test(sleep ~ group, data = melatonin, var.equal = TRUE)$estimate
 ```
 
@@ -156,7 +156,7 @@ Here we see that the subjects in the melatonin group sleep an average of about 1
 With a big enough sample size, we could make an effect size of say, four minutes statistically significant. Is it worth taking a pill every night to get an extra four minutes of sleep? (Probably not.)
 
 
-```r
+``` r
 boxplot(sleep ~ group, data = melatonin, col = 5:6)
 ```
 
@@ -332,7 +332,7 @@ H_0: \mu_A = \mu_B = \mu_C = \mu_D
 where, for example, $\mu_A$ is the mean blood coagulation time for an animal that ate diet `A`.
 
 
-```r
+``` r
 library(faraway)
 names(coagulation)
 ```
@@ -341,7 +341,7 @@ names(coagulation)
 ## [1] "coag" "diet"
 ```
 
-```r
+``` r
 plot(coag ~ diet, data = coagulation, col = 2:5)
 ```
 
@@ -352,7 +352,7 @@ plot(coag ~ diet, data = coagulation, col = 2:5)
 We first load the data and create the relevant boxplot. The plot alone suggests a difference of means. The `aov()` function is used to obtain the relevant sums of squares. Using the `summary()` function on the output from `aov()` creates the desired ANOVA table (without the unneeded row for total).
 
 
-```r
+``` r
 coag_aov = aov(coag ~ diet, data = coagulation)
 coag_aov
 ```
@@ -370,7 +370,7 @@ coag_aov
 ## Estimated effects may be unbalanced
 ```
 
-```r
+``` r
 summary(coag_aov)
 ```
 
@@ -385,7 +385,7 @@ summary(coag_aov)
 Were we to run this experiment, we would have pre-specified a significance level. However, notice that the p-value of this test is incredibly low, so using any reasonable significance level we would reject the null hypothesis. Thus we believe the diets had an effect on blood coagulation time.
 
 
-```r
+``` r
 diets = data.frame(diet = unique(coagulation$diet))
 data.frame(diets, coag = predict(coag_aov, diets))
 ```
@@ -405,7 +405,7 @@ Here, we've created a dataframe with a row for each diet. By predicting on this
 When performing ANOVA in `R`, be sure the grouping variable is a factor variable. If it is not, your result might not be ANOVA, but instead a linear regression with the predictor variable considered numeric.
 
 
-```r
+``` r
 set.seed(42)
 response = rnorm(15)
 group    = c(rep(1, 5), rep(2, 5), rep(3, 5))
@@ -419,7 +419,7 @@ summary(aov(response ~ group, data = bad)) # wrong DF!
 ## Residuals   13 14.698  1.1306
 ```
 
-```r
+``` r
 good = data.frame(response, group = as.factor(group))
 summary(aov(response ~ group, data = good))
 ```
@@ -430,7 +430,7 @@ summary(aov(response ~ group, data = good))
 ## Residuals   12 14.484  1.2070
 ```
 
-```r
+``` r
 is.factor(bad$group)  # 1, 2, and 3 are numbers.
 ```
 
@@ -438,7 +438,7 @@ is.factor(bad$group)  # 1, 2, and 3 are numbers.
 ## [1] FALSE
 ```
 
-```r
+``` r
 is.factor(good$group) # 1, 2, and 3 are labels.
 ```
 
@@ -451,7 +451,7 @@ is.factor(good$group) # 1, 2, and 3 are labels.
 Here we verify the distribution of the test statistic under the null hypothesis. We simulate from a null model (equal variance) to obtain an empirical distribution of the $F$ statistic. We add the curve for the expected distribution.
 
 
-```r
+``` r
 library(broom)
 
 sim_anova = function(n = 10, mu_a = 0, mu_b = 0, mu_c = 0, mu_d = 0, sigma = 1, stat = TRUE) {
@@ -481,7 +481,7 @@ sim_anova = function(n = 10, mu_a = 0, mu_b = 0, mu_c = 0, mu_d = 0, sigma = 1,
 ```
 
 
-```r
+``` r
 f_stats = replicate(n = 5000, sim_anova(stat = TRUE))
 ```
 
@@ -491,7 +491,7 @@ f_stats = replicate(n = 5000, sim_anova(stat = TRUE))
 
 
 
-```r
+``` r
 hist(f_stats, breaks = 100, prob = TRUE, border = "dodgerblue", main = "Empirical Distribution of F")
 curve(df(x, df1 = 4 - 1, df2 = 40 - 4), col = "darkorange", add = TRUE, lwd = 2)
 ```
@@ -526,7 +526,7 @@ A number of things can affect the power of a test:
 The following simulations look at the effect of significance level, effect size, and noise level on the power of an ANOVA $F$-test. Homework will look into sample size and balance.
 
 
-```r
+``` r
 p_vals = replicate(n = 1000, sim_anova(mu_a = -1, mu_b = 0, mu_c = 0, mu_d = 1,
                                      sigma = 1.5, stat = FALSE))
 mean(p_vals < 0.05)
@@ -536,7 +536,7 @@ mean(p_vals < 0.05)
 ## [1] 0.661
 ```
 
-```r
+``` r
 mean(p_vals < 0.01)
 ```
 
@@ -545,7 +545,7 @@ mean(p_vals < 0.01)
 ```
 
 
-```r
+``` r
 p_vals = replicate(n = 1000, sim_anova(mu_a = -1, mu_b = 0, mu_c = 0, mu_d = 1,
                                      sigma = 2.0, stat = FALSE))
 mean(p_vals < 0.05)
@@ -555,7 +555,7 @@ mean(p_vals < 0.05)
 ## [1] 0.425
 ```
 
-```r
+``` r
 mean(p_vals < 0.01)
 ```
 
@@ -564,7 +564,7 @@ mean(p_vals < 0.01)
 ```
 
 
-```r
+``` r
 p_vals = replicate(n = 1000, sim_anova(mu_a = -2, mu_b = 0, mu_c = 0, mu_d = 2,
                                      sigma = 2.0, stat = FALSE))
 mean(p_vals < 0.05)
@@ -574,7 +574,7 @@ mean(p_vals < 0.05)
 ## [1] 0.956
 ```
 
-```r
+``` r
 mean(p_vals < 0.01)
 ```
 
@@ -587,7 +587,7 @@ mean(p_vals < 0.01)
 Suppose we reject the null hypothesis from the ANOVA test for equal means. That tells us that the means are different. But which means? All of them? Some of them? The obvious strategy is to test all possible comparisons of two means. We can do this easily in `R`.
 
 
-```r
+``` r
 with(coagulation, pairwise.t.test(coag, diet, p.adj = "none"))
 ```
 
@@ -605,7 +605,7 @@ with(coagulation, pairwise.t.test(coag, diet, p.adj = "none"))
 ## P value adjustment method: none
 ```
 
-```r
+``` r
 # pairwise.t.test(coagulation$coag, coagulation$diet, p.adj = "none")
 ```
 
@@ -624,7 +624,7 @@ With this in mind, one of the simplest adjustments we can make, is to increase t
 \]
 
 
-```r
+``` r
 with(coagulation, pairwise.t.test(coag, diet, p.adj = "bonferroni"))
 ```
 
@@ -647,7 +647,7 @@ We see that these p-values are much higher than the unadjusted p-values, thus, w
 We can simulate the 100 test scenarios to illustrate this point.
 
 
-```r
+``` r
 get_p_val = function() {
   
   # create data for two groups, equal mean
@@ -663,7 +663,7 @@ get_p_val = function() {
 
 
 
-```r
+``` r
 set.seed(1337)
 
 # FWER with 100 tests
@@ -678,7 +678,7 @@ mean(replicate(1000, any(replicate(100, get_p_val()) < 0.05)))
 ```
 
 
-```r
+``` r
 # FWER with 100 tests
 # desired rate = 0.05
 # bonferroni adjustment
@@ -695,7 +695,7 @@ For the specific case of testing all two-way mean differences after an ANOVA tes
 Tukey's Honest Significance difference can be applied directly to an object which was created using `aov()`. It will adjust the p-values of the pairwise comparisons of the means to control the FWER, in this case, for 0.05. Notice it also gives confidence intervals for the difference of the means.
 
 
-```r
+``` r
 TukeyHSD(coag_aov, conf.level = 0.95)
 ```
 
@@ -720,7 +720,7 @@ Based on these results, we see no difference between `A` and `D` as well as `B`
 Also, nicely, we can easily produce a plot of these confidence intervals.
 
 
-```r
+``` r
 plot(TukeyHSD(coag_aov, conf.level = 0.95))
 ```
 
@@ -811,7 +811,7 @@ The question then, is which of these models should we use if we have two factors
 Let's discuss these comparisons by looking at some examples. We'll first look at the `rats` data from the `faraway` package. There are two factors here: `poison` and `treat`. We use the `levels()` function to extract the levels of a factor variable.
 
 
-```r
+``` r
 levels(rats$poison)
 ```
 
@@ -819,7 +819,7 @@ levels(rats$poison)
 ## [1] "I"   "II"  "III"
 ```
 
-```r
+``` r
 levels(rats$treat)
 ```
 
@@ -832,7 +832,7 @@ Here, 48 rats were randomly assigned both one of three poisons and one of four p
 Before running any tests, we should first look at the data. We will create **interaction plots**, which will help us visualize the effect of one factor, as we move through the levels of another factor.
 
 
-```r
+``` r
 par(mfrow = c(1, 2))
 with(rats, interaction.plot(poison, treat, time, lwd = 2, col = 1:4))
 with(rats, interaction.plot(treat, poison, time, lwd = 2, col = 1:3))
@@ -849,7 +849,7 @@ The obvious indication of interaction would be lines that cross while heading in
 Let's fit each of the possible models, then investigate their estimates for each of the group means.
 
 
-```r
+``` r
 rats_int   = aov(time ~ poison * treat, data = rats) # interaction model
 rats_add   = aov(time ~ poison + treat, data = rats) # additive model
 rats_pois  = aov(time ~ poison , data = rats)        # single factor model   
@@ -860,7 +860,7 @@ rats_null  = aov(time ~ 1, data = rats)              # null model
 To get the estimates, we'll create a table which we will predict on. 
 
 
-```r
+``` r
 rats_table = expand.grid(poison = unique(rats$poison), treat = unique(rats$treat))
 rats_table
 ```
@@ -881,7 +881,7 @@ rats_table
 ## 12    III     D
 ```
 
-```r
+``` r
 matrix(paste0(rats_table$poison, "-", rats_table$treat) , 4, 3, byrow = TRUE)
 ```
 
@@ -896,7 +896,7 @@ matrix(paste0(rats_table$poison, "-", rats_table$treat) , 4, 3, byrow = TRUE)
 Since we'll be repeating ourselves a number of times, we write a function to perform the prediction. Some housekeeping is done to keep the estimates in order, and provide row and column names. Above, we've shown where each of the estimates will be placed in the resulting matrix.
 
 
-```r
+``` r
 get_est_means = function(model, table) {
   mat = matrix(predict(model, table), nrow = 4, ncol = 3, byrow = TRUE)
   colnames(mat) = c("I", "II", "III")
@@ -908,7 +908,7 @@ get_est_means = function(model, table) {
 First, we obtain the estimates from the **interaction** model. Note that each cell has a different value. 
 
 
-```r
+``` r
 knitr::kable(get_est_means(model = rats_int, table = rats_table))
 ```
 
@@ -932,7 +932,7 @@ D & 0.6100 & 0.6675 & 0.325\\
 Next, we obtain the estimates from the **additive** model. Again, each cell has a different value. We also see that these estimates are somewhat close to those from the interaction model.
 
 
-```r
+``` r
 knitr::kable(get_est_means(model = rats_add, table = rats_table))
 ```
 
@@ -954,7 +954,7 @@ D & 0.6722917 & 0.5991667 & 0.3310417\\
 To understand the difference, let's consider the effect of the treatments.
 
 
-```r
+``` r
 additive_means = get_est_means(model = rats_add, table = rats_table)
 additive_means["A",] - additive_means["B",]
 ```
@@ -965,7 +965,7 @@ additive_means["A",] - additive_means["B",]
 ```
 
 
-```r
+``` r
 interaction_means = get_est_means(model = rats_int, table = rats_table)
 interaction_means["A",] - interaction_means["B",]
 ```
@@ -980,7 +980,7 @@ This is the key difference between the interaction and additive models. The diff
 The remaining three models are much simpler, having either only row or only column effects. Or no effects in the case of the null model.
 
 
-```r
+``` r
 knitr::kable(get_est_means(model = rats_pois, table = rats_table))
 ```
 
@@ -1000,7 +1000,7 @@ D & 0.6175 & 0.544375 & 0.27625\\
 \end{tabular}
 
 
-```r
+``` r
 knitr::kable(get_est_means(model = rats_treat, table = rats_table))
 ```
 
@@ -1020,7 +1020,7 @@ D & 0.5341667 & 0.5341667 & 0.5341667\\
 \end{tabular}
 
 
-```r
+``` r
 knitr::kable(get_est_means(model = rats_null, table = rats_table))
 ```
 
@@ -1087,7 +1087,7 @@ These tests should be performed according to the model **hierarchy**. First cons
 ![Model Hierarchy](images/hierarchy.png)
 
 
-```r
+``` r
 summary(aov(time ~ poison * treat, data = rats))
 ```
 
@@ -1110,7 +1110,7 @@ Using a significance level of $\alpha = 0.05$, we see that the interaction is no
 Within the additive model, we could do further testing about the main effects.
 
 
-```r
+``` r
 TukeyHSD(aov(time ~ poison + treat, data = rats))
 ```
 
@@ -1141,7 +1141,7 @@ TukeyHSD(aov(time ~ poison + treat, data = rats))
 For an example **with** interaction, we investigate the `warpbreaks` dataset, a default dataset in `R`.
 
 
-```r
+``` r
 par(mfrow = c(1, 2))
 with(warpbreaks, interaction.plot(wool, tension, breaks, lwd = 2, col = 2:4))
 with(warpbreaks, interaction.plot(tension, wool, breaks, lwd = 2, col = 2:3))
@@ -1154,7 +1154,7 @@ with(warpbreaks, interaction.plot(tension, wool, breaks, lwd = 2, col = 2:3))
 Either plot makes it rather clear that the `wool` and `tensions` factors interact.
 
 
-```r
+``` r
 summary(aov(breaks ~ wool * tension, data = warpbreaks))
 ```
 
@@ -1180,4 +1180,4 @@ The `R` Markdown file for this chapter can be found here:
 
 - [`anova.Rmd`](anova.Rmd){target="_blank"}
 
-The file was created using `R` version `4.3.2`.
+The file was created using `R` version `4.4.1`.
diff --git a/appendix.html b/appendix.html
index fcebc02..affcf44 100644
--- a/appendix.html
+++ b/appendix.html
@@ -6,7 +6,7 @@
   <meta http-equiv="X-UA-Compatible" content="IE=edge" />
   <title>Chapter 19 Appendix | Applied Statistics with R</title>
   <meta name="description" content="Chapter 19 Appendix | Applied Statistics with R" />
-  <meta name="generator" content="bookdown 0.37 and GitBook 2.6.7" />
+  <meta name="generator" content="bookdown 0.40 and GitBook 2.6.7" />
 
   <meta property="og:title" content="Chapter 19 Appendix | Applied Statistics with R" />
   <meta property="og:type" content="book" />
@@ -55,7 +55,7 @@
 
 <style type="text/css">
 pre > code.sourceCode { white-space: pre; position: relative; }
-pre > code.sourceCode > span { display: inline-block; line-height: 1.25; }
+pre > code.sourceCode > span { line-height: 1.25; }
 pre > code.sourceCode > span:empty { height: 1.2em; }
 .sourceCode { overflow: visible; }
 code.sourceCode > span { color: inherit; text-decoration: inherit; }
diff --git a/applied_statistics.pdf b/applied_statistics.pdf
index 0eab757..fbf2cfc 100644
Binary files a/applied_statistics.pdf and b/applied_statistics.pdf differ
diff --git a/beyond.html b/beyond.html
index ba4162b..55452f2 100644
--- a/beyond.html
+++ b/beyond.html
@@ -6,7 +6,7 @@
   <meta http-equiv="X-UA-Compatible" content="IE=edge" />
   <title>Chapter 18 Beyond | Applied Statistics with R</title>
   <meta name="description" content="Chapter 18 Beyond | Applied Statistics with R" />
-  <meta name="generator" content="bookdown 0.37 and GitBook 2.6.7" />
+  <meta name="generator" content="bookdown 0.40 and GitBook 2.6.7" />
 
   <meta property="og:title" content="Chapter 18 Beyond | Applied Statistics with R" />
   <meta property="og:type" content="book" />
@@ -55,7 +55,7 @@
 
 <style type="text/css">
 pre > code.sourceCode { white-space: pre; position: relative; }
-pre > code.sourceCode > span { display: inline-block; line-height: 1.25; }
+pre > code.sourceCode > span { line-height: 1.25; }
 pre > code.sourceCode > span:empty { height: 1.2em; }
 .sourceCode { overflow: visible; }
 code.sourceCode > span { color: inherit; text-decoration: inherit; }
diff --git a/cat-int.md b/cat-int.md
index 59c5299..903aad6 100644
--- a/cat-int.md
+++ b/cat-int.md
@@ -20,7 +20,7 @@ So far in each of our analyses, we have only used numeric variables as predictor
 For this chapter, we will briefly use the built-in dataset `mtcars` before returning to our `autompg` dataset that we created in the last chapter. The `mtcars` dataset is somewhat smaller, so we'll quickly take a look at the entire dataset.
 
 
-```r
+``` r
 mtcars
 ```
 
@@ -69,7 +69,7 @@ We will be interested in three of the variables: `mpg `, `hp`, and `am`.
 As we often do, we will start by plotting the data. We are interested in `mpg` as the response variable, and `hp` as a predictor.
 
 
-```r
+``` r
 plot(mpg ~ hp, data = mtcars, cex = 2)
 ```
 
@@ -80,7 +80,7 @@ plot(mpg ~ hp, data = mtcars, cex = 2)
 Since we are also interested in the transmission type, we could also label the points accordingly.
 
 
-```r
+``` r
 plot(mpg ~ hp, data = mtcars, col = am + 1, pch = am + 1, cex = 2)
 legend("topright", c("Automatic", "Manual"), col = c(1, 2), pch = c(1, 2))
 ```
@@ -100,14 +100,14 @@ Y = \beta_0 + \beta_1 x_1 + \epsilon,
 where $Y$ is `mpg` and $x_1$ is `hp`. For notational brevity, we drop the index $i$ for observations.
 
 
-```r
+``` r
 mpg_hp_slr = lm(mpg ~ hp, data = mtcars)
 ```
 
 We then re-plot the data and add the fitted line to the plot.
 
 
-```r
+``` r
 plot(mpg ~ hp, data = mtcars, col = am + 1, pch = am + 1, cex = 2)
 abline(mpg_hp_slr, lwd = 3, col = "grey")
 legend("topright", c("Automatic", "Manual"), col = c(1, 2), pch = c(1, 2))
@@ -142,14 +142,14 @@ First, note that `am` is already a dummy variable, since it uses the values `0`
 So, to fit the above model, we do so like any other multiple regression model we have seen before.
 
 
-```r
+``` r
 mpg_hp_add = lm(mpg ~ hp + am, data = mtcars)
 ```
 
 Briefly checking the output, we see that `R` has estimated the three $\beta$ parameters.
 
 
-```r
+``` r
 mpg_hp_add
 ```
 
@@ -188,7 +188,7 @@ We'll now calculate the estimated slope and intercept of these two models so tha
 We can then combine these to calculate the estimated slope and intercepts.
 
 
-```r
+``` r
 int_auto = coef(mpg_hp_add)[1]
 int_manu = coef(mpg_hp_add)[1] + coef(mpg_hp_add)[3]
 
@@ -199,7 +199,7 @@ slope_manu = coef(mpg_hp_add)[2]
 Re-plotting the data, we use these slopes and intercepts to add the "two" fitted models to the plot.
 
 
-```r
+``` r
 plot(mpg ~ hp, data = mtcars, col = am + 1, pch = am + 1, cex = 2)
 abline(int_auto, slope_auto, col = 1, lty = 1, lwd = 2) # add line for auto
 abline(int_manu, slope_manu, col = 2, lty = 2, lwd = 2) # add line for manual
@@ -223,7 +223,7 @@ This is nothing new. Again, the math is the same as the multiple regression anal
 To obtain the test statistic and p-value for the $t$-test, we would use
 
 
-```r
+``` r
 summary(mpg_hp_add)$coefficients["am",]
 ```
 
@@ -235,7 +235,7 @@ summary(mpg_hp_add)$coefficients["am",]
 To do the same for the $F$ test, we would use
 
 
-```r
+``` r
 anova(mpg_hp_slr, mpg_hp_add)
 ```
 
@@ -276,7 +276,7 @@ Here we restricted ourselves to a single numerical predictor $x_1$ and one dummy
 To remove the "same slope" restriction, we will now discuss **interaction**. To illustrate this concept, we will return to the `autompg` dataset we created in the last chapter, with a few more modifications.
 
 
-```r
+``` r
 # read data frame from the web
 autompg = read.table(
   "http://archive.ics.uci.edu/ml/machine-learning-databases/auto-mpg/auto-mpg.data",
@@ -307,7 +307,7 @@ autompg$cyl = as.factor(autompg$cyl)
 ```
 
 
-```r
+``` r
 str(autompg)
 ```
 
@@ -349,7 +349,7 @@ x_2 =
 We will fit this model, extract the slope and intercept for the "two lines," plot the data and add the lines.
 
 
-```r
+``` r
 mpg_disp_add = lm(mpg ~ disp + domestic, data = autompg)
 
 int_for = coef(mpg_disp_add)[1]
@@ -404,7 +404,7 @@ How do we fit this model in `R`? There are a number of ways.
 One method would be to simply create a new variable, then fit a model like any other.
 
 
-```r
+``` r
 autompg$x3 = autompg$disp * autompg$domestic # THIS CODE NOT RUN!
 do_not_do_this = lm(mpg ~ disp + domestic + x3, data = autompg) # THIS CODE NOT RUN!
 ```
@@ -412,21 +412,21 @@ do_not_do_this = lm(mpg ~ disp + domestic + x3, data = autompg) # THIS CODE NOT
 You should only do this as a last resort. We greatly prefer not to have to modify our data simply to fit a model. Instead, we can tell `R` we would like to use the existing data with an interaction term, which it will create automatically when we use the `:` operator.
 
 
-```r
+``` r
 mpg_disp_int = lm(mpg ~ disp + domestic + disp:domestic, data = autompg)
 ```
 
 An alternative method, which will fit the exact same model as above would be to use the `*` operator. This method automatically creates the interaction term, as well as any "lower order terms," which in this case are the first order terms for `disp` and `domestic`
 
 
-```r
+``` r
 mpg_disp_int2 = lm(mpg ~ disp * domestic, data = autompg)
 ```
 
 We can quickly verify that these are doing the same thing.
 
 
-```r
+``` r
 coef(mpg_disp_int)
 ```
 
@@ -435,7 +435,7 @@ coef(mpg_disp_int)
 ##    46.0548423    -0.1569239   -12.5754714     0.1025184
 ```
 
-```r
+``` r
 coef(mpg_disp_int2)
 ```
 
@@ -447,7 +447,7 @@ coef(mpg_disp_int2)
 We see that both the variables, and their coefficient estimates are indeed the same for both models.
 
 
-```r
+``` r
 summary(mpg_disp_int)
 ```
 
@@ -485,7 +485,7 @@ In this case, testing for $\beta_3 = 0$ is testing for two lines with parallel s
 We could also use an ANOVA $F$-test. The additive model without interaction is our null model, and the interaction model is the alternative.
 
 
-```r
+``` r
 anova(mpg_disp_add, mpg_disp_int)
 ```
 
@@ -494,9 +494,9 @@ anova(mpg_disp_add, mpg_disp_int)
 ## 
 ## Model 1: mpg ~ disp + domestic
 ## Model 2: mpg ~ disp + domestic + disp:domestic
-##   Res.Df    RSS Df Sum of Sq      F    Pr(>F)    
-## 1    380 7714.0                                  
-## 2    379 7032.6  1    681.36 36.719 3.294e-09 ***
+##   Res.Df    RSS Df Sum of Sq     F    Pr(>F)    
+## 1    380 7714.0                                 
+## 2    379 7032.6  1    681.36 36.72 3.294e-09 ***
 ## ---
 ## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
 ```
@@ -504,7 +504,7 @@ anova(mpg_disp_add, mpg_disp_int)
 Again we see this test has the same p-value as the $t$-test. Also the p-value is extremely low, so between the two, we choose the interaction model.
 
 
-```r
+``` r
 int_for = coef(mpg_disp_int)[1]
 int_dom = coef(mpg_disp_int)[1] + coef(mpg_disp_int)[3]
 
@@ -515,7 +515,7 @@ slope_dom = coef(mpg_disp_int)[2] + coef(mpg_disp_int)[4]
 Here we again calculate the slope and intercepts for the two lines for use in plotting.
 
 
-```r
+``` r
 plot(mpg ~ disp, data = autompg, col = domestic + 1, pch = domestic + 1)
 abline(int_for, slope_for, col = 1, lty = 1, lwd = 2) # line for foreign cars
 abline(int_dom, slope_dom, col = 2, lty = 2, lwd = 2) # line for domestic cars
@@ -553,7 +553,7 @@ So, for a one unit increase in $x_1$ (`disp`), the mean of $Y$ (`mpg`) increases
 Since we're now working in three dimensions, this model can't be easily justified via visualizations like the previous example. Instead, we will have to rely on a test.
 
 
-```r
+``` r
 mpg_disp_add_hp = lm(mpg ~ disp + hp, data = autompg)
 mpg_disp_int_hp = lm(mpg ~ disp * hp, data = autompg)
 summary(mpg_disp_int_hp)
@@ -591,7 +591,7 @@ H_0: \beta_3 = 0.
 Again, we see a very low p-value so we reject the null (additive model) in favor of the interaction model. Again, there is an equivalent $F$-test.
 
 
-```r
+``` r
 anova(mpg_disp_add_hp, mpg_disp_int_hp)
 ```
 
@@ -610,7 +610,7 @@ anova(mpg_disp_add_hp, mpg_disp_int_hp)
 We can take a closer look at the coefficients of our fitted interaction model.
 
 
-```r
+``` r
 coef(mpg_disp_int_hp)
 ```
 
@@ -659,7 +659,7 @@ So far in this chapter, we have limited our use of categorical variables to bina
 We will now discuss **factor** variables, which is a special way that `R` deals with categorical variables. With factor variables, a human user can simply think about the categories of a variable, and `R` will take care of the necessary dummy variables without any 0/1 assignment being done by the user.
 
 
-```r
+``` r
 is.factor(autompg$domestic)
 ```
 
@@ -670,7 +670,7 @@ is.factor(autompg$domestic)
 Earlier when we used the `domestic` variable, it was **not** a factor variable. It was simply a numerical variable that only took two possible values, `1` for domestic, and `0` for foreign. Let's create a new variable `origin` that stores the same information, but in a different way.
 
 
-```r
+``` r
 autompg$origin[autompg$domestic == 1] = "domestic"
 autompg$origin[autompg$domestic == 0] = "foreign"
 head(autompg$origin)
@@ -683,7 +683,7 @@ head(autompg$origin)
 Now the `origin` variable stores `"domestic"` for domestic cars and `"foreign"` for foreign cars.
 
 
-```r
+``` r
 is.factor(autompg$origin)
 ```
 
@@ -694,14 +694,14 @@ is.factor(autompg$origin)
 However, this is simply a vector of character values. A vector of car models is a character variable in `R`. A vector of Vehicle Identification Numbers (VINs) is a character variable as well. But those don't represent a short list of levels that might influence a response variable. We will want to **coerce** this origin variable to be something more: a factor variable.
 
 
-```r
+``` r
 autompg$origin = as.factor(autompg$origin)
 ```
 
 Now when we check the structure of the `autompg` dataset, we see that `origin` is a factor variable.
 
 
-```r
+``` r
 str(autompg)
 ```
 
@@ -721,7 +721,7 @@ str(autompg)
 Factor variables have **levels** which are the possible values (categories) that the variable may take, in this case foreign or domestic.
 
 
-```r
+``` r
 levels(autompg$origin)
 ```
 
@@ -742,7 +742,7 @@ where
 - $x_2$ is `domestic`, a dummy variable where `1` indicates a domestic car.
 
 
-```r
+``` r
 (mod_dummy = lm(mpg ~ disp * domestic, data = autompg))
 ```
 
@@ -767,7 +767,7 @@ is the estimated average `mpg` for a **domestic** car with 0 `disp`.
 Now let's try to do the same, but using our new factor variable.
 
 
-```r
+``` r
 (mod_factor = lm(mpg ~ disp * origin, data = autompg))
 ```
 
@@ -814,7 +814,7 @@ So the two models have different estimated coefficients, but due to the differen
 Let's now consider a factor variable with more than two levels. In this dataset, `cyl` is an example.
 
 
-```r
+``` r
 is.factor(autompg$cyl)
 ```
 
@@ -822,7 +822,7 @@ is.factor(autompg$cyl)
 ## [1] TRUE
 ```
 
-```r
+``` r
 levels(autompg$cyl)
 ```
 
@@ -863,7 +863,7 @@ v_3 =
 Now, let's fit an additive model in `R`, using `mpg` as the response, and `disp` and `cyl` as predictors. This should be a model that uses "three regression lines" to model `mpg`, one for each of the possible `cyl` levels. They will all have the same slope (since it is an additive model), but each will have its own intercept.
 
 
-```r
+``` r
 (mpg_disp_add_cyl = lm(mpg ~ disp + cyl, data = autompg))
 ```
 
@@ -906,7 +906,7 @@ So because 4 cylinder is the reference level, $\beta_0$ is specific to 4 cylinde
 As we have done before, we can extract these intercepts and slopes for the three lines, and plot them accordingly.
 
 
-```r
+``` r
 int_4cyl = coef(mpg_disp_add_cyl)[1]
 int_6cyl = coef(mpg_disp_add_cyl)[1] + coef(mpg_disp_add_cyl)[3]
 int_8cyl = coef(mpg_disp_add_cyl)[1] + coef(mpg_disp_add_cyl)[4]
@@ -937,7 +937,7 @@ The odd result here is that we're estimating that 8 cylinder cars have better fu
 To attempt to fix this, we will try using an interaction model, that is, instead of simply three intercepts and one slope, we will allow for three slopes. Again, we'll let `R` take the wheel (no pun intended), then figure out what model it has applied.
 
 
-```r
+``` r
 (mpg_disp_int_cyl = lm(mpg ~ disp * cyl, data = autompg))
 ```
 
@@ -951,7 +951,7 @@ To attempt to fix this, we will try using an interaction model, that is, instead
 ##    43.59052     -0.13069    -13.20026    -20.85706      0.08299      0.10817
 ```
 
-```r
+``` r
 # could also use mpg ~ disp + cyl + disp:cyl
 ```
 
@@ -981,7 +981,7 @@ Now, similarly $\gamma_2$ and $\gamma_3$ change the slopes for 6 and 8 cylinder
 Once again, we extract the coefficients and plot the results.
 
 
-```r
+``` r
 int_4cyl = coef(mpg_disp_int_cyl)[1]
 int_6cyl = coef(mpg_disp_int_cyl)[1] + coef(mpg_disp_int_cyl)[3]
 int_8cyl = coef(mpg_disp_int_cyl)[1] + coef(mpg_disp_int_cyl)[4]
@@ -1022,7 +1022,7 @@ Y = \beta_0 + \beta_1 x + \beta_2 v_2 + \beta_3 v_3 + \epsilon.
 Again, this is a difference of two parameters, thus no $t$-test will be useful.
 
 
-```r
+``` r
 anova(mpg_disp_add_cyl, mpg_disp_int_cyl)
 ```
 
@@ -1049,7 +1049,7 @@ Recapping a bit:
     
     
 
-```r
+``` r
 length(coef(mpg_disp_int_cyl)) - length(coef(mpg_disp_add_cyl))
 ```
 
@@ -1060,7 +1060,7 @@ length(coef(mpg_disp_int_cyl)) - length(coef(mpg_disp_add_cyl))
 We see there is a difference of two parameters, which is also displayed in the resulting ANOVA table from `R`. Notice that the following two values also appear on the ANOVA table.
 
 
-```r
+``` r
 nrow(autompg) - length(coef(mpg_disp_int_cyl))
 ```
 
@@ -1068,7 +1068,7 @@ nrow(autompg) - length(coef(mpg_disp_int_cyl))
 ## [1] 377
 ```
 
-```r
+``` r
 nrow(autompg) - length(coef(mpg_disp_add_cyl))
 ```
 
@@ -1081,7 +1081,7 @@ nrow(autompg) - length(coef(mpg_disp_add_cyl))
 So far we have been simply letting `R` decide how to create the dummy variables, and thus `R` has been deciding the parameterization of the models. To illustrate the ability to use alternative parameterizations, we will recreate the data, but directly creating the dummy variables ourselves.
 
 
-```r
+``` r
 new_param_data = data.frame(
   y = autompg$mpg,
   x = autompg$disp,
@@ -1125,7 +1125,7 @@ Now,
 First let's try to fit an additive model using `x` as well as the three dummy variables.
 
 
-```r
+``` r
 lm(y ~ x + v1 + v2 + v3, data = new_param_data)
 ```
 
@@ -1150,7 +1150,7 @@ which means that $\boldsymbol{1}$, $v_1$, $v_2$, and $v_3$ are linearly dependen
 If we remove the intercept, then we can directly obtain all "three intercepts" without a reference level.
 
 
-```r
+``` r
 lm(y ~ 0 + x + v1 + v2 + v3, data = new_param_data)
 ```
 
@@ -1179,7 +1179,7 @@ Thus we have:
 We could also do something similar with the interaction model, and give each line an intercept and slope, without the need for a reference level.
 
 
-```r
+``` r
 lm(y ~ 0 + v1 + v2 + v3 + x:v1 + x:v2 + x:v3, data = new_param_data)
 ```
 
@@ -1204,7 +1204,7 @@ Y = \mu_1 v_1 + \mu_2 v_2 + \mu_3 v_3 + \beta_1 x v_1 + \beta_2 x v_2 + \beta_3
 Using the original data, we have (at least) three equivalent ways to specify the interaction model with `R`.
 
 
-```r
+``` r
 lm(mpg ~ disp * cyl, data = autompg)
 ```
 
@@ -1218,7 +1218,7 @@ lm(mpg ~ disp * cyl, data = autompg)
 ##    43.59052     -0.13069    -13.20026    -20.85706      0.08299      0.10817
 ```
 
-```r
+``` r
 lm(mpg ~ 0 + cyl + disp : cyl, data = autompg)
 ```
 
@@ -1232,7 +1232,7 @@ lm(mpg ~ 0 + cyl + disp : cyl, data = autompg)
 ##  43.59052   30.39026   22.73346   -0.13069   -0.04770   -0.02252
 ```
 
-```r
+``` r
 lm(mpg ~ 0 + disp + cyl + disp : cyl, data = autompg)
 ```
 
@@ -1251,7 +1251,7 @@ They all fit the same model, importantly each using six parameters, but the coef
 Use `?all.equal` to learn about the `all.equal()` function, and think about how the following code verifies that the residuals of the two models are the same.
 
 
-```r
+``` r
 all.equal(fitted(lm(mpg ~ disp * cyl, data = autompg)), 
           fitted(lm(mpg ~ 0 + cyl + disp : cyl, data = autompg)))
 ```
@@ -1332,7 +1332,7 @@ It is so flexible, it is becoming hard to interpret!
 Let's fit this three-way interaction model in `R`.
 
 
-```r
+``` r
 big_model = lm(mpg ~ disp * hp * domestic, data = autompg)
 summary(big_model)
 ```
@@ -1378,7 +1378,7 @@ So,
 We fit the null model in `R` as `two_way_int_mod`, then use `anova()` to perform an $F$-test as usual.
 
 
-```r
+``` r
 two_way_int_mod = lm(mpg ~ disp * hp + disp * domestic + hp * domestic, data = autompg)
 #two_way_int_mod = lm(mpg ~ (disp + hp + domestic) ^ 2, data = autompg)
 anova(two_way_int_mod, big_model)
@@ -1399,7 +1399,7 @@ We see the p-value is somewhat large, so we would fail to reject. We prefer the
 A quick note here: the full model does still "fit better." Notice that it has a smaller RMSE than the null model, which means the full model makes smaller (squared) errors on average.
 
 
-```r
+``` r
 mean(resid(big_model) ^ 2)
 ```
 
@@ -1407,7 +1407,7 @@ mean(resid(big_model) ^ 2)
 ## [1] 14.74053
 ```
 
-```r
+``` r
 mean(resid(two_way_int_mod) ^ 2)
 ```
 
@@ -1431,7 +1431,7 @@ Remember we already chose $\beta_7 = 0$, so,
 We fit the null model in `R` as `additive_mod`, then use `anova()` to perform an $F$-test as usual.
 
 
-```r
+``` r
 additive_mod = lm(mpg ~ disp + hp + domestic, data = autompg)
 anova(additive_mod, two_way_int_mod)
 ```
@@ -1460,4 +1460,4 @@ The `R` Markdown file for this chapter can be found here:
 
 - [`cat-int.Rmd`](cat-int.Rmd){target="_blank"}
 
-The file was created using `R` version `4.3.2`.
+The file was created using `R` version `4.4.1`.
diff --git a/categorical-predictors-and-interactions.html b/categorical-predictors-and-interactions.html
index 902b3ec..674aee6 100644
--- a/categorical-predictors-and-interactions.html
+++ b/categorical-predictors-and-interactions.html
@@ -6,7 +6,7 @@
   <meta http-equiv="X-UA-Compatible" content="IE=edge" />
   <title>Chapter 11 Categorical Predictors and Interactions | Applied Statistics with R</title>
   <meta name="description" content="Chapter 11 Categorical Predictors and Interactions | Applied Statistics with R" />
-  <meta name="generator" content="bookdown 0.37 and GitBook 2.6.7" />
+  <meta name="generator" content="bookdown 0.40 and GitBook 2.6.7" />
 
   <meta property="og:title" content="Chapter 11 Categorical Predictors and Interactions | Applied Statistics with R" />
   <meta property="og:type" content="book" />
@@ -55,7 +55,7 @@
 
 <style type="text/css">
 pre > code.sourceCode { white-space: pre; position: relative; }
-pre > code.sourceCode > span { display: inline-block; line-height: 1.25; }
+pre > code.sourceCode > span { line-height: 1.25; }
 pre > code.sourceCode > span:empty { height: 1.2em; }
 .sourceCode { overflow: visible; }
 code.sourceCode > span { color: inherit; text-decoration: inherit; }
@@ -481,7 +481,7 @@ <h1><span class="header-section-number">Chapter 11</span> Categorical Predictors
 <div id="dummy-variables" class="section level2 hasAnchor" number="11.1">
 <h2><span class="header-section-number">11.1</span> Dummy Variables<a href="categorical-predictors-and-interactions.html#dummy-variables" class="anchor-section" aria-label="Anchor link to header"></a></h2>
 <p>For this chapter, we will briefly use the built-in dataset <code>mtcars</code> before returning to our <code>autompg</code> dataset that we created in the last chapter. The <code>mtcars</code> dataset is somewhat smaller, so we’ll quickly take a look at the entire dataset.</p>
-<div class="sourceCode" id="cb709"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb709-1"><a href="categorical-predictors-and-interactions.html#cb709-1" aria-hidden="true" tabindex="-1"></a>mtcars</span></code></pre></div>
+<div class="sourceCode" id="cb709"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb709-1"><a href="categorical-predictors-and-interactions.html#cb709-1" tabindex="-1"></a>mtcars</span></code></pre></div>
 <pre><code>##                      mpg cyl  disp  hp drat    wt  qsec vs am gear carb
 ## Mazda RX4           21.0   6 160.0 110 3.90 2.620 16.46  0  1    4    4
 ## Mazda RX4 Wag       21.0   6 160.0 110 3.90 2.875 17.02  0  1    4    4
@@ -522,11 +522,11 @@ <h2><span class="header-section-number">11.1</span> Dummy Variables<a href="cate
 <li><code>am</code>: transmission. Automatic or manual.</li>
 </ul>
 <p>As we often do, we will start by plotting the data. We are interested in <code>mpg</code> as the response variable, and <code>hp</code> as a predictor.</p>
-<div class="sourceCode" id="cb711"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb711-1"><a href="categorical-predictors-and-interactions.html#cb711-1" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(mpg <span class="sc">~</span> hp, <span class="at">data =</span> mtcars, <span class="at">cex =</span> <span class="dv">2</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb711"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb711-1"><a href="categorical-predictors-and-interactions.html#cb711-1" tabindex="-1"></a><span class="fu">plot</span>(mpg <span class="sc">~</span> hp, <span class="at">data =</span> mtcars, <span class="at">cex =</span> <span class="dv">2</span>)</span></code></pre></div>
 <p><img src="cat-int_files/figure-html/unnamed-chunk-3-1.png" width="672" style="display: block; margin: auto;" /></p>
 <p>Since we are also interested in the transmission type, we could also label the points accordingly.</p>
-<div class="sourceCode" id="cb712"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb712-1"><a href="categorical-predictors-and-interactions.html#cb712-1" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(mpg <span class="sc">~</span> hp, <span class="at">data =</span> mtcars, <span class="at">col =</span> am <span class="sc">+</span> <span class="dv">1</span>, <span class="at">pch =</span> am <span class="sc">+</span> <span class="dv">1</span>, <span class="at">cex =</span> <span class="dv">2</span>)</span>
-<span id="cb712-2"><a href="categorical-predictors-and-interactions.html#cb712-2" aria-hidden="true" tabindex="-1"></a><span class="fu">legend</span>(<span class="st">&quot;topright&quot;</span>, <span class="fu">c</span>(<span class="st">&quot;Automatic&quot;</span>, <span class="st">&quot;Manual&quot;</span>), <span class="at">col =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>), <span class="at">pch =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>))</span></code></pre></div>
+<div class="sourceCode" id="cb712"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb712-1"><a href="categorical-predictors-and-interactions.html#cb712-1" tabindex="-1"></a><span class="fu">plot</span>(mpg <span class="sc">~</span> hp, <span class="at">data =</span> mtcars, <span class="at">col =</span> am <span class="sc">+</span> <span class="dv">1</span>, <span class="at">pch =</span> am <span class="sc">+</span> <span class="dv">1</span>, <span class="at">cex =</span> <span class="dv">2</span>)</span>
+<span id="cb712-2"><a href="categorical-predictors-and-interactions.html#cb712-2" tabindex="-1"></a><span class="fu">legend</span>(<span class="st">&quot;topright&quot;</span>, <span class="fu">c</span>(<span class="st">&quot;Automatic&quot;</span>, <span class="st">&quot;Manual&quot;</span>), <span class="at">col =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>), <span class="at">pch =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>))</span></code></pre></div>
 <p><img src="cat-int_files/figure-html/unnamed-chunk-4-1.png" width="672" style="display: block; margin: auto;" /></p>
 <p>We used a common <code>R</code> “trick” when plotting this data. The <code>am</code> variable takes two possible values; <code>0</code> for automatic transmission, and <code>1</code> for manual transmissions. <code>R</code> can use numbers to represent colors, however the color for <code>0</code> is white. So we take the <code>am</code> vector and add <code>1</code> to it. Then observations with automatic transmissions are now represented by <code>1</code>, which is black in <code>R</code>, and manual transmission are represented by <code>2</code>, which is red in <code>R</code>. (Note, we are only adding <code>1</code> inside the call to <code>plot()</code>, we are not actually modifying the values stored in <code>am</code>.)</p>
 <p>We now fit the SLR model</p>
@@ -534,11 +534,11 @@ <h2><span class="header-section-number">11.1</span> Dummy Variables<a href="cate
 Y = \beta_0 + \beta_1 x_1 + \epsilon,
 \]</span></p>
 <p>where <span class="math inline">\(Y\)</span> is <code>mpg</code> and <span class="math inline">\(x_1\)</span> is <code>hp</code>. For notational brevity, we drop the index <span class="math inline">\(i\)</span> for observations.</p>
-<div class="sourceCode" id="cb713"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb713-1"><a href="categorical-predictors-and-interactions.html#cb713-1" aria-hidden="true" tabindex="-1"></a>mpg_hp_slr <span class="ot">=</span> <span class="fu">lm</span>(mpg <span class="sc">~</span> hp, <span class="at">data =</span> mtcars)</span></code></pre></div>
+<div class="sourceCode" id="cb713"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb713-1"><a href="categorical-predictors-and-interactions.html#cb713-1" tabindex="-1"></a>mpg_hp_slr <span class="ot">=</span> <span class="fu">lm</span>(mpg <span class="sc">~</span> hp, <span class="at">data =</span> mtcars)</span></code></pre></div>
 <p>We then re-plot the data and add the fitted line to the plot.</p>
-<div class="sourceCode" id="cb714"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb714-1"><a href="categorical-predictors-and-interactions.html#cb714-1" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(mpg <span class="sc">~</span> hp, <span class="at">data =</span> mtcars, <span class="at">col =</span> am <span class="sc">+</span> <span class="dv">1</span>, <span class="at">pch =</span> am <span class="sc">+</span> <span class="dv">1</span>, <span class="at">cex =</span> <span class="dv">2</span>)</span>
-<span id="cb714-2"><a href="categorical-predictors-and-interactions.html#cb714-2" aria-hidden="true" tabindex="-1"></a><span class="fu">abline</span>(mpg_hp_slr, <span class="at">lwd =</span> <span class="dv">3</span>, <span class="at">col =</span> <span class="st">&quot;grey&quot;</span>)</span>
-<span id="cb714-3"><a href="categorical-predictors-and-interactions.html#cb714-3" aria-hidden="true" tabindex="-1"></a><span class="fu">legend</span>(<span class="st">&quot;topright&quot;</span>, <span class="fu">c</span>(<span class="st">&quot;Automatic&quot;</span>, <span class="st">&quot;Manual&quot;</span>), <span class="at">col =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>), <span class="at">pch =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>))</span></code></pre></div>
+<div class="sourceCode" id="cb714"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb714-1"><a href="categorical-predictors-and-interactions.html#cb714-1" tabindex="-1"></a><span class="fu">plot</span>(mpg <span class="sc">~</span> hp, <span class="at">data =</span> mtcars, <span class="at">col =</span> am <span class="sc">+</span> <span class="dv">1</span>, <span class="at">pch =</span> am <span class="sc">+</span> <span class="dv">1</span>, <span class="at">cex =</span> <span class="dv">2</span>)</span>
+<span id="cb714-2"><a href="categorical-predictors-and-interactions.html#cb714-2" tabindex="-1"></a><span class="fu">abline</span>(mpg_hp_slr, <span class="at">lwd =</span> <span class="dv">3</span>, <span class="at">col =</span> <span class="st">&quot;grey&quot;</span>)</span>
+<span id="cb714-3"><a href="categorical-predictors-and-interactions.html#cb714-3" tabindex="-1"></a><span class="fu">legend</span>(<span class="st">&quot;topright&quot;</span>, <span class="fu">c</span>(<span class="st">&quot;Automatic&quot;</span>, <span class="st">&quot;Manual&quot;</span>), <span class="at">col =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>), <span class="at">pch =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>))</span></code></pre></div>
 <p><img src="cat-int_files/figure-html/unnamed-chunk-6-1.png" width="672" style="display: block; margin: auto;" /></p>
 <p>We should notice a pattern here. The red, manual observations largely fall above the line, while the black, automatic observations are mostly below the line. This means our model underestimates the fuel efficiency of manual transmissions, and overestimates the fuel efficiency of automatic transmissions. To correct for this, we will add a predictor to our model, namely, <code>am</code> as <span class="math inline">\(x_2\)</span>.</p>
 <p>Our new model is</p>
@@ -556,9 +556,9 @@ <h2><span class="header-section-number">11.1</span> Dummy Variables<a href="cate
 <p>In this case, we call <span class="math inline">\(x_2\)</span> a <strong>dummy variable</strong>. A dummy variable is somewhat unfortunately named, as it is in no way “dumb.” In fact, it is actually somewhat clever. A dummy variable is a numerical variable that is used in a regression analysis to “code” for a binary categorical variable. Let’s see how this works.</p>
 <p>First, note that <code>am</code> is already a dummy variable, since it uses the values <code>0</code> and <code>1</code> to represent automatic and manual transmissions. Often, a variable like <code>am</code> would store the character values <code>auto</code> and <code>man</code> and we would either have to convert these to <code>0</code> and <code>1</code>, or, as we will see later, <code>R</code> will take care of creating dummy variables for us.</p>
 <p>So, to fit the above model, we do so like any other multiple regression model we have seen before.</p>
-<div class="sourceCode" id="cb715"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb715-1"><a href="categorical-predictors-and-interactions.html#cb715-1" aria-hidden="true" tabindex="-1"></a>mpg_hp_add <span class="ot">=</span> <span class="fu">lm</span>(mpg <span class="sc">~</span> hp <span class="sc">+</span> am, <span class="at">data =</span> mtcars)</span></code></pre></div>
+<div class="sourceCode" id="cb715"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb715-1"><a href="categorical-predictors-and-interactions.html#cb715-1" tabindex="-1"></a>mpg_hp_add <span class="ot">=</span> <span class="fu">lm</span>(mpg <span class="sc">~</span> hp <span class="sc">+</span> am, <span class="at">data =</span> mtcars)</span></code></pre></div>
 <p>Briefly checking the output, we see that <code>R</code> has estimated the three <span class="math inline">\(\beta\)</span> parameters.</p>
-<div class="sourceCode" id="cb716"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb716-1"><a href="categorical-predictors-and-interactions.html#cb716-1" aria-hidden="true" tabindex="-1"></a>mpg_hp_add</span></code></pre></div>
+<div class="sourceCode" id="cb716"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb716-1"><a href="categorical-predictors-and-interactions.html#cb716-1" tabindex="-1"></a>mpg_hp_add</span></code></pre></div>
 <pre><code>## 
 ## Call:
 ## lm(formula = mpg ~ hp + am, data = mtcars)
@@ -583,16 +583,16 @@ <h2><span class="header-section-number">11.1</span> Dummy Variables<a href="cate
 <li><span class="math inline">\(\hat{\beta}_2\)</span> = <code>coef(mpg_hp_add)[3]</code> = 5.2770853</li>
 </ul>
 <p>We can then combine these to calculate the estimated slope and intercepts.</p>
-<div class="sourceCode" id="cb718"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb718-1"><a href="categorical-predictors-and-interactions.html#cb718-1" aria-hidden="true" tabindex="-1"></a>int_auto <span class="ot">=</span> <span class="fu">coef</span>(mpg_hp_add)[<span class="dv">1</span>]</span>
-<span id="cb718-2"><a href="categorical-predictors-and-interactions.html#cb718-2" aria-hidden="true" tabindex="-1"></a>int_manu <span class="ot">=</span> <span class="fu">coef</span>(mpg_hp_add)[<span class="dv">1</span>] <span class="sc">+</span> <span class="fu">coef</span>(mpg_hp_add)[<span class="dv">3</span>]</span>
-<span id="cb718-3"><a href="categorical-predictors-and-interactions.html#cb718-3" aria-hidden="true" tabindex="-1"></a></span>
-<span id="cb718-4"><a href="categorical-predictors-and-interactions.html#cb718-4" aria-hidden="true" tabindex="-1"></a>slope_auto <span class="ot">=</span> <span class="fu">coef</span>(mpg_hp_add)[<span class="dv">2</span>]</span>
-<span id="cb718-5"><a href="categorical-predictors-and-interactions.html#cb718-5" aria-hidden="true" tabindex="-1"></a>slope_manu <span class="ot">=</span> <span class="fu">coef</span>(mpg_hp_add)[<span class="dv">2</span>]</span></code></pre></div>
+<div class="sourceCode" id="cb718"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb718-1"><a href="categorical-predictors-and-interactions.html#cb718-1" tabindex="-1"></a>int_auto <span class="ot">=</span> <span class="fu">coef</span>(mpg_hp_add)[<span class="dv">1</span>]</span>
+<span id="cb718-2"><a href="categorical-predictors-and-interactions.html#cb718-2" tabindex="-1"></a>int_manu <span class="ot">=</span> <span class="fu">coef</span>(mpg_hp_add)[<span class="dv">1</span>] <span class="sc">+</span> <span class="fu">coef</span>(mpg_hp_add)[<span class="dv">3</span>]</span>
+<span id="cb718-3"><a href="categorical-predictors-and-interactions.html#cb718-3" tabindex="-1"></a></span>
+<span id="cb718-4"><a href="categorical-predictors-and-interactions.html#cb718-4" tabindex="-1"></a>slope_auto <span class="ot">=</span> <span class="fu">coef</span>(mpg_hp_add)[<span class="dv">2</span>]</span>
+<span id="cb718-5"><a href="categorical-predictors-and-interactions.html#cb718-5" tabindex="-1"></a>slope_manu <span class="ot">=</span> <span class="fu">coef</span>(mpg_hp_add)[<span class="dv">2</span>]</span></code></pre></div>
 <p>Re-plotting the data, we use these slopes and intercepts to add the “two” fitted models to the plot.</p>
-<div class="sourceCode" id="cb719"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb719-1"><a href="categorical-predictors-and-interactions.html#cb719-1" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(mpg <span class="sc">~</span> hp, <span class="at">data =</span> mtcars, <span class="at">col =</span> am <span class="sc">+</span> <span class="dv">1</span>, <span class="at">pch =</span> am <span class="sc">+</span> <span class="dv">1</span>, <span class="at">cex =</span> <span class="dv">2</span>)</span>
-<span id="cb719-2"><a href="categorical-predictors-and-interactions.html#cb719-2" aria-hidden="true" tabindex="-1"></a><span class="fu">abline</span>(int_auto, slope_auto, <span class="at">col =</span> <span class="dv">1</span>, <span class="at">lty =</span> <span class="dv">1</span>, <span class="at">lwd =</span> <span class="dv">2</span>) <span class="co"># add line for auto</span></span>
-<span id="cb719-3"><a href="categorical-predictors-and-interactions.html#cb719-3" aria-hidden="true" tabindex="-1"></a><span class="fu">abline</span>(int_manu, slope_manu, <span class="at">col =</span> <span class="dv">2</span>, <span class="at">lty =</span> <span class="dv">2</span>, <span class="at">lwd =</span> <span class="dv">2</span>) <span class="co"># add line for manual</span></span>
-<span id="cb719-4"><a href="categorical-predictors-and-interactions.html#cb719-4" aria-hidden="true" tabindex="-1"></a><span class="fu">legend</span>(<span class="st">&quot;topright&quot;</span>, <span class="fu">c</span>(<span class="st">&quot;Automatic&quot;</span>, <span class="st">&quot;Manual&quot;</span>), <span class="at">col =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>), <span class="at">pch =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>))</span></code></pre></div>
+<div class="sourceCode" id="cb719"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb719-1"><a href="categorical-predictors-and-interactions.html#cb719-1" tabindex="-1"></a><span class="fu">plot</span>(mpg <span class="sc">~</span> hp, <span class="at">data =</span> mtcars, <span class="at">col =</span> am <span class="sc">+</span> <span class="dv">1</span>, <span class="at">pch =</span> am <span class="sc">+</span> <span class="dv">1</span>, <span class="at">cex =</span> <span class="dv">2</span>)</span>
+<span id="cb719-2"><a href="categorical-predictors-and-interactions.html#cb719-2" tabindex="-1"></a><span class="fu">abline</span>(int_auto, slope_auto, <span class="at">col =</span> <span class="dv">1</span>, <span class="at">lty =</span> <span class="dv">1</span>, <span class="at">lwd =</span> <span class="dv">2</span>) <span class="co"># add line for auto</span></span>
+<span id="cb719-3"><a href="categorical-predictors-and-interactions.html#cb719-3" tabindex="-1"></a><span class="fu">abline</span>(int_manu, slope_manu, <span class="at">col =</span> <span class="dv">2</span>, <span class="at">lty =</span> <span class="dv">2</span>, <span class="at">lwd =</span> <span class="dv">2</span>) <span class="co"># add line for manual</span></span>
+<span id="cb719-4"><a href="categorical-predictors-and-interactions.html#cb719-4" tabindex="-1"></a><span class="fu">legend</span>(<span class="st">&quot;topright&quot;</span>, <span class="fu">c</span>(<span class="st">&quot;Automatic&quot;</span>, <span class="st">&quot;Manual&quot;</span>), <span class="at">col =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>), <span class="at">pch =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>))</span></code></pre></div>
 <p><img src="cat-int_files/figure-html/unnamed-chunk-10-1.png" width="672" style="display: block; margin: auto;" /></p>
 <p>We notice right away that the points are no longer systematically incorrect. The red, manual observations vary about the red line in no particular pattern without underestimating the observations as before. The black, automatic points vary about the black line, also without an obvious pattern.</p>
 <p>They say a picture is worth a thousand words, but as a statistician, sometimes a picture is worth an entire analysis. The above picture makes it plainly obvious that <span class="math inline">\(\beta_2\)</span> is significant, but let’s verify mathematically. Essentially we would like to test:</p>
@@ -601,11 +601,11 @@ <h2><span class="header-section-number">11.1</span> Dummy Variables<a href="cate
 \]</span></p>
 <p>This is nothing new. Again, the math is the same as the multiple regression analyses we have seen before. We could perform either a <span class="math inline">\(t\)</span> or <span class="math inline">\(F\)</span> test here. The only difference is a slight change in interpretation. We could think of this as testing a model with a single line (<span class="math inline">\(H_0\)</span>) against a model that allows two lines (<span class="math inline">\(H_1\)</span>).</p>
 <p>To obtain the test statistic and p-value for the <span class="math inline">\(t\)</span>-test, we would use</p>
-<div class="sourceCode" id="cb720"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb720-1"><a href="categorical-predictors-and-interactions.html#cb720-1" aria-hidden="true" tabindex="-1"></a><span class="fu">summary</span>(mpg_hp_add)<span class="sc">$</span>coefficients[<span class="st">&quot;am&quot;</span>,]</span></code></pre></div>
+<div class="sourceCode" id="cb720"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb720-1"><a href="categorical-predictors-and-interactions.html#cb720-1" tabindex="-1"></a><span class="fu">summary</span>(mpg_hp_add)<span class="sc">$</span>coefficients[<span class="st">&quot;am&quot;</span>,]</span></code></pre></div>
 <pre><code>##     Estimate   Std. Error      t value     Pr(&gt;|t|) 
 ## 5.277085e+00 1.079541e+00 4.888270e+00 3.460318e-05</code></pre>
 <p>To do the same for the <span class="math inline">\(F\)</span> test, we would use</p>
-<div class="sourceCode" id="cb722"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb722-1"><a href="categorical-predictors-and-interactions.html#cb722-1" aria-hidden="true" tabindex="-1"></a><span class="fu">anova</span>(mpg_hp_slr, mpg_hp_add)</span></code></pre></div>
+<div class="sourceCode" id="cb722"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb722-1"><a href="categorical-predictors-and-interactions.html#cb722-1" tabindex="-1"></a><span class="fu">anova</span>(mpg_hp_slr, mpg_hp_add)</span></code></pre></div>
 <pre><code>## Analysis of Variance Table
 ## 
 ## Model 1: mpg ~ hp
@@ -633,34 +633,34 @@ <h2><span class="header-section-number">11.1</span> Dummy Variables<a href="cate
 <div id="interactions" class="section level2 hasAnchor" number="11.2">
 <h2><span class="header-section-number">11.2</span> Interactions<a href="categorical-predictors-and-interactions.html#interactions" class="anchor-section" aria-label="Anchor link to header"></a></h2>
 <p>To remove the “same slope” restriction, we will now discuss <strong>interaction</strong>. To illustrate this concept, we will return to the <code>autompg</code> dataset we created in the last chapter, with a few more modifications.</p>
-<div class="sourceCode" id="cb724"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb724-1"><a href="categorical-predictors-and-interactions.html#cb724-1" aria-hidden="true" tabindex="-1"></a><span class="co"># read data frame from the web</span></span>
-<span id="cb724-2"><a href="categorical-predictors-and-interactions.html#cb724-2" aria-hidden="true" tabindex="-1"></a>autompg <span class="ot">=</span> <span class="fu">read.table</span>(</span>
-<span id="cb724-3"><a href="categorical-predictors-and-interactions.html#cb724-3" aria-hidden="true" tabindex="-1"></a>  <span class="st">&quot;http://archive.ics.uci.edu/ml/machine-learning-databases/auto-mpg/auto-mpg.data&quot;</span>,</span>
-<span id="cb724-4"><a href="categorical-predictors-and-interactions.html#cb724-4" aria-hidden="true" tabindex="-1"></a>  <span class="at">quote =</span> <span class="st">&quot;</span><span class="sc">\&quot;</span><span class="st">&quot;</span>,</span>
-<span id="cb724-5"><a href="categorical-predictors-and-interactions.html#cb724-5" aria-hidden="true" tabindex="-1"></a>  <span class="at">comment.char =</span> <span class="st">&quot;&quot;</span>,</span>
-<span id="cb724-6"><a href="categorical-predictors-and-interactions.html#cb724-6" aria-hidden="true" tabindex="-1"></a>  <span class="at">stringsAsFactors =</span> <span class="cn">FALSE</span>)</span>
-<span id="cb724-7"><a href="categorical-predictors-and-interactions.html#cb724-7" aria-hidden="true" tabindex="-1"></a><span class="co"># give the dataframe headers</span></span>
-<span id="cb724-8"><a href="categorical-predictors-and-interactions.html#cb724-8" aria-hidden="true" tabindex="-1"></a><span class="fu">colnames</span>(autompg) <span class="ot">=</span> <span class="fu">c</span>(<span class="st">&quot;mpg&quot;</span>, <span class="st">&quot;cyl&quot;</span>, <span class="st">&quot;disp&quot;</span>, <span class="st">&quot;hp&quot;</span>, <span class="st">&quot;wt&quot;</span>, <span class="st">&quot;acc&quot;</span>, <span class="st">&quot;year&quot;</span>, <span class="st">&quot;origin&quot;</span>, <span class="st">&quot;name&quot;</span>)</span>
-<span id="cb724-9"><a href="categorical-predictors-and-interactions.html#cb724-9" aria-hidden="true" tabindex="-1"></a><span class="co"># remove missing data, which is stored as &quot;?&quot;</span></span>
-<span id="cb724-10"><a href="categorical-predictors-and-interactions.html#cb724-10" aria-hidden="true" tabindex="-1"></a>autompg <span class="ot">=</span> <span class="fu">subset</span>(autompg, autompg<span class="sc">$</span>hp <span class="sc">!=</span> <span class="st">&quot;?&quot;</span>)</span>
-<span id="cb724-11"><a href="categorical-predictors-and-interactions.html#cb724-11" aria-hidden="true" tabindex="-1"></a><span class="co"># remove the plymouth reliant, as it causes some issues</span></span>
-<span id="cb724-12"><a href="categorical-predictors-and-interactions.html#cb724-12" aria-hidden="true" tabindex="-1"></a>autompg <span class="ot">=</span> <span class="fu">subset</span>(autompg, autompg<span class="sc">$</span>name <span class="sc">!=</span> <span class="st">&quot;plymouth reliant&quot;</span>)</span>
-<span id="cb724-13"><a href="categorical-predictors-and-interactions.html#cb724-13" aria-hidden="true" tabindex="-1"></a><span class="co"># give the dataset row names, based on the engine, year and name</span></span>
-<span id="cb724-14"><a href="categorical-predictors-and-interactions.html#cb724-14" aria-hidden="true" tabindex="-1"></a><span class="fu">rownames</span>(autompg) <span class="ot">=</span> <span class="fu">paste</span>(autompg<span class="sc">$</span>cyl, <span class="st">&quot;cylinder&quot;</span>, autompg<span class="sc">$</span>year, autompg<span class="sc">$</span>name)</span>
-<span id="cb724-15"><a href="categorical-predictors-and-interactions.html#cb724-15" aria-hidden="true" tabindex="-1"></a><span class="co"># remove the variable for name</span></span>
-<span id="cb724-16"><a href="categorical-predictors-and-interactions.html#cb724-16" aria-hidden="true" tabindex="-1"></a>autompg <span class="ot">=</span> <span class="fu">subset</span>(autompg, <span class="at">select =</span> <span class="fu">c</span>(<span class="st">&quot;mpg&quot;</span>, <span class="st">&quot;cyl&quot;</span>, <span class="st">&quot;disp&quot;</span>, <span class="st">&quot;hp&quot;</span>, <span class="st">&quot;wt&quot;</span>, <span class="st">&quot;acc&quot;</span>, <span class="st">&quot;year&quot;</span>, <span class="st">&quot;origin&quot;</span>))</span>
-<span id="cb724-17"><a href="categorical-predictors-and-interactions.html#cb724-17" aria-hidden="true" tabindex="-1"></a><span class="co"># change horsepower from character to numeric</span></span>
-<span id="cb724-18"><a href="categorical-predictors-and-interactions.html#cb724-18" aria-hidden="true" tabindex="-1"></a>autompg<span class="sc">$</span>hp <span class="ot">=</span> <span class="fu">as.numeric</span>(autompg<span class="sc">$</span>hp)</span>
-<span id="cb724-19"><a href="categorical-predictors-and-interactions.html#cb724-19" aria-hidden="true" tabindex="-1"></a><span class="co"># create a dummy variable for foreign vs domestic cars. domestic = 1.</span></span>
-<span id="cb724-20"><a href="categorical-predictors-and-interactions.html#cb724-20" aria-hidden="true" tabindex="-1"></a>autompg<span class="sc">$</span>domestic <span class="ot">=</span> <span class="fu">as.numeric</span>(autompg<span class="sc">$</span>origin <span class="sc">==</span> <span class="dv">1</span>)</span>
-<span id="cb724-21"><a href="categorical-predictors-and-interactions.html#cb724-21" aria-hidden="true" tabindex="-1"></a><span class="co"># remove 3 and 5 cylinder cars (which are very rare.)</span></span>
-<span id="cb724-22"><a href="categorical-predictors-and-interactions.html#cb724-22" aria-hidden="true" tabindex="-1"></a>autompg <span class="ot">=</span> autompg[autompg<span class="sc">$</span>cyl <span class="sc">!=</span> <span class="dv">5</span>,]</span>
-<span id="cb724-23"><a href="categorical-predictors-and-interactions.html#cb724-23" aria-hidden="true" tabindex="-1"></a>autompg <span class="ot">=</span> autompg[autompg<span class="sc">$</span>cyl <span class="sc">!=</span> <span class="dv">3</span>,]</span>
-<span id="cb724-24"><a href="categorical-predictors-and-interactions.html#cb724-24" aria-hidden="true" tabindex="-1"></a><span class="co"># the following line would verify the remaining cylinder possibilities are 4, 6, 8</span></span>
-<span id="cb724-25"><a href="categorical-predictors-and-interactions.html#cb724-25" aria-hidden="true" tabindex="-1"></a><span class="co">#unique(autompg$cyl)</span></span>
-<span id="cb724-26"><a href="categorical-predictors-and-interactions.html#cb724-26" aria-hidden="true" tabindex="-1"></a><span class="co"># change cyl to a factor variable</span></span>
-<span id="cb724-27"><a href="categorical-predictors-and-interactions.html#cb724-27" aria-hidden="true" tabindex="-1"></a>autompg<span class="sc">$</span>cyl <span class="ot">=</span> <span class="fu">as.factor</span>(autompg<span class="sc">$</span>cyl)</span></code></pre></div>
-<div class="sourceCode" id="cb725"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb725-1"><a href="categorical-predictors-and-interactions.html#cb725-1" aria-hidden="true" tabindex="-1"></a><span class="fu">str</span>(autompg)</span></code></pre></div>
+<div class="sourceCode" id="cb724"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb724-1"><a href="categorical-predictors-and-interactions.html#cb724-1" tabindex="-1"></a><span class="co"># read data frame from the web</span></span>
+<span id="cb724-2"><a href="categorical-predictors-and-interactions.html#cb724-2" tabindex="-1"></a>autompg <span class="ot">=</span> <span class="fu">read.table</span>(</span>
+<span id="cb724-3"><a href="categorical-predictors-and-interactions.html#cb724-3" tabindex="-1"></a>  <span class="st">&quot;http://archive.ics.uci.edu/ml/machine-learning-databases/auto-mpg/auto-mpg.data&quot;</span>,</span>
+<span id="cb724-4"><a href="categorical-predictors-and-interactions.html#cb724-4" tabindex="-1"></a>  <span class="at">quote =</span> <span class="st">&quot;</span><span class="sc">\&quot;</span><span class="st">&quot;</span>,</span>
+<span id="cb724-5"><a href="categorical-predictors-and-interactions.html#cb724-5" tabindex="-1"></a>  <span class="at">comment.char =</span> <span class="st">&quot;&quot;</span>,</span>
+<span id="cb724-6"><a href="categorical-predictors-and-interactions.html#cb724-6" tabindex="-1"></a>  <span class="at">stringsAsFactors =</span> <span class="cn">FALSE</span>)</span>
+<span id="cb724-7"><a href="categorical-predictors-and-interactions.html#cb724-7" tabindex="-1"></a><span class="co"># give the dataframe headers</span></span>
+<span id="cb724-8"><a href="categorical-predictors-and-interactions.html#cb724-8" tabindex="-1"></a><span class="fu">colnames</span>(autompg) <span class="ot">=</span> <span class="fu">c</span>(<span class="st">&quot;mpg&quot;</span>, <span class="st">&quot;cyl&quot;</span>, <span class="st">&quot;disp&quot;</span>, <span class="st">&quot;hp&quot;</span>, <span class="st">&quot;wt&quot;</span>, <span class="st">&quot;acc&quot;</span>, <span class="st">&quot;year&quot;</span>, <span class="st">&quot;origin&quot;</span>, <span class="st">&quot;name&quot;</span>)</span>
+<span id="cb724-9"><a href="categorical-predictors-and-interactions.html#cb724-9" tabindex="-1"></a><span class="co"># remove missing data, which is stored as &quot;?&quot;</span></span>
+<span id="cb724-10"><a href="categorical-predictors-and-interactions.html#cb724-10" tabindex="-1"></a>autompg <span class="ot">=</span> <span class="fu">subset</span>(autompg, autompg<span class="sc">$</span>hp <span class="sc">!=</span> <span class="st">&quot;?&quot;</span>)</span>
+<span id="cb724-11"><a href="categorical-predictors-and-interactions.html#cb724-11" tabindex="-1"></a><span class="co"># remove the plymouth reliant, as it causes some issues</span></span>
+<span id="cb724-12"><a href="categorical-predictors-and-interactions.html#cb724-12" tabindex="-1"></a>autompg <span class="ot">=</span> <span class="fu">subset</span>(autompg, autompg<span class="sc">$</span>name <span class="sc">!=</span> <span class="st">&quot;plymouth reliant&quot;</span>)</span>
+<span id="cb724-13"><a href="categorical-predictors-and-interactions.html#cb724-13" tabindex="-1"></a><span class="co"># give the dataset row names, based on the engine, year and name</span></span>
+<span id="cb724-14"><a href="categorical-predictors-and-interactions.html#cb724-14" tabindex="-1"></a><span class="fu">rownames</span>(autompg) <span class="ot">=</span> <span class="fu">paste</span>(autompg<span class="sc">$</span>cyl, <span class="st">&quot;cylinder&quot;</span>, autompg<span class="sc">$</span>year, autompg<span class="sc">$</span>name)</span>
+<span id="cb724-15"><a href="categorical-predictors-and-interactions.html#cb724-15" tabindex="-1"></a><span class="co"># remove the variable for name</span></span>
+<span id="cb724-16"><a href="categorical-predictors-and-interactions.html#cb724-16" tabindex="-1"></a>autompg <span class="ot">=</span> <span class="fu">subset</span>(autompg, <span class="at">select =</span> <span class="fu">c</span>(<span class="st">&quot;mpg&quot;</span>, <span class="st">&quot;cyl&quot;</span>, <span class="st">&quot;disp&quot;</span>, <span class="st">&quot;hp&quot;</span>, <span class="st">&quot;wt&quot;</span>, <span class="st">&quot;acc&quot;</span>, <span class="st">&quot;year&quot;</span>, <span class="st">&quot;origin&quot;</span>))</span>
+<span id="cb724-17"><a href="categorical-predictors-and-interactions.html#cb724-17" tabindex="-1"></a><span class="co"># change horsepower from character to numeric</span></span>
+<span id="cb724-18"><a href="categorical-predictors-and-interactions.html#cb724-18" tabindex="-1"></a>autompg<span class="sc">$</span>hp <span class="ot">=</span> <span class="fu">as.numeric</span>(autompg<span class="sc">$</span>hp)</span>
+<span id="cb724-19"><a href="categorical-predictors-and-interactions.html#cb724-19" tabindex="-1"></a><span class="co"># create a dummy variable for foreign vs domestic cars. domestic = 1.</span></span>
+<span id="cb724-20"><a href="categorical-predictors-and-interactions.html#cb724-20" tabindex="-1"></a>autompg<span class="sc">$</span>domestic <span class="ot">=</span> <span class="fu">as.numeric</span>(autompg<span class="sc">$</span>origin <span class="sc">==</span> <span class="dv">1</span>)</span>
+<span id="cb724-21"><a href="categorical-predictors-and-interactions.html#cb724-21" tabindex="-1"></a><span class="co"># remove 3 and 5 cylinder cars (which are very rare.)</span></span>
+<span id="cb724-22"><a href="categorical-predictors-and-interactions.html#cb724-22" tabindex="-1"></a>autompg <span class="ot">=</span> autompg[autompg<span class="sc">$</span>cyl <span class="sc">!=</span> <span class="dv">5</span>,]</span>
+<span id="cb724-23"><a href="categorical-predictors-and-interactions.html#cb724-23" tabindex="-1"></a>autompg <span class="ot">=</span> autompg[autompg<span class="sc">$</span>cyl <span class="sc">!=</span> <span class="dv">3</span>,]</span>
+<span id="cb724-24"><a href="categorical-predictors-and-interactions.html#cb724-24" tabindex="-1"></a><span class="co"># the following line would verify the remaining cylinder possibilities are 4, 6, 8</span></span>
+<span id="cb724-25"><a href="categorical-predictors-and-interactions.html#cb724-25" tabindex="-1"></a><span class="co">#unique(autompg$cyl)</span></span>
+<span id="cb724-26"><a href="categorical-predictors-and-interactions.html#cb724-26" tabindex="-1"></a><span class="co"># change cyl to a factor variable</span></span>
+<span id="cb724-27"><a href="categorical-predictors-and-interactions.html#cb724-27" tabindex="-1"></a>autompg<span class="sc">$</span>cyl <span class="ot">=</span> <span class="fu">as.factor</span>(autompg<span class="sc">$</span>cyl)</span></code></pre></div>
+<div class="sourceCode" id="cb725"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb725-1"><a href="categorical-predictors-and-interactions.html#cb725-1" tabindex="-1"></a><span class="fu">str</span>(autompg)</span></code></pre></div>
 <pre><code>## &#39;data.frame&#39;:    383 obs. of  9 variables:
 ##  $ mpg     : num  18 15 18 16 17 15 14 14 14 15 ...
 ##  $ cyl     : Factor w/ 3 levels &quot;4&quot;,&quot;6&quot;,&quot;8&quot;: 3 3 3 3 3 3 3 3 3 3 ...
@@ -690,18 +690,18 @@ <h2><span class="header-section-number">11.2</span> Interactions<a href="categor
   \end{cases}
 \]</span></p>
 <p>We will fit this model, extract the slope and intercept for the “two lines,” plot the data and add the lines.</p>
-<div class="sourceCode" id="cb727"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb727-1"><a href="categorical-predictors-and-interactions.html#cb727-1" aria-hidden="true" tabindex="-1"></a>mpg_disp_add <span class="ot">=</span> <span class="fu">lm</span>(mpg <span class="sc">~</span> disp <span class="sc">+</span> domestic, <span class="at">data =</span> autompg)</span>
-<span id="cb727-2"><a href="categorical-predictors-and-interactions.html#cb727-2" aria-hidden="true" tabindex="-1"></a></span>
-<span id="cb727-3"><a href="categorical-predictors-and-interactions.html#cb727-3" aria-hidden="true" tabindex="-1"></a>int_for <span class="ot">=</span> <span class="fu">coef</span>(mpg_disp_add)[<span class="dv">1</span>]</span>
-<span id="cb727-4"><a href="categorical-predictors-and-interactions.html#cb727-4" aria-hidden="true" tabindex="-1"></a>int_dom <span class="ot">=</span> <span class="fu">coef</span>(mpg_disp_add)[<span class="dv">1</span>] <span class="sc">+</span> <span class="fu">coef</span>(mpg_disp_add)[<span class="dv">3</span>]</span>
-<span id="cb727-5"><a href="categorical-predictors-and-interactions.html#cb727-5" aria-hidden="true" tabindex="-1"></a></span>
-<span id="cb727-6"><a href="categorical-predictors-and-interactions.html#cb727-6" aria-hidden="true" tabindex="-1"></a>slope_for <span class="ot">=</span> <span class="fu">coef</span>(mpg_disp_add)[<span class="dv">2</span>]</span>
-<span id="cb727-7"><a href="categorical-predictors-and-interactions.html#cb727-7" aria-hidden="true" tabindex="-1"></a>slope_dom <span class="ot">=</span> <span class="fu">coef</span>(mpg_disp_add)[<span class="dv">2</span>]</span>
-<span id="cb727-8"><a href="categorical-predictors-and-interactions.html#cb727-8" aria-hidden="true" tabindex="-1"></a></span>
-<span id="cb727-9"><a href="categorical-predictors-and-interactions.html#cb727-9" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(mpg <span class="sc">~</span> disp, <span class="at">data =</span> autompg, <span class="at">col =</span> domestic <span class="sc">+</span> <span class="dv">1</span>, <span class="at">pch =</span> domestic <span class="sc">+</span> <span class="dv">1</span>)</span>
-<span id="cb727-10"><a href="categorical-predictors-and-interactions.html#cb727-10" aria-hidden="true" tabindex="-1"></a><span class="fu">abline</span>(int_for, slope_for, <span class="at">col =</span> <span class="dv">1</span>, <span class="at">lty =</span> <span class="dv">1</span>, <span class="at">lwd =</span> <span class="dv">2</span>) <span class="co"># add line for foreign cars</span></span>
-<span id="cb727-11"><a href="categorical-predictors-and-interactions.html#cb727-11" aria-hidden="true" tabindex="-1"></a><span class="fu">abline</span>(int_dom, slope_dom, <span class="at">col =</span> <span class="dv">2</span>, <span class="at">lty =</span> <span class="dv">2</span>, <span class="at">lwd =</span> <span class="dv">2</span>) <span class="co"># add line for domestic cars</span></span>
-<span id="cb727-12"><a href="categorical-predictors-and-interactions.html#cb727-12" aria-hidden="true" tabindex="-1"></a><span class="fu">legend</span>(<span class="st">&quot;topright&quot;</span>, <span class="fu">c</span>(<span class="st">&quot;Foreign&quot;</span>, <span class="st">&quot;Domestic&quot;</span>), <span class="at">pch =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>), <span class="at">col =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>))</span></code></pre></div>
+<div class="sourceCode" id="cb727"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb727-1"><a href="categorical-predictors-and-interactions.html#cb727-1" tabindex="-1"></a>mpg_disp_add <span class="ot">=</span> <span class="fu">lm</span>(mpg <span class="sc">~</span> disp <span class="sc">+</span> domestic, <span class="at">data =</span> autompg)</span>
+<span id="cb727-2"><a href="categorical-predictors-and-interactions.html#cb727-2" tabindex="-1"></a></span>
+<span id="cb727-3"><a href="categorical-predictors-and-interactions.html#cb727-3" tabindex="-1"></a>int_for <span class="ot">=</span> <span class="fu">coef</span>(mpg_disp_add)[<span class="dv">1</span>]</span>
+<span id="cb727-4"><a href="categorical-predictors-and-interactions.html#cb727-4" tabindex="-1"></a>int_dom <span class="ot">=</span> <span class="fu">coef</span>(mpg_disp_add)[<span class="dv">1</span>] <span class="sc">+</span> <span class="fu">coef</span>(mpg_disp_add)[<span class="dv">3</span>]</span>
+<span id="cb727-5"><a href="categorical-predictors-and-interactions.html#cb727-5" tabindex="-1"></a></span>
+<span id="cb727-6"><a href="categorical-predictors-and-interactions.html#cb727-6" tabindex="-1"></a>slope_for <span class="ot">=</span> <span class="fu">coef</span>(mpg_disp_add)[<span class="dv">2</span>]</span>
+<span id="cb727-7"><a href="categorical-predictors-and-interactions.html#cb727-7" tabindex="-1"></a>slope_dom <span class="ot">=</span> <span class="fu">coef</span>(mpg_disp_add)[<span class="dv">2</span>]</span>
+<span id="cb727-8"><a href="categorical-predictors-and-interactions.html#cb727-8" tabindex="-1"></a></span>
+<span id="cb727-9"><a href="categorical-predictors-and-interactions.html#cb727-9" tabindex="-1"></a><span class="fu">plot</span>(mpg <span class="sc">~</span> disp, <span class="at">data =</span> autompg, <span class="at">col =</span> domestic <span class="sc">+</span> <span class="dv">1</span>, <span class="at">pch =</span> domestic <span class="sc">+</span> <span class="dv">1</span>)</span>
+<span id="cb727-10"><a href="categorical-predictors-and-interactions.html#cb727-10" tabindex="-1"></a><span class="fu">abline</span>(int_for, slope_for, <span class="at">col =</span> <span class="dv">1</span>, <span class="at">lty =</span> <span class="dv">1</span>, <span class="at">lwd =</span> <span class="dv">2</span>) <span class="co"># add line for foreign cars</span></span>
+<span id="cb727-11"><a href="categorical-predictors-and-interactions.html#cb727-11" tabindex="-1"></a><span class="fu">abline</span>(int_dom, slope_dom, <span class="at">col =</span> <span class="dv">2</span>, <span class="at">lty =</span> <span class="dv">2</span>, <span class="at">lwd =</span> <span class="dv">2</span>) <span class="co"># add line for domestic cars</span></span>
+<span id="cb727-12"><a href="categorical-predictors-and-interactions.html#cb727-12" tabindex="-1"></a><span class="fu">legend</span>(<span class="st">&quot;topright&quot;</span>, <span class="fu">c</span>(<span class="st">&quot;Foreign&quot;</span>, <span class="st">&quot;Domestic&quot;</span>), <span class="at">pch =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>), <span class="at">col =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>))</span></code></pre></div>
 <p><img src="cat-int_files/figure-html/unnamed-chunk-15-1.png" width="672" style="display: block; margin: auto;" /></p>
 <p>This is a model that allows for two <em>parallel</em> lines, meaning the <code>mpg</code> can be different on average between foreign and domestic cars of the same engine displacement, but the change in average <code>mpg</code> for an increase in displacement is the same for both. We can see this model isn’t doing very well here. The red line fits the red points fairly well, but the black line isn’t doing very well for the black points, it should clearly have a more negative slope. Essentially, we would like a model that allows for two different slopes.</p>
 <p>Consider the following model,</p>
@@ -727,21 +727,21 @@ <h2><span class="header-section-number">11.2</span> Interactions<a href="categor
 </ul>
 <p>How do we fit this model in <code>R</code>? There are a number of ways.</p>
 <p>One method would be to simply create a new variable, then fit a model like any other.</p>
-<div class="sourceCode" id="cb728"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb728-1"><a href="categorical-predictors-and-interactions.html#cb728-1" aria-hidden="true" tabindex="-1"></a>autompg<span class="sc">$</span>x3 <span class="ot">=</span> autompg<span class="sc">$</span>disp <span class="sc">*</span> autompg<span class="sc">$</span>domestic <span class="co"># THIS CODE NOT RUN!</span></span>
-<span id="cb728-2"><a href="categorical-predictors-and-interactions.html#cb728-2" aria-hidden="true" tabindex="-1"></a>do_not_do_this <span class="ot">=</span> <span class="fu">lm</span>(mpg <span class="sc">~</span> disp <span class="sc">+</span> domestic <span class="sc">+</span> x3, <span class="at">data =</span> autompg) <span class="co"># THIS CODE NOT RUN!</span></span></code></pre></div>
+<div class="sourceCode" id="cb728"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb728-1"><a href="categorical-predictors-and-interactions.html#cb728-1" tabindex="-1"></a>autompg<span class="sc">$</span>x3 <span class="ot">=</span> autompg<span class="sc">$</span>disp <span class="sc">*</span> autompg<span class="sc">$</span>domestic <span class="co"># THIS CODE NOT RUN!</span></span>
+<span id="cb728-2"><a href="categorical-predictors-and-interactions.html#cb728-2" tabindex="-1"></a>do_not_do_this <span class="ot">=</span> <span class="fu">lm</span>(mpg <span class="sc">~</span> disp <span class="sc">+</span> domestic <span class="sc">+</span> x3, <span class="at">data =</span> autompg) <span class="co"># THIS CODE NOT RUN!</span></span></code></pre></div>
 <p>You should only do this as a last resort. We greatly prefer not to have to modify our data simply to fit a model. Instead, we can tell <code>R</code> we would like to use the existing data with an interaction term, which it will create automatically when we use the <code>:</code> operator.</p>
-<div class="sourceCode" id="cb729"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb729-1"><a href="categorical-predictors-and-interactions.html#cb729-1" aria-hidden="true" tabindex="-1"></a>mpg_disp_int <span class="ot">=</span> <span class="fu">lm</span>(mpg <span class="sc">~</span> disp <span class="sc">+</span> domestic <span class="sc">+</span> disp<span class="sc">:</span>domestic, <span class="at">data =</span> autompg)</span></code></pre></div>
+<div class="sourceCode" id="cb729"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb729-1"><a href="categorical-predictors-and-interactions.html#cb729-1" tabindex="-1"></a>mpg_disp_int <span class="ot">=</span> <span class="fu">lm</span>(mpg <span class="sc">~</span> disp <span class="sc">+</span> domestic <span class="sc">+</span> disp<span class="sc">:</span>domestic, <span class="at">data =</span> autompg)</span></code></pre></div>
 <p>An alternative method, which will fit the exact same model as above would be to use the <code>*</code> operator. This method automatically creates the interaction term, as well as any “lower order terms,” which in this case are the first order terms for <code>disp</code> and <code>domestic</code></p>
-<div class="sourceCode" id="cb730"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb730-1"><a href="categorical-predictors-and-interactions.html#cb730-1" aria-hidden="true" tabindex="-1"></a>mpg_disp_int2 <span class="ot">=</span> <span class="fu">lm</span>(mpg <span class="sc">~</span> disp <span class="sc">*</span> domestic, <span class="at">data =</span> autompg)</span></code></pre></div>
+<div class="sourceCode" id="cb730"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb730-1"><a href="categorical-predictors-and-interactions.html#cb730-1" tabindex="-1"></a>mpg_disp_int2 <span class="ot">=</span> <span class="fu">lm</span>(mpg <span class="sc">~</span> disp <span class="sc">*</span> domestic, <span class="at">data =</span> autompg)</span></code></pre></div>
 <p>We can quickly verify that these are doing the same thing.</p>
-<div class="sourceCode" id="cb731"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb731-1"><a href="categorical-predictors-and-interactions.html#cb731-1" aria-hidden="true" tabindex="-1"></a><span class="fu">coef</span>(mpg_disp_int)</span></code></pre></div>
+<div class="sourceCode" id="cb731"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb731-1"><a href="categorical-predictors-and-interactions.html#cb731-1" tabindex="-1"></a><span class="fu">coef</span>(mpg_disp_int)</span></code></pre></div>
 <pre><code>##   (Intercept)          disp      domestic disp:domestic 
 ##    46.0548423    -0.1569239   -12.5754714     0.1025184</code></pre>
-<div class="sourceCode" id="cb733"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb733-1"><a href="categorical-predictors-and-interactions.html#cb733-1" aria-hidden="true" tabindex="-1"></a><span class="fu">coef</span>(mpg_disp_int2)</span></code></pre></div>
+<div class="sourceCode" id="cb733"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb733-1"><a href="categorical-predictors-and-interactions.html#cb733-1" tabindex="-1"></a><span class="fu">coef</span>(mpg_disp_int2)</span></code></pre></div>
 <pre><code>##   (Intercept)          disp      domestic disp:domestic 
 ##    46.0548423    -0.1569239   -12.5754714     0.1025184</code></pre>
 <p>We see that both the variables, and their coefficient estimates are indeed the same for both models.</p>
-<div class="sourceCode" id="cb735"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb735-1"><a href="categorical-predictors-and-interactions.html#cb735-1" aria-hidden="true" tabindex="-1"></a><span class="fu">summary</span>(mpg_disp_int)</span></code></pre></div>
+<div class="sourceCode" id="cb735"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb735-1"><a href="categorical-predictors-and-interactions.html#cb735-1" tabindex="-1"></a><span class="fu">summary</span>(mpg_disp_int)</span></code></pre></div>
 <pre><code>## 
 ## Call:
 ## lm(formula = mpg ~ disp + domestic + disp:domestic, data = autompg)
@@ -768,27 +768,27 @@ <h2><span class="header-section-number">11.2</span> Interactions<a href="categor
 \]</span></p>
 <p>In this case, testing for <span class="math inline">\(\beta_3 = 0\)</span> is testing for two lines with parallel slopes versus two lines with possibly different slopes. The <code>disp:domestic</code> line in the <code>summary()</code> output uses a <span class="math inline">\(t\)</span>-test to perform the test.</p>
 <p>We could also use an ANOVA <span class="math inline">\(F\)</span>-test. The additive model without interaction is our null model, and the interaction model is the alternative.</p>
-<div class="sourceCode" id="cb737"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb737-1"><a href="categorical-predictors-and-interactions.html#cb737-1" aria-hidden="true" tabindex="-1"></a><span class="fu">anova</span>(mpg_disp_add, mpg_disp_int)</span></code></pre></div>
+<div class="sourceCode" id="cb737"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb737-1"><a href="categorical-predictors-and-interactions.html#cb737-1" tabindex="-1"></a><span class="fu">anova</span>(mpg_disp_add, mpg_disp_int)</span></code></pre></div>
 <pre><code>## Analysis of Variance Table
 ## 
 ## Model 1: mpg ~ disp + domestic
 ## Model 2: mpg ~ disp + domestic + disp:domestic
-##   Res.Df    RSS Df Sum of Sq      F    Pr(&gt;F)    
-## 1    380 7714.0                                  
-## 2    379 7032.6  1    681.36 36.719 3.294e-09 ***
+##   Res.Df    RSS Df Sum of Sq     F    Pr(&gt;F)    
+## 1    380 7714.0                                 
+## 2    379 7032.6  1    681.36 36.72 3.294e-09 ***
 ## ---
 ## Signif. codes:  0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1</code></pre>
 <p>Again we see this test has the same p-value as the <span class="math inline">\(t\)</span>-test. Also the p-value is extremely low, so between the two, we choose the interaction model.</p>
-<div class="sourceCode" id="cb739"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb739-1"><a href="categorical-predictors-and-interactions.html#cb739-1" aria-hidden="true" tabindex="-1"></a>int_for <span class="ot">=</span> <span class="fu">coef</span>(mpg_disp_int)[<span class="dv">1</span>]</span>
-<span id="cb739-2"><a href="categorical-predictors-and-interactions.html#cb739-2" aria-hidden="true" tabindex="-1"></a>int_dom <span class="ot">=</span> <span class="fu">coef</span>(mpg_disp_int)[<span class="dv">1</span>] <span class="sc">+</span> <span class="fu">coef</span>(mpg_disp_int)[<span class="dv">3</span>]</span>
-<span id="cb739-3"><a href="categorical-predictors-and-interactions.html#cb739-3" aria-hidden="true" tabindex="-1"></a></span>
-<span id="cb739-4"><a href="categorical-predictors-and-interactions.html#cb739-4" aria-hidden="true" tabindex="-1"></a>slope_for <span class="ot">=</span> <span class="fu">coef</span>(mpg_disp_int)[<span class="dv">2</span>]</span>
-<span id="cb739-5"><a href="categorical-predictors-and-interactions.html#cb739-5" aria-hidden="true" tabindex="-1"></a>slope_dom <span class="ot">=</span> <span class="fu">coef</span>(mpg_disp_int)[<span class="dv">2</span>] <span class="sc">+</span> <span class="fu">coef</span>(mpg_disp_int)[<span class="dv">4</span>]</span></code></pre></div>
+<div class="sourceCode" id="cb739"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb739-1"><a href="categorical-predictors-and-interactions.html#cb739-1" tabindex="-1"></a>int_for <span class="ot">=</span> <span class="fu">coef</span>(mpg_disp_int)[<span class="dv">1</span>]</span>
+<span id="cb739-2"><a href="categorical-predictors-and-interactions.html#cb739-2" tabindex="-1"></a>int_dom <span class="ot">=</span> <span class="fu">coef</span>(mpg_disp_int)[<span class="dv">1</span>] <span class="sc">+</span> <span class="fu">coef</span>(mpg_disp_int)[<span class="dv">3</span>]</span>
+<span id="cb739-3"><a href="categorical-predictors-and-interactions.html#cb739-3" tabindex="-1"></a></span>
+<span id="cb739-4"><a href="categorical-predictors-and-interactions.html#cb739-4" tabindex="-1"></a>slope_for <span class="ot">=</span> <span class="fu">coef</span>(mpg_disp_int)[<span class="dv">2</span>]</span>
+<span id="cb739-5"><a href="categorical-predictors-and-interactions.html#cb739-5" tabindex="-1"></a>slope_dom <span class="ot">=</span> <span class="fu">coef</span>(mpg_disp_int)[<span class="dv">2</span>] <span class="sc">+</span> <span class="fu">coef</span>(mpg_disp_int)[<span class="dv">4</span>]</span></code></pre></div>
 <p>Here we again calculate the slope and intercepts for the two lines for use in plotting.</p>
-<div class="sourceCode" id="cb740"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb740-1"><a href="categorical-predictors-and-interactions.html#cb740-1" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(mpg <span class="sc">~</span> disp, <span class="at">data =</span> autompg, <span class="at">col =</span> domestic <span class="sc">+</span> <span class="dv">1</span>, <span class="at">pch =</span> domestic <span class="sc">+</span> <span class="dv">1</span>)</span>
-<span id="cb740-2"><a href="categorical-predictors-and-interactions.html#cb740-2" aria-hidden="true" tabindex="-1"></a><span class="fu">abline</span>(int_for, slope_for, <span class="at">col =</span> <span class="dv">1</span>, <span class="at">lty =</span> <span class="dv">1</span>, <span class="at">lwd =</span> <span class="dv">2</span>) <span class="co"># line for foreign cars</span></span>
-<span id="cb740-3"><a href="categorical-predictors-and-interactions.html#cb740-3" aria-hidden="true" tabindex="-1"></a><span class="fu">abline</span>(int_dom, slope_dom, <span class="at">col =</span> <span class="dv">2</span>, <span class="at">lty =</span> <span class="dv">2</span>, <span class="at">lwd =</span> <span class="dv">2</span>) <span class="co"># line for domestic cars</span></span>
-<span id="cb740-4"><a href="categorical-predictors-and-interactions.html#cb740-4" aria-hidden="true" tabindex="-1"></a><span class="fu">legend</span>(<span class="st">&quot;topright&quot;</span>, <span class="fu">c</span>(<span class="st">&quot;Foreign&quot;</span>, <span class="st">&quot;Domestic&quot;</span>), <span class="at">pch =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>), <span class="at">col =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>))</span></code></pre></div>
+<div class="sourceCode" id="cb740"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb740-1"><a href="categorical-predictors-and-interactions.html#cb740-1" tabindex="-1"></a><span class="fu">plot</span>(mpg <span class="sc">~</span> disp, <span class="at">data =</span> autompg, <span class="at">col =</span> domestic <span class="sc">+</span> <span class="dv">1</span>, <span class="at">pch =</span> domestic <span class="sc">+</span> <span class="dv">1</span>)</span>
+<span id="cb740-2"><a href="categorical-predictors-and-interactions.html#cb740-2" tabindex="-1"></a><span class="fu">abline</span>(int_for, slope_for, <span class="at">col =</span> <span class="dv">1</span>, <span class="at">lty =</span> <span class="dv">1</span>, <span class="at">lwd =</span> <span class="dv">2</span>) <span class="co"># line for foreign cars</span></span>
+<span id="cb740-3"><a href="categorical-predictors-and-interactions.html#cb740-3" tabindex="-1"></a><span class="fu">abline</span>(int_dom, slope_dom, <span class="at">col =</span> <span class="dv">2</span>, <span class="at">lty =</span> <span class="dv">2</span>, <span class="at">lwd =</span> <span class="dv">2</span>) <span class="co"># line for domestic cars</span></span>
+<span id="cb740-4"><a href="categorical-predictors-and-interactions.html#cb740-4" tabindex="-1"></a><span class="fu">legend</span>(<span class="st">&quot;topright&quot;</span>, <span class="fu">c</span>(<span class="st">&quot;Foreign&quot;</span>, <span class="st">&quot;Domestic&quot;</span>), <span class="at">pch =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>), <span class="at">col =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>))</span></code></pre></div>
 <p><img src="cat-int_files/figure-html/unnamed-chunk-23-1.png" width="672" style="display: block; margin: auto;" /></p>
 <p>We see that these lines fit the data much better, which matches the result of our tests.</p>
 <p>So far we have only seen interaction between a categorical variable (<code>domestic</code>) and a numerical variable (<code>disp</code>). While this is easy to visualize, since it allows for different slopes for two lines, it is not the only type of interaction we can use in a model. We can also consider interactions between two numerical variables.</p>
@@ -808,9 +808,9 @@ <h2><span class="header-section-number">11.2</span> Interactions<a href="categor
 \]</span></p>
 <p>So, for a one unit increase in <span class="math inline">\(x_1\)</span> (<code>disp</code>), the mean of <span class="math inline">\(Y\)</span> (<code>mpg</code>) increases <span class="math inline">\(\beta_1 + \beta_3 x_2\)</span>, which is a different value depending on the value of <span class="math inline">\(x_2\)</span> (<code>hp</code>)!</p>
 <p>Since we’re now working in three dimensions, this model can’t be easily justified via visualizations like the previous example. Instead, we will have to rely on a test.</p>
-<div class="sourceCode" id="cb741"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb741-1"><a href="categorical-predictors-and-interactions.html#cb741-1" aria-hidden="true" tabindex="-1"></a>mpg_disp_add_hp <span class="ot">=</span> <span class="fu">lm</span>(mpg <span class="sc">~</span> disp <span class="sc">+</span> hp, <span class="at">data =</span> autompg)</span>
-<span id="cb741-2"><a href="categorical-predictors-and-interactions.html#cb741-2" aria-hidden="true" tabindex="-1"></a>mpg_disp_int_hp <span class="ot">=</span> <span class="fu">lm</span>(mpg <span class="sc">~</span> disp <span class="sc">*</span> hp, <span class="at">data =</span> autompg)</span>
-<span id="cb741-3"><a href="categorical-predictors-and-interactions.html#cb741-3" aria-hidden="true" tabindex="-1"></a><span class="fu">summary</span>(mpg_disp_int_hp)</span></code></pre></div>
+<div class="sourceCode" id="cb741"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb741-1"><a href="categorical-predictors-and-interactions.html#cb741-1" tabindex="-1"></a>mpg_disp_add_hp <span class="ot">=</span> <span class="fu">lm</span>(mpg <span class="sc">~</span> disp <span class="sc">+</span> hp, <span class="at">data =</span> autompg)</span>
+<span id="cb741-2"><a href="categorical-predictors-and-interactions.html#cb741-2" tabindex="-1"></a>mpg_disp_int_hp <span class="ot">=</span> <span class="fu">lm</span>(mpg <span class="sc">~</span> disp <span class="sc">*</span> hp, <span class="at">data =</span> autompg)</span>
+<span id="cb741-3"><a href="categorical-predictors-and-interactions.html#cb741-3" tabindex="-1"></a><span class="fu">summary</span>(mpg_disp_int_hp)</span></code></pre></div>
 <pre><code>## 
 ## Call:
 ## lm(formula = mpg ~ disp * hp, data = autompg)
@@ -836,7 +836,7 @@ <h2><span class="header-section-number">11.2</span> Interactions<a href="categor
 H_0: \beta_3 = 0.
 \]</span></p>
 <p>Again, we see a very low p-value so we reject the null (additive model) in favor of the interaction model. Again, there is an equivalent <span class="math inline">\(F\)</span>-test.</p>
-<div class="sourceCode" id="cb743"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb743-1"><a href="categorical-predictors-and-interactions.html#cb743-1" aria-hidden="true" tabindex="-1"></a><span class="fu">anova</span>(mpg_disp_add_hp, mpg_disp_int_hp)</span></code></pre></div>
+<div class="sourceCode" id="cb743"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb743-1"><a href="categorical-predictors-and-interactions.html#cb743-1" tabindex="-1"></a><span class="fu">anova</span>(mpg_disp_add_hp, mpg_disp_int_hp)</span></code></pre></div>
 <pre><code>## Analysis of Variance Table
 ## 
 ## Model 1: mpg ~ disp + hp
@@ -847,7 +847,7 @@ <h2><span class="header-section-number">11.2</span> Interactions<a href="categor
 ## ---
 ## Signif. codes:  0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1</code></pre>
 <p>We can take a closer look at the coefficients of our fitted interaction model.</p>
-<div class="sourceCode" id="cb745"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb745-1"><a href="categorical-predictors-and-interactions.html#cb745-1" aria-hidden="true" tabindex="-1"></a><span class="fu">coef</span>(mpg_disp_int_hp)</span></code></pre></div>
+<div class="sourceCode" id="cb745"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb745-1"><a href="categorical-predictors-and-interactions.html#cb745-1" tabindex="-1"></a><span class="fu">coef</span>(mpg_disp_int_hp)</span></code></pre></div>
 <pre><code>##   (Intercept)          disp            hp       disp:hp 
 ## 52.4081997848 -0.1001737655 -0.2198199720  0.0005658269</code></pre>
 <ul>
@@ -879,20 +879,20 @@ <h2><span class="header-section-number">11.2</span> Interactions<a href="categor
 <h2><span class="header-section-number">11.3</span> Factor Variables<a href="categorical-predictors-and-interactions.html#factor-variables" class="anchor-section" aria-label="Anchor link to header"></a></h2>
 <p>So far in this chapter, we have limited our use of categorical variables to binary categorical variables. Specifically, we have limited ourselves to dummy variables which take a value of <code>0</code> or <code>1</code> and represent a categorical variable numerically.</p>
 <p>We will now discuss <strong>factor</strong> variables, which is a special way that <code>R</code> deals with categorical variables. With factor variables, a human user can simply think about the categories of a variable, and <code>R</code> will take care of the necessary dummy variables without any 0/1 assignment being done by the user.</p>
-<div class="sourceCode" id="cb747"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb747-1"><a href="categorical-predictors-and-interactions.html#cb747-1" aria-hidden="true" tabindex="-1"></a><span class="fu">is.factor</span>(autompg<span class="sc">$</span>domestic)</span></code></pre></div>
+<div class="sourceCode" id="cb747"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb747-1"><a href="categorical-predictors-and-interactions.html#cb747-1" tabindex="-1"></a><span class="fu">is.factor</span>(autompg<span class="sc">$</span>domestic)</span></code></pre></div>
 <pre><code>## [1] FALSE</code></pre>
 <p>Earlier when we used the <code>domestic</code> variable, it was <strong>not</strong> a factor variable. It was simply a numerical variable that only took two possible values, <code>1</code> for domestic, and <code>0</code> for foreign. Let’s create a new variable <code>origin</code> that stores the same information, but in a different way.</p>
-<div class="sourceCode" id="cb749"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb749-1"><a href="categorical-predictors-and-interactions.html#cb749-1" aria-hidden="true" tabindex="-1"></a>autompg<span class="sc">$</span>origin[autompg<span class="sc">$</span>domestic <span class="sc">==</span> <span class="dv">1</span>] <span class="ot">=</span> <span class="st">&quot;domestic&quot;</span></span>
-<span id="cb749-2"><a href="categorical-predictors-and-interactions.html#cb749-2" aria-hidden="true" tabindex="-1"></a>autompg<span class="sc">$</span>origin[autompg<span class="sc">$</span>domestic <span class="sc">==</span> <span class="dv">0</span>] <span class="ot">=</span> <span class="st">&quot;foreign&quot;</span></span>
-<span id="cb749-3"><a href="categorical-predictors-and-interactions.html#cb749-3" aria-hidden="true" tabindex="-1"></a><span class="fu">head</span>(autompg<span class="sc">$</span>origin)</span></code></pre></div>
+<div class="sourceCode" id="cb749"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb749-1"><a href="categorical-predictors-and-interactions.html#cb749-1" tabindex="-1"></a>autompg<span class="sc">$</span>origin[autompg<span class="sc">$</span>domestic <span class="sc">==</span> <span class="dv">1</span>] <span class="ot">=</span> <span class="st">&quot;domestic&quot;</span></span>
+<span id="cb749-2"><a href="categorical-predictors-and-interactions.html#cb749-2" tabindex="-1"></a>autompg<span class="sc">$</span>origin[autompg<span class="sc">$</span>domestic <span class="sc">==</span> <span class="dv">0</span>] <span class="ot">=</span> <span class="st">&quot;foreign&quot;</span></span>
+<span id="cb749-3"><a href="categorical-predictors-and-interactions.html#cb749-3" tabindex="-1"></a><span class="fu">head</span>(autompg<span class="sc">$</span>origin)</span></code></pre></div>
 <pre><code>## [1] &quot;domestic&quot; &quot;domestic&quot; &quot;domestic&quot; &quot;domestic&quot; &quot;domestic&quot; &quot;domestic&quot;</code></pre>
 <p>Now the <code>origin</code> variable stores <code>"domestic"</code> for domestic cars and <code>"foreign"</code> for foreign cars.</p>
-<div class="sourceCode" id="cb751"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb751-1"><a href="categorical-predictors-and-interactions.html#cb751-1" aria-hidden="true" tabindex="-1"></a><span class="fu">is.factor</span>(autompg<span class="sc">$</span>origin)</span></code></pre></div>
+<div class="sourceCode" id="cb751"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb751-1"><a href="categorical-predictors-and-interactions.html#cb751-1" tabindex="-1"></a><span class="fu">is.factor</span>(autompg<span class="sc">$</span>origin)</span></code></pre></div>
 <pre><code>## [1] FALSE</code></pre>
 <p>However, this is simply a vector of character values. A vector of car models is a character variable in <code>R</code>. A vector of Vehicle Identification Numbers (VINs) is a character variable as well. But those don’t represent a short list of levels that might influence a response variable. We will want to <strong>coerce</strong> this origin variable to be something more: a factor variable.</p>
-<div class="sourceCode" id="cb753"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb753-1"><a href="categorical-predictors-and-interactions.html#cb753-1" aria-hidden="true" tabindex="-1"></a>autompg<span class="sc">$</span>origin <span class="ot">=</span> <span class="fu">as.factor</span>(autompg<span class="sc">$</span>origin)</span></code></pre></div>
+<div class="sourceCode" id="cb753"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb753-1"><a href="categorical-predictors-and-interactions.html#cb753-1" tabindex="-1"></a>autompg<span class="sc">$</span>origin <span class="ot">=</span> <span class="fu">as.factor</span>(autompg<span class="sc">$</span>origin)</span></code></pre></div>
 <p>Now when we check the structure of the <code>autompg</code> dataset, we see that <code>origin</code> is a factor variable.</p>
-<div class="sourceCode" id="cb754"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb754-1"><a href="categorical-predictors-and-interactions.html#cb754-1" aria-hidden="true" tabindex="-1"></a><span class="fu">str</span>(autompg)</span></code></pre></div>
+<div class="sourceCode" id="cb754"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb754-1"><a href="categorical-predictors-and-interactions.html#cb754-1" tabindex="-1"></a><span class="fu">str</span>(autompg)</span></code></pre></div>
 <pre><code>## &#39;data.frame&#39;:    383 obs. of  9 variables:
 ##  $ mpg     : num  18 15 18 16 17 15 14 14 14 15 ...
 ##  $ cyl     : Factor w/ 3 levels &quot;4&quot;,&quot;6&quot;,&quot;8&quot;: 3 3 3 3 3 3 3 3 3 3 ...
@@ -904,7 +904,7 @@ <h2><span class="header-section-number">11.3</span> Factor Variables<a href="cat
 ##  $ origin  : Factor w/ 2 levels &quot;domestic&quot;,&quot;foreign&quot;: 1 1 1 1 1 1 1 1 1 1 ...
 ##  $ domestic: num  1 1 1 1 1 1 1 1 1 1 ...</code></pre>
 <p>Factor variables have <strong>levels</strong> which are the possible values (categories) that the variable may take, in this case foreign or domestic.</p>
-<div class="sourceCode" id="cb756"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb756-1"><a href="categorical-predictors-and-interactions.html#cb756-1" aria-hidden="true" tabindex="-1"></a><span class="fu">levels</span>(autompg<span class="sc">$</span>origin)</span></code></pre></div>
+<div class="sourceCode" id="cb756"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb756-1"><a href="categorical-predictors-and-interactions.html#cb756-1" tabindex="-1"></a><span class="fu">levels</span>(autompg<span class="sc">$</span>origin)</span></code></pre></div>
 <pre><code>## [1] &quot;domestic&quot; &quot;foreign&quot;</code></pre>
 <p>Recall that previously we have fit the model</p>
 <p><span class="math display">\[
@@ -916,7 +916,7 @@ <h2><span class="header-section-number">11.3</span> Factor Variables<a href="cat
 <li><span class="math inline">\(x_1\)</span> is <code>disp</code>, the displacement in cubic inches,</li>
 <li><span class="math inline">\(x_2\)</span> is <code>domestic</code>, a dummy variable where <code>1</code> indicates a domestic car.</li>
 </ul>
-<div class="sourceCode" id="cb758"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb758-1"><a href="categorical-predictors-and-interactions.html#cb758-1" aria-hidden="true" tabindex="-1"></a>(<span class="at">mod_dummy =</span> <span class="fu">lm</span>(mpg <span class="sc">~</span> disp <span class="sc">*</span> domestic, <span class="at">data =</span> autompg))</span></code></pre></div>
+<div class="sourceCode" id="cb758"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb758-1"><a href="categorical-predictors-and-interactions.html#cb758-1" tabindex="-1"></a>(<span class="at">mod_dummy =</span> <span class="fu">lm</span>(mpg <span class="sc">~</span> disp <span class="sc">*</span> domestic, <span class="at">data =</span> autompg))</span></code></pre></div>
 <pre><code>## 
 ## Call:
 ## lm(formula = mpg ~ disp * domestic, data = autompg)
@@ -930,7 +930,7 @@ <h2><span class="header-section-number">11.3</span> Factor Variables<a href="cat
 \]</span></p>
 <p>is the estimated average <code>mpg</code> for a <strong>domestic</strong> car with 0 <code>disp</code>.</p>
 <p>Now let’s try to do the same, but using our new factor variable.</p>
-<div class="sourceCode" id="cb760"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb760-1"><a href="categorical-predictors-and-interactions.html#cb760-1" aria-hidden="true" tabindex="-1"></a>(<span class="at">mod_factor =</span> <span class="fu">lm</span>(mpg <span class="sc">~</span> disp <span class="sc">*</span> origin, <span class="at">data =</span> autompg))</span></code></pre></div>
+<div class="sourceCode" id="cb760"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb760-1"><a href="categorical-predictors-and-interactions.html#cb760-1" tabindex="-1"></a>(<span class="at">mod_factor =</span> <span class="fu">lm</span>(mpg <span class="sc">~</span> disp <span class="sc">*</span> origin, <span class="at">data =</span> autompg))</span></code></pre></div>
 <pre><code>## 
 ## Call:
 ## lm(formula = mpg ~ disp * origin, data = autompg)
@@ -960,9 +960,9 @@ <h2><span class="header-section-number">11.3</span> Factor Variables<a href="cat
 <div id="factors-with-more-than-two-levels" class="section level3 hasAnchor" number="11.3.1">
 <h3><span class="header-section-number">11.3.1</span> Factors with More Than Two Levels<a href="categorical-predictors-and-interactions.html#factors-with-more-than-two-levels" class="anchor-section" aria-label="Anchor link to header"></a></h3>
 <p>Let’s now consider a factor variable with more than two levels. In this dataset, <code>cyl</code> is an example.</p>
-<div class="sourceCode" id="cb762"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb762-1"><a href="categorical-predictors-and-interactions.html#cb762-1" aria-hidden="true" tabindex="-1"></a><span class="fu">is.factor</span>(autompg<span class="sc">$</span>cyl)</span></code></pre></div>
+<div class="sourceCode" id="cb762"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb762-1"><a href="categorical-predictors-and-interactions.html#cb762-1" tabindex="-1"></a><span class="fu">is.factor</span>(autompg<span class="sc">$</span>cyl)</span></code></pre></div>
 <pre><code>## [1] TRUE</code></pre>
-<div class="sourceCode" id="cb764"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb764-1"><a href="categorical-predictors-and-interactions.html#cb764-1" aria-hidden="true" tabindex="-1"></a><span class="fu">levels</span>(autompg<span class="sc">$</span>cyl)</span></code></pre></div>
+<div class="sourceCode" id="cb764"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb764-1"><a href="categorical-predictors-and-interactions.html#cb764-1" tabindex="-1"></a><span class="fu">levels</span>(autompg<span class="sc">$</span>cyl)</span></code></pre></div>
 <pre><code>## [1] &quot;4&quot; &quot;6&quot; &quot;8&quot;</code></pre>
 <p>Here the <code>cyl</code> variable has three possible levels: <code>4</code>, <code>6</code>, and <code>8</code>. You may wonder, why not simply use <code>cyl</code> as a numerical variable? You certainly could.</p>
 <p>However, that would force the difference in average <code>mpg</code> between <code>4</code> and <code>6</code> cylinders to be the same as the difference in average mpg between <code>6</code> and <code>8</code> cylinders. That usually makes sense for a continuous variable, but not for a discrete variable with so few possible values. In the case of this variable, there is no such thing as a 7-cylinder engine or a 6.23-cylinder engine in personal vehicles. For these reasons, we will simply consider <code>cyl</code> to be categorical. This is a decision that will commonly need to be made with ordinal variables. Often, with a large number of categories, the decision to treat them as numerical variables is appropriate because, otherwise, a large number of dummy variables are then needed to represent these variables.</p>
@@ -989,7 +989,7 @@ <h3><span class="header-section-number">11.3.1</span> Factors with More Than Two
   \end{cases}
 \]</span></p>
 <p>Now, let’s fit an additive model in <code>R</code>, using <code>mpg</code> as the response, and <code>disp</code> and <code>cyl</code> as predictors. This should be a model that uses “three regression lines” to model <code>mpg</code>, one for each of the possible <code>cyl</code> levels. They will all have the same slope (since it is an additive model), but each will have its own intercept.</p>
-<div class="sourceCode" id="cb766"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb766-1"><a href="categorical-predictors-and-interactions.html#cb766-1" aria-hidden="true" tabindex="-1"></a>(<span class="at">mpg_disp_add_cyl =</span> <span class="fu">lm</span>(mpg <span class="sc">~</span> disp <span class="sc">+</span> cyl, <span class="at">data =</span> autompg))</span></code></pre></div>
+<div class="sourceCode" id="cb766"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb766-1"><a href="categorical-predictors-and-interactions.html#cb766-1" tabindex="-1"></a>(<span class="at">mpg_disp_add_cyl =</span> <span class="fu">lm</span>(mpg <span class="sc">~</span> disp <span class="sc">+</span> cyl, <span class="at">data =</span> autompg))</span></code></pre></div>
 <pre><code>## 
 ## Call:
 ## lm(formula = mpg ~ disp + cyl, data = autompg)
@@ -1021,19 +1021,19 @@ <h3><span class="header-section-number">11.3.1</span> Factors with More Than Two
 </ul>
 <p>So because 4 cylinder is the reference level, <span class="math inline">\(\beta_0\)</span> is specific to 4 cylinders, but <span class="math inline">\(\beta_2\)</span> and <span class="math inline">\(\beta_3\)</span> are used to represent quantities relative to 4 cylinders.</p>
 <p>As we have done before, we can extract these intercepts and slopes for the three lines, and plot them accordingly.</p>
-<div class="sourceCode" id="cb768"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb768-1"><a href="categorical-predictors-and-interactions.html#cb768-1" aria-hidden="true" tabindex="-1"></a>int_4cyl <span class="ot">=</span> <span class="fu">coef</span>(mpg_disp_add_cyl)[<span class="dv">1</span>]</span>
-<span id="cb768-2"><a href="categorical-predictors-and-interactions.html#cb768-2" aria-hidden="true" tabindex="-1"></a>int_6cyl <span class="ot">=</span> <span class="fu">coef</span>(mpg_disp_add_cyl)[<span class="dv">1</span>] <span class="sc">+</span> <span class="fu">coef</span>(mpg_disp_add_cyl)[<span class="dv">3</span>]</span>
-<span id="cb768-3"><a href="categorical-predictors-and-interactions.html#cb768-3" aria-hidden="true" tabindex="-1"></a>int_8cyl <span class="ot">=</span> <span class="fu">coef</span>(mpg_disp_add_cyl)[<span class="dv">1</span>] <span class="sc">+</span> <span class="fu">coef</span>(mpg_disp_add_cyl)[<span class="dv">4</span>]</span>
-<span id="cb768-4"><a href="categorical-predictors-and-interactions.html#cb768-4" aria-hidden="true" tabindex="-1"></a></span>
-<span id="cb768-5"><a href="categorical-predictors-and-interactions.html#cb768-5" aria-hidden="true" tabindex="-1"></a>slope_all_cyl <span class="ot">=</span> <span class="fu">coef</span>(mpg_disp_add_cyl)[<span class="dv">2</span>]</span>
-<span id="cb768-6"><a href="categorical-predictors-and-interactions.html#cb768-6" aria-hidden="true" tabindex="-1"></a></span>
-<span id="cb768-7"><a href="categorical-predictors-and-interactions.html#cb768-7" aria-hidden="true" tabindex="-1"></a>plot_colors <span class="ot">=</span> <span class="fu">c</span>(<span class="st">&quot;Darkorange&quot;</span>, <span class="st">&quot;Darkgrey&quot;</span>, <span class="st">&quot;Dodgerblue&quot;</span>)</span>
-<span id="cb768-8"><a href="categorical-predictors-and-interactions.html#cb768-8" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(mpg <span class="sc">~</span> disp, <span class="at">data =</span> autompg, <span class="at">col =</span> plot_colors[cyl], <span class="at">pch =</span> <span class="fu">as.numeric</span>(cyl))</span>
-<span id="cb768-9"><a href="categorical-predictors-and-interactions.html#cb768-9" aria-hidden="true" tabindex="-1"></a><span class="fu">abline</span>(int_4cyl, slope_all_cyl, <span class="at">col =</span> plot_colors[<span class="dv">1</span>], <span class="at">lty =</span> <span class="dv">1</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span>
-<span id="cb768-10"><a href="categorical-predictors-and-interactions.html#cb768-10" aria-hidden="true" tabindex="-1"></a><span class="fu">abline</span>(int_6cyl, slope_all_cyl, <span class="at">col =</span> plot_colors[<span class="dv">2</span>], <span class="at">lty =</span> <span class="dv">2</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span>
-<span id="cb768-11"><a href="categorical-predictors-and-interactions.html#cb768-11" aria-hidden="true" tabindex="-1"></a><span class="fu">abline</span>(int_8cyl, slope_all_cyl, <span class="at">col =</span> plot_colors[<span class="dv">3</span>], <span class="at">lty =</span> <span class="dv">3</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span>
-<span id="cb768-12"><a href="categorical-predictors-and-interactions.html#cb768-12" aria-hidden="true" tabindex="-1"></a><span class="fu">legend</span>(<span class="st">&quot;topright&quot;</span>, <span class="fu">c</span>(<span class="st">&quot;4 Cylinder&quot;</span>, <span class="st">&quot;6 Cylinder&quot;</span>, <span class="st">&quot;8 Cylinder&quot;</span>),</span>
-<span id="cb768-13"><a href="categorical-predictors-and-interactions.html#cb768-13" aria-hidden="true" tabindex="-1"></a>       <span class="at">col =</span> plot_colors, <span class="at">lty =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>, <span class="dv">3</span>), <span class="at">pch =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>, <span class="dv">3</span>))</span></code></pre></div>
+<div class="sourceCode" id="cb768"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb768-1"><a href="categorical-predictors-and-interactions.html#cb768-1" tabindex="-1"></a>int_4cyl <span class="ot">=</span> <span class="fu">coef</span>(mpg_disp_add_cyl)[<span class="dv">1</span>]</span>
+<span id="cb768-2"><a href="categorical-predictors-and-interactions.html#cb768-2" tabindex="-1"></a>int_6cyl <span class="ot">=</span> <span class="fu">coef</span>(mpg_disp_add_cyl)[<span class="dv">1</span>] <span class="sc">+</span> <span class="fu">coef</span>(mpg_disp_add_cyl)[<span class="dv">3</span>]</span>
+<span id="cb768-3"><a href="categorical-predictors-and-interactions.html#cb768-3" tabindex="-1"></a>int_8cyl <span class="ot">=</span> <span class="fu">coef</span>(mpg_disp_add_cyl)[<span class="dv">1</span>] <span class="sc">+</span> <span class="fu">coef</span>(mpg_disp_add_cyl)[<span class="dv">4</span>]</span>
+<span id="cb768-4"><a href="categorical-predictors-and-interactions.html#cb768-4" tabindex="-1"></a></span>
+<span id="cb768-5"><a href="categorical-predictors-and-interactions.html#cb768-5" tabindex="-1"></a>slope_all_cyl <span class="ot">=</span> <span class="fu">coef</span>(mpg_disp_add_cyl)[<span class="dv">2</span>]</span>
+<span id="cb768-6"><a href="categorical-predictors-and-interactions.html#cb768-6" tabindex="-1"></a></span>
+<span id="cb768-7"><a href="categorical-predictors-and-interactions.html#cb768-7" tabindex="-1"></a>plot_colors <span class="ot">=</span> <span class="fu">c</span>(<span class="st">&quot;Darkorange&quot;</span>, <span class="st">&quot;Darkgrey&quot;</span>, <span class="st">&quot;Dodgerblue&quot;</span>)</span>
+<span id="cb768-8"><a href="categorical-predictors-and-interactions.html#cb768-8" tabindex="-1"></a><span class="fu">plot</span>(mpg <span class="sc">~</span> disp, <span class="at">data =</span> autompg, <span class="at">col =</span> plot_colors[cyl], <span class="at">pch =</span> <span class="fu">as.numeric</span>(cyl))</span>
+<span id="cb768-9"><a href="categorical-predictors-and-interactions.html#cb768-9" tabindex="-1"></a><span class="fu">abline</span>(int_4cyl, slope_all_cyl, <span class="at">col =</span> plot_colors[<span class="dv">1</span>], <span class="at">lty =</span> <span class="dv">1</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span>
+<span id="cb768-10"><a href="categorical-predictors-and-interactions.html#cb768-10" tabindex="-1"></a><span class="fu">abline</span>(int_6cyl, slope_all_cyl, <span class="at">col =</span> plot_colors[<span class="dv">2</span>], <span class="at">lty =</span> <span class="dv">2</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span>
+<span id="cb768-11"><a href="categorical-predictors-and-interactions.html#cb768-11" tabindex="-1"></a><span class="fu">abline</span>(int_8cyl, slope_all_cyl, <span class="at">col =</span> plot_colors[<span class="dv">3</span>], <span class="at">lty =</span> <span class="dv">3</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span>
+<span id="cb768-12"><a href="categorical-predictors-and-interactions.html#cb768-12" tabindex="-1"></a><span class="fu">legend</span>(<span class="st">&quot;topright&quot;</span>, <span class="fu">c</span>(<span class="st">&quot;4 Cylinder&quot;</span>, <span class="st">&quot;6 Cylinder&quot;</span>, <span class="st">&quot;8 Cylinder&quot;</span>),</span>
+<span id="cb768-13"><a href="categorical-predictors-and-interactions.html#cb768-13" tabindex="-1"></a>       <span class="at">col =</span> plot_colors, <span class="at">lty =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>, <span class="dv">3</span>), <span class="at">pch =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>, <span class="dv">3</span>))</span></code></pre></div>
 <p><img src="cat-int_files/figure-html/unnamed-chunk-37-1.png" width="672" style="display: block; margin: auto;" /></p>
 <p>On this plot, we have</p>
 <ul>
@@ -1043,7 +1043,7 @@ <h3><span class="header-section-number">11.3.1</span> Factors with More Than Two
 </ul>
 <p>The odd result here is that we’re estimating that 8 cylinder cars have better fuel efficiency than 6 cylinder cars at <strong>any</strong> displacement! The dotted blue line is always above the dashed grey line. That doesn’t seem right. Maybe for very large displacement engines that could be true, but that seems wrong for medium to low displacement.</p>
 <p>To attempt to fix this, we will try using an interaction model, that is, instead of simply three intercepts and one slope, we will allow for three slopes. Again, we’ll let <code>R</code> take the wheel (no pun intended), then figure out what model it has applied.</p>
-<div class="sourceCode" id="cb769"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb769-1"><a href="categorical-predictors-and-interactions.html#cb769-1" aria-hidden="true" tabindex="-1"></a>(<span class="at">mpg_disp_int_cyl =</span> <span class="fu">lm</span>(mpg <span class="sc">~</span> disp <span class="sc">*</span> cyl, <span class="at">data =</span> autompg))</span></code></pre></div>
+<div class="sourceCode" id="cb769"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb769-1"><a href="categorical-predictors-and-interactions.html#cb769-1" tabindex="-1"></a>(<span class="at">mpg_disp_int_cyl =</span> <span class="fu">lm</span>(mpg <span class="sc">~</span> disp <span class="sc">*</span> cyl, <span class="at">data =</span> autompg))</span></code></pre></div>
 <pre><code>## 
 ## Call:
 ## lm(formula = mpg ~ disp * cyl, data = autompg)
@@ -1051,7 +1051,7 @@ <h3><span class="header-section-number">11.3.1</span> Factors with More Than Two
 ## Coefficients:
 ## (Intercept)         disp         cyl6         cyl8    disp:cyl6    disp:cyl8  
 ##    43.59052     -0.13069    -13.20026    -20.85706      0.08299      0.10817</code></pre>
-<div class="sourceCode" id="cb771"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb771-1"><a href="categorical-predictors-and-interactions.html#cb771-1" aria-hidden="true" tabindex="-1"></a><span class="co"># could also use mpg ~ disp + cyl + disp:cyl</span></span></code></pre></div>
+<div class="sourceCode" id="cb771"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb771-1"><a href="categorical-predictors-and-interactions.html#cb771-1" tabindex="-1"></a><span class="co"># could also use mpg ~ disp + cyl + disp:cyl</span></span></code></pre></div>
 <p><code>R</code> has again chosen to use 4 cylinder cars as the reference level, but this also now has an effect on the interaction terms. <code>R</code> has fit the model.</p>
 <p><span class="math display">\[
 Y = \beta_0 + \beta_1 x + \beta_2 v_2 + \beta_3 v_3 + \gamma_2 x v_2 + \gamma_3 x v_3 + \epsilon
@@ -1071,21 +1071,21 @@ <h3><span class="header-section-number">11.3.1</span> Factors with More Than Two
 <p>So, as we have seen before <span class="math inline">\(\beta_2\)</span> and <span class="math inline">\(\beta_3\)</span> change the intercepts for 6 and 8 cylinder cars relative to the reference level of <span class="math inline">\(\beta_0\)</span> for 4 cylinder cars.</p>
 <p>Now, similarly <span class="math inline">\(\gamma_2\)</span> and <span class="math inline">\(\gamma_3\)</span> change the slopes for 6 and 8 cylinder cars relative to the reference level of <span class="math inline">\(\beta_1\)</span> for 4 cylinder cars.</p>
 <p>Once again, we extract the coefficients and plot the results.</p>
-<div class="sourceCode" id="cb772"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb772-1"><a href="categorical-predictors-and-interactions.html#cb772-1" aria-hidden="true" tabindex="-1"></a>int_4cyl <span class="ot">=</span> <span class="fu">coef</span>(mpg_disp_int_cyl)[<span class="dv">1</span>]</span>
-<span id="cb772-2"><a href="categorical-predictors-and-interactions.html#cb772-2" aria-hidden="true" tabindex="-1"></a>int_6cyl <span class="ot">=</span> <span class="fu">coef</span>(mpg_disp_int_cyl)[<span class="dv">1</span>] <span class="sc">+</span> <span class="fu">coef</span>(mpg_disp_int_cyl)[<span class="dv">3</span>]</span>
-<span id="cb772-3"><a href="categorical-predictors-and-interactions.html#cb772-3" aria-hidden="true" tabindex="-1"></a>int_8cyl <span class="ot">=</span> <span class="fu">coef</span>(mpg_disp_int_cyl)[<span class="dv">1</span>] <span class="sc">+</span> <span class="fu">coef</span>(mpg_disp_int_cyl)[<span class="dv">4</span>]</span>
-<span id="cb772-4"><a href="categorical-predictors-and-interactions.html#cb772-4" aria-hidden="true" tabindex="-1"></a></span>
-<span id="cb772-5"><a href="categorical-predictors-and-interactions.html#cb772-5" aria-hidden="true" tabindex="-1"></a>slope_4cyl <span class="ot">=</span> <span class="fu">coef</span>(mpg_disp_int_cyl)[<span class="dv">2</span>]</span>
-<span id="cb772-6"><a href="categorical-predictors-and-interactions.html#cb772-6" aria-hidden="true" tabindex="-1"></a>slope_6cyl <span class="ot">=</span> <span class="fu">coef</span>(mpg_disp_int_cyl)[<span class="dv">2</span>] <span class="sc">+</span> <span class="fu">coef</span>(mpg_disp_int_cyl)[<span class="dv">5</span>]</span>
-<span id="cb772-7"><a href="categorical-predictors-and-interactions.html#cb772-7" aria-hidden="true" tabindex="-1"></a>slope_8cyl <span class="ot">=</span> <span class="fu">coef</span>(mpg_disp_int_cyl)[<span class="dv">2</span>] <span class="sc">+</span> <span class="fu">coef</span>(mpg_disp_int_cyl)[<span class="dv">6</span>]</span>
-<span id="cb772-8"><a href="categorical-predictors-and-interactions.html#cb772-8" aria-hidden="true" tabindex="-1"></a></span>
-<span id="cb772-9"><a href="categorical-predictors-and-interactions.html#cb772-9" aria-hidden="true" tabindex="-1"></a>plot_colors <span class="ot">=</span> <span class="fu">c</span>(<span class="st">&quot;Darkorange&quot;</span>, <span class="st">&quot;Darkgrey&quot;</span>, <span class="st">&quot;Dodgerblue&quot;</span>)</span>
-<span id="cb772-10"><a href="categorical-predictors-and-interactions.html#cb772-10" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(mpg <span class="sc">~</span> disp, <span class="at">data =</span> autompg, <span class="at">col =</span> plot_colors[cyl], <span class="at">pch =</span> <span class="fu">as.numeric</span>(cyl))</span>
-<span id="cb772-11"><a href="categorical-predictors-and-interactions.html#cb772-11" aria-hidden="true" tabindex="-1"></a><span class="fu">abline</span>(int_4cyl, slope_4cyl, <span class="at">col =</span> plot_colors[<span class="dv">1</span>], <span class="at">lty =</span> <span class="dv">1</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span>
-<span id="cb772-12"><a href="categorical-predictors-and-interactions.html#cb772-12" aria-hidden="true" tabindex="-1"></a><span class="fu">abline</span>(int_6cyl, slope_6cyl, <span class="at">col =</span> plot_colors[<span class="dv">2</span>], <span class="at">lty =</span> <span class="dv">2</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span>
-<span id="cb772-13"><a href="categorical-predictors-and-interactions.html#cb772-13" aria-hidden="true" tabindex="-1"></a><span class="fu">abline</span>(int_8cyl, slope_8cyl, <span class="at">col =</span> plot_colors[<span class="dv">3</span>], <span class="at">lty =</span> <span class="dv">3</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span>
-<span id="cb772-14"><a href="categorical-predictors-and-interactions.html#cb772-14" aria-hidden="true" tabindex="-1"></a><span class="fu">legend</span>(<span class="st">&quot;topright&quot;</span>, <span class="fu">c</span>(<span class="st">&quot;4 Cylinder&quot;</span>, <span class="st">&quot;6 Cylinder&quot;</span>, <span class="st">&quot;8 Cylinder&quot;</span>),</span>
-<span id="cb772-15"><a href="categorical-predictors-and-interactions.html#cb772-15" aria-hidden="true" tabindex="-1"></a>       <span class="at">col =</span> plot_colors, <span class="at">lty =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>, <span class="dv">3</span>), <span class="at">pch =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>, <span class="dv">3</span>))</span></code></pre></div>
+<div class="sourceCode" id="cb772"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb772-1"><a href="categorical-predictors-and-interactions.html#cb772-1" tabindex="-1"></a>int_4cyl <span class="ot">=</span> <span class="fu">coef</span>(mpg_disp_int_cyl)[<span class="dv">1</span>]</span>
+<span id="cb772-2"><a href="categorical-predictors-and-interactions.html#cb772-2" tabindex="-1"></a>int_6cyl <span class="ot">=</span> <span class="fu">coef</span>(mpg_disp_int_cyl)[<span class="dv">1</span>] <span class="sc">+</span> <span class="fu">coef</span>(mpg_disp_int_cyl)[<span class="dv">3</span>]</span>
+<span id="cb772-3"><a href="categorical-predictors-and-interactions.html#cb772-3" tabindex="-1"></a>int_8cyl <span class="ot">=</span> <span class="fu">coef</span>(mpg_disp_int_cyl)[<span class="dv">1</span>] <span class="sc">+</span> <span class="fu">coef</span>(mpg_disp_int_cyl)[<span class="dv">4</span>]</span>
+<span id="cb772-4"><a href="categorical-predictors-and-interactions.html#cb772-4" tabindex="-1"></a></span>
+<span id="cb772-5"><a href="categorical-predictors-and-interactions.html#cb772-5" tabindex="-1"></a>slope_4cyl <span class="ot">=</span> <span class="fu">coef</span>(mpg_disp_int_cyl)[<span class="dv">2</span>]</span>
+<span id="cb772-6"><a href="categorical-predictors-and-interactions.html#cb772-6" tabindex="-1"></a>slope_6cyl <span class="ot">=</span> <span class="fu">coef</span>(mpg_disp_int_cyl)[<span class="dv">2</span>] <span class="sc">+</span> <span class="fu">coef</span>(mpg_disp_int_cyl)[<span class="dv">5</span>]</span>
+<span id="cb772-7"><a href="categorical-predictors-and-interactions.html#cb772-7" tabindex="-1"></a>slope_8cyl <span class="ot">=</span> <span class="fu">coef</span>(mpg_disp_int_cyl)[<span class="dv">2</span>] <span class="sc">+</span> <span class="fu">coef</span>(mpg_disp_int_cyl)[<span class="dv">6</span>]</span>
+<span id="cb772-8"><a href="categorical-predictors-and-interactions.html#cb772-8" tabindex="-1"></a></span>
+<span id="cb772-9"><a href="categorical-predictors-and-interactions.html#cb772-9" tabindex="-1"></a>plot_colors <span class="ot">=</span> <span class="fu">c</span>(<span class="st">&quot;Darkorange&quot;</span>, <span class="st">&quot;Darkgrey&quot;</span>, <span class="st">&quot;Dodgerblue&quot;</span>)</span>
+<span id="cb772-10"><a href="categorical-predictors-and-interactions.html#cb772-10" tabindex="-1"></a><span class="fu">plot</span>(mpg <span class="sc">~</span> disp, <span class="at">data =</span> autompg, <span class="at">col =</span> plot_colors[cyl], <span class="at">pch =</span> <span class="fu">as.numeric</span>(cyl))</span>
+<span id="cb772-11"><a href="categorical-predictors-and-interactions.html#cb772-11" tabindex="-1"></a><span class="fu">abline</span>(int_4cyl, slope_4cyl, <span class="at">col =</span> plot_colors[<span class="dv">1</span>], <span class="at">lty =</span> <span class="dv">1</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span>
+<span id="cb772-12"><a href="categorical-predictors-and-interactions.html#cb772-12" tabindex="-1"></a><span class="fu">abline</span>(int_6cyl, slope_6cyl, <span class="at">col =</span> plot_colors[<span class="dv">2</span>], <span class="at">lty =</span> <span class="dv">2</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span>
+<span id="cb772-13"><a href="categorical-predictors-and-interactions.html#cb772-13" tabindex="-1"></a><span class="fu">abline</span>(int_8cyl, slope_8cyl, <span class="at">col =</span> plot_colors[<span class="dv">3</span>], <span class="at">lty =</span> <span class="dv">3</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span>
+<span id="cb772-14"><a href="categorical-predictors-and-interactions.html#cb772-14" tabindex="-1"></a><span class="fu">legend</span>(<span class="st">&quot;topright&quot;</span>, <span class="fu">c</span>(<span class="st">&quot;4 Cylinder&quot;</span>, <span class="st">&quot;6 Cylinder&quot;</span>, <span class="st">&quot;8 Cylinder&quot;</span>),</span>
+<span id="cb772-15"><a href="categorical-predictors-and-interactions.html#cb772-15" tabindex="-1"></a>       <span class="at">col =</span> plot_colors, <span class="at">lty =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>, <span class="dv">3</span>), <span class="at">pch =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>, <span class="dv">3</span>))</span></code></pre></div>
 <p><img src="cat-int_files/figure-html/unnamed-chunk-39-1.png" width="672" style="display: block; margin: auto;" /></p>
 <p>This looks much better! We can see that for medium displacement cars, 6 cylinder cars now perform better than 8 cylinder cars, which seems much more reasonable than before.</p>
 <p>To completely justify the interaction model (i.e., a unique slope for each <code>cyl</code> level) compared to the additive model (single slope), we can perform an <span class="math inline">\(F\)</span>-test. Notice first, that there is no <span class="math inline">\(t\)</span>-test that will be able to do this since the difference between the two models is not a single parameter.</p>
@@ -1098,7 +1098,7 @@ <h3><span class="header-section-number">11.3.1</span> Factors with More Than Two
 Y = \beta_0 + \beta_1 x + \beta_2 v_2 + \beta_3 v_3 + \epsilon.
 \]</span></p>
 <p>Again, this is a difference of two parameters, thus no <span class="math inline">\(t\)</span>-test will be useful.</p>
-<div class="sourceCode" id="cb773"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb773-1"><a href="categorical-predictors-and-interactions.html#cb773-1" aria-hidden="true" tabindex="-1"></a><span class="fu">anova</span>(mpg_disp_add_cyl, mpg_disp_int_cyl)</span></code></pre></div>
+<div class="sourceCode" id="cb773"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb773-1"><a href="categorical-predictors-and-interactions.html#cb773-1" tabindex="-1"></a><span class="fu">anova</span>(mpg_disp_add_cyl, mpg_disp_int_cyl)</span></code></pre></div>
 <pre><code>## Analysis of Variance Table
 ## 
 ## Model 1: mpg ~ disp + cyl
@@ -1120,26 +1120,26 @@ <h3><span class="header-section-number">11.3.1</span> Factors with More Than Two
 <li>Number of parameters: <span class="math inline">\(p = 6\)</span></li>
 </ul></li>
 </ul>
-<div class="sourceCode" id="cb775"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb775-1"><a href="categorical-predictors-and-interactions.html#cb775-1" aria-hidden="true" tabindex="-1"></a><span class="fu">length</span>(<span class="fu">coef</span>(mpg_disp_int_cyl)) <span class="sc">-</span> <span class="fu">length</span>(<span class="fu">coef</span>(mpg_disp_add_cyl))</span></code></pre></div>
+<div class="sourceCode" id="cb775"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb775-1"><a href="categorical-predictors-and-interactions.html#cb775-1" tabindex="-1"></a><span class="fu">length</span>(<span class="fu">coef</span>(mpg_disp_int_cyl)) <span class="sc">-</span> <span class="fu">length</span>(<span class="fu">coef</span>(mpg_disp_add_cyl))</span></code></pre></div>
 <pre><code>## [1] 2</code></pre>
 <p>We see there is a difference of two parameters, which is also displayed in the resulting ANOVA table from <code>R</code>. Notice that the following two values also appear on the ANOVA table.</p>
-<div class="sourceCode" id="cb777"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb777-1"><a href="categorical-predictors-and-interactions.html#cb777-1" aria-hidden="true" tabindex="-1"></a><span class="fu">nrow</span>(autompg) <span class="sc">-</span> <span class="fu">length</span>(<span class="fu">coef</span>(mpg_disp_int_cyl))</span></code></pre></div>
+<div class="sourceCode" id="cb777"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb777-1"><a href="categorical-predictors-and-interactions.html#cb777-1" tabindex="-1"></a><span class="fu">nrow</span>(autompg) <span class="sc">-</span> <span class="fu">length</span>(<span class="fu">coef</span>(mpg_disp_int_cyl))</span></code></pre></div>
 <pre><code>## [1] 377</code></pre>
-<div class="sourceCode" id="cb779"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb779-1"><a href="categorical-predictors-and-interactions.html#cb779-1" aria-hidden="true" tabindex="-1"></a><span class="fu">nrow</span>(autompg) <span class="sc">-</span> <span class="fu">length</span>(<span class="fu">coef</span>(mpg_disp_add_cyl))</span></code></pre></div>
+<div class="sourceCode" id="cb779"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb779-1"><a href="categorical-predictors-and-interactions.html#cb779-1" tabindex="-1"></a><span class="fu">nrow</span>(autompg) <span class="sc">-</span> <span class="fu">length</span>(<span class="fu">coef</span>(mpg_disp_add_cyl))</span></code></pre></div>
 <pre><code>## [1] 379</code></pre>
 </div>
 </div>
 <div id="parameterization" class="section level2 hasAnchor" number="11.4">
 <h2><span class="header-section-number">11.4</span> Parameterization<a href="categorical-predictors-and-interactions.html#parameterization" class="anchor-section" aria-label="Anchor link to header"></a></h2>
 <p>So far we have been simply letting <code>R</code> decide how to create the dummy variables, and thus <code>R</code> has been deciding the parameterization of the models. To illustrate the ability to use alternative parameterizations, we will recreate the data, but directly creating the dummy variables ourselves.</p>
-<div class="sourceCode" id="cb781"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb781-1"><a href="categorical-predictors-and-interactions.html#cb781-1" aria-hidden="true" tabindex="-1"></a>new_param_data <span class="ot">=</span> <span class="fu">data.frame</span>(</span>
-<span id="cb781-2"><a href="categorical-predictors-and-interactions.html#cb781-2" aria-hidden="true" tabindex="-1"></a>  <span class="at">y =</span> autompg<span class="sc">$</span>mpg,</span>
-<span id="cb781-3"><a href="categorical-predictors-and-interactions.html#cb781-3" aria-hidden="true" tabindex="-1"></a>  <span class="at">x =</span> autompg<span class="sc">$</span>disp,</span>
-<span id="cb781-4"><a href="categorical-predictors-and-interactions.html#cb781-4" aria-hidden="true" tabindex="-1"></a>  <span class="at">v1 =</span> <span class="dv">1</span> <span class="sc">*</span> <span class="fu">as.numeric</span>(autompg<span class="sc">$</span>cyl <span class="sc">==</span> <span class="dv">4</span>),</span>
-<span id="cb781-5"><a href="categorical-predictors-and-interactions.html#cb781-5" aria-hidden="true" tabindex="-1"></a>  <span class="at">v2 =</span> <span class="dv">1</span> <span class="sc">*</span> <span class="fu">as.numeric</span>(autompg<span class="sc">$</span>cyl <span class="sc">==</span> <span class="dv">6</span>),</span>
-<span id="cb781-6"><a href="categorical-predictors-and-interactions.html#cb781-6" aria-hidden="true" tabindex="-1"></a>  <span class="at">v3 =</span> <span class="dv">1</span> <span class="sc">*</span> <span class="fu">as.numeric</span>(autompg<span class="sc">$</span>cyl <span class="sc">==</span> <span class="dv">8</span>))</span>
-<span id="cb781-7"><a href="categorical-predictors-and-interactions.html#cb781-7" aria-hidden="true" tabindex="-1"></a></span>
-<span id="cb781-8"><a href="categorical-predictors-and-interactions.html#cb781-8" aria-hidden="true" tabindex="-1"></a><span class="fu">head</span>(new_param_data, <span class="dv">20</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb781"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb781-1"><a href="categorical-predictors-and-interactions.html#cb781-1" tabindex="-1"></a>new_param_data <span class="ot">=</span> <span class="fu">data.frame</span>(</span>
+<span id="cb781-2"><a href="categorical-predictors-and-interactions.html#cb781-2" tabindex="-1"></a>  <span class="at">y =</span> autompg<span class="sc">$</span>mpg,</span>
+<span id="cb781-3"><a href="categorical-predictors-and-interactions.html#cb781-3" tabindex="-1"></a>  <span class="at">x =</span> autompg<span class="sc">$</span>disp,</span>
+<span id="cb781-4"><a href="categorical-predictors-and-interactions.html#cb781-4" tabindex="-1"></a>  <span class="at">v1 =</span> <span class="dv">1</span> <span class="sc">*</span> <span class="fu">as.numeric</span>(autompg<span class="sc">$</span>cyl <span class="sc">==</span> <span class="dv">4</span>),</span>
+<span id="cb781-5"><a href="categorical-predictors-and-interactions.html#cb781-5" tabindex="-1"></a>  <span class="at">v2 =</span> <span class="dv">1</span> <span class="sc">*</span> <span class="fu">as.numeric</span>(autompg<span class="sc">$</span>cyl <span class="sc">==</span> <span class="dv">6</span>),</span>
+<span id="cb781-6"><a href="categorical-predictors-and-interactions.html#cb781-6" tabindex="-1"></a>  <span class="at">v3 =</span> <span class="dv">1</span> <span class="sc">*</span> <span class="fu">as.numeric</span>(autompg<span class="sc">$</span>cyl <span class="sc">==</span> <span class="dv">8</span>))</span>
+<span id="cb781-7"><a href="categorical-predictors-and-interactions.html#cb781-7" tabindex="-1"></a></span>
+<span id="cb781-8"><a href="categorical-predictors-and-interactions.html#cb781-8" tabindex="-1"></a><span class="fu">head</span>(new_param_data, <span class="dv">20</span>)</span></code></pre></div>
 <pre><code>##     y   x v1 v2 v3
 ## 1  18 307  0  0  1
 ## 2  15 350  0  0  1
@@ -1168,7 +1168,7 @@ <h2><span class="header-section-number">11.4</span> Parameterization<a href="cat
 <li><code>v1</code>, <code>v2</code>, and <code>v3</code> are dummy variables as defined above.</li>
 </ul>
 <p>First let’s try to fit an additive model using <code>x</code> as well as the three dummy variables.</p>
-<div class="sourceCode" id="cb783"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb783-1"><a href="categorical-predictors-and-interactions.html#cb783-1" aria-hidden="true" tabindex="-1"></a><span class="fu">lm</span>(y <span class="sc">~</span> x <span class="sc">+</span> v1 <span class="sc">+</span> v2 <span class="sc">+</span> v3, <span class="at">data =</span> new_param_data)</span></code></pre></div>
+<div class="sourceCode" id="cb783"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb783-1"><a href="categorical-predictors-and-interactions.html#cb783-1" tabindex="-1"></a><span class="fu">lm</span>(y <span class="sc">~</span> x <span class="sc">+</span> v1 <span class="sc">+</span> v2 <span class="sc">+</span> v3, <span class="at">data =</span> new_param_data)</span></code></pre></div>
 <pre><code>## 
 ## Call:
 ## lm(formula = y ~ x + v1 + v2 + v3, data = new_param_data)
@@ -1182,7 +1182,7 @@ <h2><span class="header-section-number">11.4</span> Parameterization<a href="cat
 \]</span></p>
 <p>which means that <span class="math inline">\(\boldsymbol{1}\)</span>, <span class="math inline">\(v_1\)</span>, <span class="math inline">\(v_2\)</span>, and <span class="math inline">\(v_3\)</span> are linearly dependent. This would make the <span class="math inline">\(X^\top X\)</span> matrix singular, but we need to be able to invert it to solve the normal equations and obtain <span class="math inline">\(\hat{\beta}.\)</span> With the intercept, <code>v1</code>, and <code>v2</code>, <code>R</code> can make the necessary “three intercepts.” So, in this case <code>v3</code> is the reference level.</p>
 <p>If we remove the intercept, then we can directly obtain all “three intercepts” without a reference level.</p>
-<div class="sourceCode" id="cb785"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb785-1"><a href="categorical-predictors-and-interactions.html#cb785-1" aria-hidden="true" tabindex="-1"></a><span class="fu">lm</span>(y <span class="sc">~</span> <span class="dv">0</span> <span class="sc">+</span> x <span class="sc">+</span> v1 <span class="sc">+</span> v2 <span class="sc">+</span> v3, <span class="at">data =</span> new_param_data)</span></code></pre></div>
+<div class="sourceCode" id="cb785"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb785-1"><a href="categorical-predictors-and-interactions.html#cb785-1" tabindex="-1"></a><span class="fu">lm</span>(y <span class="sc">~</span> <span class="dv">0</span> <span class="sc">+</span> x <span class="sc">+</span> v1 <span class="sc">+</span> v2 <span class="sc">+</span> v3, <span class="at">data =</span> new_param_data)</span></code></pre></div>
 <pre><code>## 
 ## Call:
 ## lm(formula = y ~ 0 + x + v1 + v2 + v3, data = new_param_data)
@@ -1201,7 +1201,7 @@ <h2><span class="header-section-number">11.4</span> Parameterization<a href="cat
 <li>8 Cylinder: <span class="math inline">\(Y = \mu_3 + \beta x + \epsilon\)</span></li>
 </ul>
 <p>We could also do something similar with the interaction model, and give each line an intercept and slope, without the need for a reference level.</p>
-<div class="sourceCode" id="cb787"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb787-1"><a href="categorical-predictors-and-interactions.html#cb787-1" aria-hidden="true" tabindex="-1"></a><span class="fu">lm</span>(y <span class="sc">~</span> <span class="dv">0</span> <span class="sc">+</span> v1 <span class="sc">+</span> v2 <span class="sc">+</span> v3 <span class="sc">+</span> x<span class="sc">:</span>v1 <span class="sc">+</span> x<span class="sc">:</span>v2 <span class="sc">+</span> x<span class="sc">:</span>v3, <span class="at">data =</span> new_param_data)</span></code></pre></div>
+<div class="sourceCode" id="cb787"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb787-1"><a href="categorical-predictors-and-interactions.html#cb787-1" tabindex="-1"></a><span class="fu">lm</span>(y <span class="sc">~</span> <span class="dv">0</span> <span class="sc">+</span> v1 <span class="sc">+</span> v2 <span class="sc">+</span> v3 <span class="sc">+</span> x<span class="sc">:</span>v1 <span class="sc">+</span> x<span class="sc">:</span>v2 <span class="sc">+</span> x<span class="sc">:</span>v3, <span class="at">data =</span> new_param_data)</span></code></pre></div>
 <pre><code>## 
 ## Call:
 ## lm(formula = y ~ 0 + v1 + v2 + v3 + x:v1 + x:v2 + x:v3, data = new_param_data)
@@ -1218,7 +1218,7 @@ <h2><span class="header-section-number">11.4</span> Parameterization<a href="cat
 <li>8 Cylinder: <span class="math inline">\(Y = \mu_3 + \beta_3 x + \epsilon\)</span></li>
 </ul>
 <p>Using the original data, we have (at least) three equivalent ways to specify the interaction model with <code>R</code>.</p>
-<div class="sourceCode" id="cb789"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb789-1"><a href="categorical-predictors-and-interactions.html#cb789-1" aria-hidden="true" tabindex="-1"></a><span class="fu">lm</span>(mpg <span class="sc">~</span> disp <span class="sc">*</span> cyl, <span class="at">data =</span> autompg)</span></code></pre></div>
+<div class="sourceCode" id="cb789"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb789-1"><a href="categorical-predictors-and-interactions.html#cb789-1" tabindex="-1"></a><span class="fu">lm</span>(mpg <span class="sc">~</span> disp <span class="sc">*</span> cyl, <span class="at">data =</span> autompg)</span></code></pre></div>
 <pre><code>## 
 ## Call:
 ## lm(formula = mpg ~ disp * cyl, data = autompg)
@@ -1226,7 +1226,7 @@ <h2><span class="header-section-number">11.4</span> Parameterization<a href="cat
 ## Coefficients:
 ## (Intercept)         disp         cyl6         cyl8    disp:cyl6    disp:cyl8  
 ##    43.59052     -0.13069    -13.20026    -20.85706      0.08299      0.10817</code></pre>
-<div class="sourceCode" id="cb791"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb791-1"><a href="categorical-predictors-and-interactions.html#cb791-1" aria-hidden="true" tabindex="-1"></a><span class="fu">lm</span>(mpg <span class="sc">~</span> <span class="dv">0</span> <span class="sc">+</span> cyl <span class="sc">+</span> disp <span class="sc">:</span> cyl, <span class="at">data =</span> autompg)</span></code></pre></div>
+<div class="sourceCode" id="cb791"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb791-1"><a href="categorical-predictors-and-interactions.html#cb791-1" tabindex="-1"></a><span class="fu">lm</span>(mpg <span class="sc">~</span> <span class="dv">0</span> <span class="sc">+</span> cyl <span class="sc">+</span> disp <span class="sc">:</span> cyl, <span class="at">data =</span> autompg)</span></code></pre></div>
 <pre><code>## 
 ## Call:
 ## lm(formula = mpg ~ 0 + cyl + disp:cyl, data = autompg)
@@ -1234,7 +1234,7 @@ <h2><span class="header-section-number">11.4</span> Parameterization<a href="cat
 ## Coefficients:
 ##      cyl4       cyl6       cyl8  cyl4:disp  cyl6:disp  cyl8:disp  
 ##  43.59052   30.39026   22.73346   -0.13069   -0.04770   -0.02252</code></pre>
-<div class="sourceCode" id="cb793"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb793-1"><a href="categorical-predictors-and-interactions.html#cb793-1" aria-hidden="true" tabindex="-1"></a><span class="fu">lm</span>(mpg <span class="sc">~</span> <span class="dv">0</span> <span class="sc">+</span> disp <span class="sc">+</span> cyl <span class="sc">+</span> disp <span class="sc">:</span> cyl, <span class="at">data =</span> autompg)</span></code></pre></div>
+<div class="sourceCode" id="cb793"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb793-1"><a href="categorical-predictors-and-interactions.html#cb793-1" tabindex="-1"></a><span class="fu">lm</span>(mpg <span class="sc">~</span> <span class="dv">0</span> <span class="sc">+</span> disp <span class="sc">+</span> cyl <span class="sc">+</span> disp <span class="sc">:</span> cyl, <span class="at">data =</span> autompg)</span></code></pre></div>
 <pre><code>## 
 ## Call:
 ## lm(formula = mpg ~ 0 + disp + cyl + disp:cyl, data = autompg)
@@ -1244,8 +1244,8 @@ <h2><span class="header-section-number">11.4</span> Parameterization<a href="cat
 ##  -0.13069   43.59052   30.39026   22.73346    0.08299    0.10817</code></pre>
 <p>They all fit the same model, importantly each using six parameters, but the coefficients mean slightly different things in each. However, once they are interpreted as slopes and intercepts for the “three lines” they will have the same result.</p>
 <p>Use <code>?all.equal</code> to learn about the <code>all.equal()</code> function, and think about how the following code verifies that the residuals of the two models are the same.</p>
-<div class="sourceCode" id="cb795"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb795-1"><a href="categorical-predictors-and-interactions.html#cb795-1" aria-hidden="true" tabindex="-1"></a><span class="fu">all.equal</span>(<span class="fu">fitted</span>(<span class="fu">lm</span>(mpg <span class="sc">~</span> disp <span class="sc">*</span> cyl, <span class="at">data =</span> autompg)), </span>
-<span id="cb795-2"><a href="categorical-predictors-and-interactions.html#cb795-2" aria-hidden="true" tabindex="-1"></a>          <span class="fu">fitted</span>(<span class="fu">lm</span>(mpg <span class="sc">~</span> <span class="dv">0</span> <span class="sc">+</span> cyl <span class="sc">+</span> disp <span class="sc">:</span> cyl, <span class="at">data =</span> autompg)))</span></code></pre></div>
+<div class="sourceCode" id="cb795"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb795-1"><a href="categorical-predictors-and-interactions.html#cb795-1" tabindex="-1"></a><span class="fu">all.equal</span>(<span class="fu">fitted</span>(<span class="fu">lm</span>(mpg <span class="sc">~</span> disp <span class="sc">*</span> cyl, <span class="at">data =</span> autompg)), </span>
+<span id="cb795-2"><a href="categorical-predictors-and-interactions.html#cb795-2" tabindex="-1"></a>          <span class="fu">fitted</span>(<span class="fu">lm</span>(mpg <span class="sc">~</span> <span class="dv">0</span> <span class="sc">+</span> cyl <span class="sc">+</span> disp <span class="sc">:</span> cyl, <span class="at">data =</span> autompg)))</span></code></pre></div>
 <pre><code>## [1] TRUE</code></pre>
 </div>
 <div id="building-larger-models" class="section level2 hasAnchor" number="11.5">
@@ -1298,8 +1298,8 @@ <h2><span class="header-section-number">11.5</span> Building Larger Models<a hre
 \]</span></p>
 <p>It is so flexible, it is becoming hard to interpret!</p>
 <p>Let’s fit this three-way interaction model in <code>R</code>.</p>
-<div class="sourceCode" id="cb797"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb797-1"><a href="categorical-predictors-and-interactions.html#cb797-1" aria-hidden="true" tabindex="-1"></a>big_model <span class="ot">=</span> <span class="fu">lm</span>(mpg <span class="sc">~</span> disp <span class="sc">*</span> hp <span class="sc">*</span> domestic, <span class="at">data =</span> autompg)</span>
-<span id="cb797-2"><a href="categorical-predictors-and-interactions.html#cb797-2" aria-hidden="true" tabindex="-1"></a><span class="fu">summary</span>(big_model)</span></code></pre></div>
+<div class="sourceCode" id="cb797"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb797-1"><a href="categorical-predictors-and-interactions.html#cb797-1" tabindex="-1"></a>big_model <span class="ot">=</span> <span class="fu">lm</span>(mpg <span class="sc">~</span> disp <span class="sc">*</span> hp <span class="sc">*</span> domestic, <span class="at">data =</span> autompg)</span>
+<span id="cb797-2"><a href="categorical-predictors-and-interactions.html#cb797-2" tabindex="-1"></a><span class="fu">summary</span>(big_model)</span></code></pre></div>
 <pre><code>## 
 ## Call:
 ## lm(formula = mpg ~ disp * hp * domestic, data = autompg)
@@ -1334,9 +1334,9 @@ <h2><span class="header-section-number">11.5</span> Building Larger Models<a hre
 <li>Null Model: <span class="math inline">\(Y = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \beta_3 x_3 + \beta_4 x_1 x_2 + \beta_5 x_1 x_3 + \beta_6 x_2 x_3 + \epsilon\)</span></li>
 </ul>
 <p>We fit the null model in <code>R</code> as <code>two_way_int_mod</code>, then use <code>anova()</code> to perform an <span class="math inline">\(F\)</span>-test as usual.</p>
-<div class="sourceCode" id="cb799"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb799-1"><a href="categorical-predictors-and-interactions.html#cb799-1" aria-hidden="true" tabindex="-1"></a>two_way_int_mod <span class="ot">=</span> <span class="fu">lm</span>(mpg <span class="sc">~</span> disp <span class="sc">*</span> hp <span class="sc">+</span> disp <span class="sc">*</span> domestic <span class="sc">+</span> hp <span class="sc">*</span> domestic, <span class="at">data =</span> autompg)</span>
-<span id="cb799-2"><a href="categorical-predictors-and-interactions.html#cb799-2" aria-hidden="true" tabindex="-1"></a><span class="co">#two_way_int_mod = lm(mpg ~ (disp + hp + domestic) ^ 2, data = autompg)</span></span>
-<span id="cb799-3"><a href="categorical-predictors-and-interactions.html#cb799-3" aria-hidden="true" tabindex="-1"></a><span class="fu">anova</span>(two_way_int_mod, big_model)</span></code></pre></div>
+<div class="sourceCode" id="cb799"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb799-1"><a href="categorical-predictors-and-interactions.html#cb799-1" tabindex="-1"></a>two_way_int_mod <span class="ot">=</span> <span class="fu">lm</span>(mpg <span class="sc">~</span> disp <span class="sc">*</span> hp <span class="sc">+</span> disp <span class="sc">*</span> domestic <span class="sc">+</span> hp <span class="sc">*</span> domestic, <span class="at">data =</span> autompg)</span>
+<span id="cb799-2"><a href="categorical-predictors-and-interactions.html#cb799-2" tabindex="-1"></a><span class="co">#two_way_int_mod = lm(mpg ~ (disp + hp + domestic) ^ 2, data = autompg)</span></span>
+<span id="cb799-3"><a href="categorical-predictors-and-interactions.html#cb799-3" tabindex="-1"></a><span class="fu">anova</span>(two_way_int_mod, big_model)</span></code></pre></div>
 <pre><code>## Analysis of Variance Table
 ## 
 ## Model 1: mpg ~ disp * hp + disp * domestic + hp * domestic
@@ -1346,9 +1346,9 @@ <h2><span class="header-section-number">11.5</span> Building Larger Models<a hre
 ## 2    375 5645.6  1    27.599 1.8332 0.1766</code></pre>
 <p>We see the p-value is somewhat large, so we would fail to reject. We prefer the smaller, less flexible, null model, without the three-way interaction.</p>
 <p>A quick note here: the full model does still “fit better.” Notice that it has a smaller RMSE than the null model, which means the full model makes smaller (squared) errors on average.</p>
-<div class="sourceCode" id="cb801"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb801-1"><a href="categorical-predictors-and-interactions.html#cb801-1" aria-hidden="true" tabindex="-1"></a><span class="fu">mean</span>(<span class="fu">resid</span>(big_model) <span class="sc">^</span> <span class="dv">2</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb801"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb801-1"><a href="categorical-predictors-and-interactions.html#cb801-1" tabindex="-1"></a><span class="fu">mean</span>(<span class="fu">resid</span>(big_model) <span class="sc">^</span> <span class="dv">2</span>)</span></code></pre></div>
 <pre><code>## [1] 14.74053</code></pre>
-<div class="sourceCode" id="cb803"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb803-1"><a href="categorical-predictors-and-interactions.html#cb803-1" aria-hidden="true" tabindex="-1"></a><span class="fu">mean</span>(<span class="fu">resid</span>(two_way_int_mod) <span class="sc">^</span> <span class="dv">2</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb803"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb803-1"><a href="categorical-predictors-and-interactions.html#cb803-1" tabindex="-1"></a><span class="fu">mean</span>(<span class="fu">resid</span>(two_way_int_mod) <span class="sc">^</span> <span class="dv">2</span>)</span></code></pre></div>
 <pre><code>## [1] 14.81259</code></pre>
 <p>However, it is not much smaller. We could even say that the difference is insignificant. This is an idea we will return to later in greater detail.</p>
 <p>Now that we have chosen the model without the three-way interaction, can we go further? Do we need the two-way interactions? Let’s test</p>
@@ -1361,8 +1361,8 @@ <h2><span class="header-section-number">11.5</span> Building Larger Models<a hre
 <li>Null Model: <span class="math inline">\(Y = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \beta_3 x_3 + \epsilon\)</span></li>
 </ul>
 <p>We fit the null model in <code>R</code> as <code>additive_mod</code>, then use <code>anova()</code> to perform an <span class="math inline">\(F\)</span>-test as usual.</p>
-<div class="sourceCode" id="cb805"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb805-1"><a href="categorical-predictors-and-interactions.html#cb805-1" aria-hidden="true" tabindex="-1"></a>additive_mod <span class="ot">=</span> <span class="fu">lm</span>(mpg <span class="sc">~</span> disp <span class="sc">+</span> hp <span class="sc">+</span> domestic, <span class="at">data =</span> autompg)</span>
-<span id="cb805-2"><a href="categorical-predictors-and-interactions.html#cb805-2" aria-hidden="true" tabindex="-1"></a><span class="fu">anova</span>(additive_mod, two_way_int_mod)</span></code></pre></div>
+<div class="sourceCode" id="cb805"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb805-1"><a href="categorical-predictors-and-interactions.html#cb805-1" tabindex="-1"></a>additive_mod <span class="ot">=</span> <span class="fu">lm</span>(mpg <span class="sc">~</span> disp <span class="sc">+</span> hp <span class="sc">+</span> domestic, <span class="at">data =</span> autompg)</span>
+<span id="cb805-2"><a href="categorical-predictors-and-interactions.html#cb805-2" tabindex="-1"></a><span class="fu">anova</span>(additive_mod, two_way_int_mod)</span></code></pre></div>
 <pre><code>## Analysis of Variance Table
 ## 
 ## Model 1: mpg ~ disp + hp + domestic
@@ -1383,7 +1383,7 @@ <h2><span class="header-section-number">11.6</span> <code>R</code> Markdown<a hr
 <ul>
 <li><a href="cat-int.Rmd" target="_blank"><code>cat-int.Rmd</code></a></li>
 </ul>
-<p>The file was created using <code>R</code> version <code>4.3.2</code>.</p>
+<p>The file was created using <code>R</code> version <code>4.4.1</code>.</p>
 
 </div>
 </div>
diff --git a/collinearity.html b/collinearity.html
index c75b605..4d3e952 100644
--- a/collinearity.html
+++ b/collinearity.html
@@ -6,7 +6,7 @@
   <meta http-equiv="X-UA-Compatible" content="IE=edge" />
   <title>Chapter 15 Collinearity | Applied Statistics with R</title>
   <meta name="description" content="Chapter 15 Collinearity | Applied Statistics with R" />
-  <meta name="generator" content="bookdown 0.37 and GitBook 2.6.7" />
+  <meta name="generator" content="bookdown 0.40 and GitBook 2.6.7" />
 
   <meta property="og:title" content="Chapter 15 Collinearity | Applied Statistics with R" />
   <meta property="og:type" content="book" />
@@ -55,7 +55,7 @@
 
 <style type="text/css">
 pre > code.sourceCode { white-space: pre; position: relative; }
-pre > code.sourceCode > span { display: inline-block; line-height: 1.25; }
+pre > code.sourceCode > span { line-height: 1.25; }
 pre > code.sourceCode > span:empty { height: 1.2em; }
 .sourceCode { overflow: visible; }
 code.sourceCode > span { color: inherit; text-decoration: inherit; }
@@ -478,17 +478,17 @@ <h1><span class="header-section-number">Chapter 15</span> Collinearity<a href="c
 <div id="exact-collinearity" class="section level2 hasAnchor" number="15.1">
 <h2><span class="header-section-number">15.1</span> Exact Collinearity<a href="collinearity.html#exact-collinearity" class="anchor-section" aria-label="Anchor link to header"></a></h2>
 <p>Let’s create a dataset where one of the predictors, <span class="math inline">\(x_3\)</span>, is a linear combination of the other predictors.</p>
-<div class="sourceCode" id="cb1210"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1210-1"><a href="collinearity.html#cb1210-1" aria-hidden="true" tabindex="-1"></a>gen_exact_collin_data <span class="ot">=</span> <span class="cf">function</span>(<span class="at">num_samples =</span> <span class="dv">100</span>) {</span>
-<span id="cb1210-2"><a href="collinearity.html#cb1210-2" aria-hidden="true" tabindex="-1"></a>  x1 <span class="ot">=</span> <span class="fu">rnorm</span>(<span class="at">n =</span> num_samples, <span class="at">mean =</span> <span class="dv">80</span>, <span class="at">sd =</span> <span class="dv">10</span>)</span>
-<span id="cb1210-3"><a href="collinearity.html#cb1210-3" aria-hidden="true" tabindex="-1"></a>  x2 <span class="ot">=</span> <span class="fu">rnorm</span>(<span class="at">n =</span> num_samples, <span class="at">mean =</span> <span class="dv">70</span>, <span class="at">sd =</span> <span class="dv">5</span>)</span>
-<span id="cb1210-4"><a href="collinearity.html#cb1210-4" aria-hidden="true" tabindex="-1"></a>  x3 <span class="ot">=</span> <span class="dv">2</span> <span class="sc">*</span> x1 <span class="sc">+</span> <span class="dv">4</span> <span class="sc">*</span> x2 <span class="sc">+</span> <span class="dv">3</span></span>
-<span id="cb1210-5"><a href="collinearity.html#cb1210-5" aria-hidden="true" tabindex="-1"></a>  y <span class="ot">=</span> <span class="dv">3</span> <span class="sc">+</span> x1 <span class="sc">+</span> x2 <span class="sc">+</span> <span class="fu">rnorm</span>(<span class="at">n =</span> num_samples, <span class="at">mean =</span> <span class="dv">0</span>, <span class="at">sd =</span> <span class="dv">1</span>)</span>
-<span id="cb1210-6"><a href="collinearity.html#cb1210-6" aria-hidden="true" tabindex="-1"></a>  <span class="fu">data.frame</span>(y, x1, x2, x3)</span>
-<span id="cb1210-7"><a href="collinearity.html#cb1210-7" aria-hidden="true" tabindex="-1"></a>}</span></code></pre></div>
+<div class="sourceCode" id="cb1210"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1210-1"><a href="collinearity.html#cb1210-1" tabindex="-1"></a>gen_exact_collin_data <span class="ot">=</span> <span class="cf">function</span>(<span class="at">num_samples =</span> <span class="dv">100</span>) {</span>
+<span id="cb1210-2"><a href="collinearity.html#cb1210-2" tabindex="-1"></a>  x1 <span class="ot">=</span> <span class="fu">rnorm</span>(<span class="at">n =</span> num_samples, <span class="at">mean =</span> <span class="dv">80</span>, <span class="at">sd =</span> <span class="dv">10</span>)</span>
+<span id="cb1210-3"><a href="collinearity.html#cb1210-3" tabindex="-1"></a>  x2 <span class="ot">=</span> <span class="fu">rnorm</span>(<span class="at">n =</span> num_samples, <span class="at">mean =</span> <span class="dv">70</span>, <span class="at">sd =</span> <span class="dv">5</span>)</span>
+<span id="cb1210-4"><a href="collinearity.html#cb1210-4" tabindex="-1"></a>  x3 <span class="ot">=</span> <span class="dv">2</span> <span class="sc">*</span> x1 <span class="sc">+</span> <span class="dv">4</span> <span class="sc">*</span> x2 <span class="sc">+</span> <span class="dv">3</span></span>
+<span id="cb1210-5"><a href="collinearity.html#cb1210-5" tabindex="-1"></a>  y <span class="ot">=</span> <span class="dv">3</span> <span class="sc">+</span> x1 <span class="sc">+</span> x2 <span class="sc">+</span> <span class="fu">rnorm</span>(<span class="at">n =</span> num_samples, <span class="at">mean =</span> <span class="dv">0</span>, <span class="at">sd =</span> <span class="dv">1</span>)</span>
+<span id="cb1210-6"><a href="collinearity.html#cb1210-6" tabindex="-1"></a>  <span class="fu">data.frame</span>(y, x1, x2, x3)</span>
+<span id="cb1210-7"><a href="collinearity.html#cb1210-7" tabindex="-1"></a>}</span></code></pre></div>
 <p>Notice that the way we are generating this data, the response <span class="math inline">\(y\)</span> only really depends on <span class="math inline">\(x_1\)</span> and <span class="math inline">\(x_2\)</span>.</p>
-<div class="sourceCode" id="cb1211"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1211-1"><a href="collinearity.html#cb1211-1" aria-hidden="true" tabindex="-1"></a><span class="fu">set.seed</span>(<span class="dv">42</span>)</span>
-<span id="cb1211-2"><a href="collinearity.html#cb1211-2" aria-hidden="true" tabindex="-1"></a>exact_collin_data <span class="ot">=</span> <span class="fu">gen_exact_collin_data</span>()</span>
-<span id="cb1211-3"><a href="collinearity.html#cb1211-3" aria-hidden="true" tabindex="-1"></a><span class="fu">head</span>(exact_collin_data)</span></code></pre></div>
+<div class="sourceCode" id="cb1211"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1211-1"><a href="collinearity.html#cb1211-1" tabindex="-1"></a><span class="fu">set.seed</span>(<span class="dv">42</span>)</span>
+<span id="cb1211-2"><a href="collinearity.html#cb1211-2" tabindex="-1"></a>exact_collin_data <span class="ot">=</span> <span class="fu">gen_exact_collin_data</span>()</span>
+<span id="cb1211-3"><a href="collinearity.html#cb1211-3" tabindex="-1"></a><span class="fu">head</span>(exact_collin_data)</span></code></pre></div>
 <pre><code>##          y       x1       x2       x3
 ## 1 170.7135 93.70958 76.00483 494.4385
 ## 2 152.9106 74.35302 75.22376 452.6011
@@ -497,8 +497,8 @@ <h2><span class="header-section-number">15.1</span> Exact Collinearity<a href="c
 ## 5 152.3320 84.04268 66.66613 437.7499
 ## 6 151.3155 78.93875 70.52757 442.9878</code></pre>
 <p>What happens when we attempt to fit a regression model in <code>R</code> using all of the predictors?</p>
-<div class="sourceCode" id="cb1213"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1213-1"><a href="collinearity.html#cb1213-1" aria-hidden="true" tabindex="-1"></a>exact_collin_fit <span class="ot">=</span> <span class="fu">lm</span>(y <span class="sc">~</span> x1 <span class="sc">+</span> x2 <span class="sc">+</span> x3, <span class="at">data =</span> exact_collin_data)</span>
-<span id="cb1213-2"><a href="collinearity.html#cb1213-2" aria-hidden="true" tabindex="-1"></a><span class="fu">summary</span>(exact_collin_fit)</span></code></pre></div>
+<div class="sourceCode" id="cb1213"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1213-1"><a href="collinearity.html#cb1213-1" tabindex="-1"></a>exact_collin_fit <span class="ot">=</span> <span class="fu">lm</span>(y <span class="sc">~</span> x1 <span class="sc">+</span> x2 <span class="sc">+</span> x3, <span class="at">data =</span> exact_collin_data)</span>
+<span id="cb1213-2"><a href="collinearity.html#cb1213-2" tabindex="-1"></a><span class="fu">summary</span>(exact_collin_fit)</span></code></pre></div>
 <pre><code>## 
 ## Call:
 ## lm(formula = y ~ x1 + x2 + x3, data = exact_collin_data)
@@ -520,27 +520,27 @@ <h2><span class="header-section-number">15.1</span> Exact Collinearity<a href="c
 ## Multiple R-squared:  0.9923, Adjusted R-squared:  0.9921 
 ## F-statistic:  6236 on 2 and 97 DF,  p-value: &lt; 2.2e-16</code></pre>
 <p>We see that <code>R</code> simply decides to exclude a variable. Why is this happening?</p>
-<div class="sourceCode" id="cb1215"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1215-1"><a href="collinearity.html#cb1215-1" aria-hidden="true" tabindex="-1"></a>X <span class="ot">=</span> <span class="fu">cbind</span>(<span class="dv">1</span>, <span class="fu">as.matrix</span>(exact_collin_data[,<span class="sc">-</span><span class="dv">1</span>]))</span>
-<span id="cb1215-2"><a href="collinearity.html#cb1215-2" aria-hidden="true" tabindex="-1"></a><span class="fu">solve</span>(<span class="fu">t</span>(X) <span class="sc">%*%</span> X)</span></code></pre></div>
-<p>If we attempt to find <span class="math inline">\(\boldsymbol{\hat{\beta}}\)</span> using <span class="math inline">\(\left( \boldsymbol{X}^T \boldsymbol{X} \right)^{-1}\)</span>, we see that this is not possible, due to the fact that the columns of <span class="math inline">\(\boldsymbol{X}\)</span> are linearly dependent. The previous lines of code were not run, because they produce an error!</p>
+<div class="sourceCode" id="cb1215"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1215-1"><a href="collinearity.html#cb1215-1" tabindex="-1"></a>X <span class="ot">=</span> <span class="fu">cbind</span>(<span class="dv">1</span>, <span class="fu">as.matrix</span>(exact_collin_data[,<span class="sc">-</span><span class="dv">1</span>]))</span>
+<span id="cb1215-2"><a href="collinearity.html#cb1215-2" tabindex="-1"></a><span class="fu">solve</span>(<span class="fu">t</span>(X) <span class="sc">%*%</span> X)</span></code></pre></div>
+<p>If we attempt to find <span class="math inline">\(\boldsymbol{\hat{\beta}}\)</span> using <span class="math inline">\(\left(  \boldsymbol{X}^T \boldsymbol{X}  \right)^{-1}\)</span>, we see that this is not possible, due to the fact that the columns of <span class="math inline">\(\boldsymbol{X}\)</span> are linearly dependent. The previous lines of code were not run, because they produce an error!</p>
 <p>When this happens, we say there is <strong>exact collinearity</strong> in the dataset.</p>
 <p>As a result of this issue, <code>R</code> essentially chose to fit the model <code>y ~ x1 + x2</code>. However notice that two other models would accomplish exactly the same fit.</p>
-<div class="sourceCode" id="cb1216"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1216-1"><a href="collinearity.html#cb1216-1" aria-hidden="true" tabindex="-1"></a>fit1 <span class="ot">=</span> <span class="fu">lm</span>(y <span class="sc">~</span> x1 <span class="sc">+</span> x2, <span class="at">data =</span> exact_collin_data)</span>
-<span id="cb1216-2"><a href="collinearity.html#cb1216-2" aria-hidden="true" tabindex="-1"></a>fit2 <span class="ot">=</span> <span class="fu">lm</span>(y <span class="sc">~</span> x1 <span class="sc">+</span> x3, <span class="at">data =</span> exact_collin_data)</span>
-<span id="cb1216-3"><a href="collinearity.html#cb1216-3" aria-hidden="true" tabindex="-1"></a>fit3 <span class="ot">=</span> <span class="fu">lm</span>(y <span class="sc">~</span> x2 <span class="sc">+</span> x3, <span class="at">data =</span> exact_collin_data)</span></code></pre></div>
+<div class="sourceCode" id="cb1216"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1216-1"><a href="collinearity.html#cb1216-1" tabindex="-1"></a>fit1 <span class="ot">=</span> <span class="fu">lm</span>(y <span class="sc">~</span> x1 <span class="sc">+</span> x2, <span class="at">data =</span> exact_collin_data)</span>
+<span id="cb1216-2"><a href="collinearity.html#cb1216-2" tabindex="-1"></a>fit2 <span class="ot">=</span> <span class="fu">lm</span>(y <span class="sc">~</span> x1 <span class="sc">+</span> x3, <span class="at">data =</span> exact_collin_data)</span>
+<span id="cb1216-3"><a href="collinearity.html#cb1216-3" tabindex="-1"></a>fit3 <span class="ot">=</span> <span class="fu">lm</span>(y <span class="sc">~</span> x2 <span class="sc">+</span> x3, <span class="at">data =</span> exact_collin_data)</span></code></pre></div>
 <p>We see that the fitted values for each of the three models are exactly the same. This is a result of <span class="math inline">\(x_3\)</span> containing all of the information from <span class="math inline">\(x_1\)</span> and <span class="math inline">\(x_2\)</span>. As long as one of <span class="math inline">\(x_1\)</span> or <span class="math inline">\(x_2\)</span> is included in the model, <span class="math inline">\(x_3\)</span> can be used to recover the information from the variable not included.</p>
-<div class="sourceCode" id="cb1217"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1217-1"><a href="collinearity.html#cb1217-1" aria-hidden="true" tabindex="-1"></a><span class="fu">all.equal</span>(<span class="fu">fitted</span>(fit1), <span class="fu">fitted</span>(fit2))</span></code></pre></div>
+<div class="sourceCode" id="cb1217"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1217-1"><a href="collinearity.html#cb1217-1" tabindex="-1"></a><span class="fu">all.equal</span>(<span class="fu">fitted</span>(fit1), <span class="fu">fitted</span>(fit2))</span></code></pre></div>
 <pre><code>## [1] TRUE</code></pre>
-<div class="sourceCode" id="cb1219"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1219-1"><a href="collinearity.html#cb1219-1" aria-hidden="true" tabindex="-1"></a><span class="fu">all.equal</span>(<span class="fu">fitted</span>(fit2), <span class="fu">fitted</span>(fit3))</span></code></pre></div>
+<div class="sourceCode" id="cb1219"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1219-1"><a href="collinearity.html#cb1219-1" tabindex="-1"></a><span class="fu">all.equal</span>(<span class="fu">fitted</span>(fit2), <span class="fu">fitted</span>(fit3))</span></code></pre></div>
 <pre><code>## [1] TRUE</code></pre>
 <p>While their fitted values are all the same, their estimated coefficients are wildly different. The sign of <span class="math inline">\(x_2\)</span> is switched in two of the models! So only <code>fit1</code> properly <em>explains</em> the relationship between the variables, <code>fit2</code> and <code>fit3</code> still <em>predict</em> as well as <code>fit1</code>, despite the coefficients having little to no meaning, a concept we will return to later.</p>
-<div class="sourceCode" id="cb1221"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1221-1"><a href="collinearity.html#cb1221-1" aria-hidden="true" tabindex="-1"></a><span class="fu">coef</span>(fit1)</span></code></pre></div>
+<div class="sourceCode" id="cb1221"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1221-1"><a href="collinearity.html#cb1221-1" tabindex="-1"></a><span class="fu">coef</span>(fit1)</span></code></pre></div>
 <pre><code>## (Intercept)          x1          x2 
 ##   2.9573357   0.9856291   1.0170586</code></pre>
-<div class="sourceCode" id="cb1223"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1223-1"><a href="collinearity.html#cb1223-1" aria-hidden="true" tabindex="-1"></a><span class="fu">coef</span>(fit2)</span></code></pre></div>
+<div class="sourceCode" id="cb1223"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1223-1"><a href="collinearity.html#cb1223-1" tabindex="-1"></a><span class="fu">coef</span>(fit2)</span></code></pre></div>
 <pre><code>## (Intercept)          x1          x3 
 ##   2.1945418   0.4770998   0.2542647</code></pre>
-<div class="sourceCode" id="cb1225"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1225-1"><a href="collinearity.html#cb1225-1" aria-hidden="true" tabindex="-1"></a><span class="fu">coef</span>(fit3)</span></code></pre></div>
+<div class="sourceCode" id="cb1225"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1225-1"><a href="collinearity.html#cb1225-1" tabindex="-1"></a><span class="fu">coef</span>(fit3)</span></code></pre></div>
 <pre><code>## (Intercept)          x2          x3 
 ##   1.4788921  -0.9541995   0.4928145</code></pre>
 </div>
@@ -549,10 +549,10 @@ <h2><span class="header-section-number">15.2</span> Collinearity<a href="colline
 <p>Exact collinearity is an extreme example of <strong>collinearity</strong>, which occurs in multiple regression when predictor variables are highly correlated. Collinearity is often called <em>multicollinearity</em>, since it is a phenomenon that really only occurs during multiple regression.</p>
 <p>Looking at the <code>seatpos</code> dataset from the <code>faraway</code> package, we will see an example of this concept. The predictors in this dataset are various attributes of car drivers, such as their height, weight and age. The response variable <code>hipcenter</code> measures the “horizontal distance of the midpoint of the hips from a fixed location in the car in mm.” Essentially, it measures the position of the seat for a given driver. This is potentially useful information for car manufacturers considering comfort and safety when designing vehicles.</p>
 <p>We will attempt to fit a model that predicts <code>hipcenter</code>. Two predictor variables are immediately interesting to us: <code>HtShoes</code> and <code>Ht</code>. We certainly expect a person’s height to be highly correlated to their height when wearing shoes. We’ll pay special attention to these two variables when fitting models.</p>
-<div class="sourceCode" id="cb1227"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1227-1"><a href="collinearity.html#cb1227-1" aria-hidden="true" tabindex="-1"></a><span class="fu">library</span>(faraway)</span>
-<span id="cb1227-2"><a href="collinearity.html#cb1227-2" aria-hidden="true" tabindex="-1"></a><span class="fu">pairs</span>(seatpos, <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1227"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1227-1"><a href="collinearity.html#cb1227-1" tabindex="-1"></a><span class="fu">library</span>(faraway)</span>
+<span id="cb1227-2"><a href="collinearity.html#cb1227-2" tabindex="-1"></a><span class="fu">pairs</span>(seatpos, <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>)</span></code></pre></div>
 <p><img src="collinearity_files/figure-html/unnamed-chunk-9-1.png" width="768" style="display: block; margin: auto;" /></p>
-<div class="sourceCode" id="cb1228"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1228-1"><a href="collinearity.html#cb1228-1" aria-hidden="true" tabindex="-1"></a><span class="fu">round</span>(<span class="fu">cor</span>(seatpos), <span class="dv">2</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1228"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1228-1"><a href="collinearity.html#cb1228-1" tabindex="-1"></a><span class="fu">round</span>(<span class="fu">cor</span>(seatpos), <span class="dv">2</span>)</span></code></pre></div>
 <pre><code>##             Age Weight HtShoes    Ht Seated   Arm Thigh   Leg hipcenter
 ## Age        1.00   0.08   -0.08 -0.09  -0.17  0.36  0.09 -0.04      0.21
 ## Weight     0.08   1.00    0.83  0.83   0.78  0.70  0.57  0.78     -0.64
@@ -566,8 +566,8 @@ <h2><span class="header-section-number">15.2</span> Collinearity<a href="colline
 <p>After loading the <code>faraway</code> package, we do some quick checks of correlation between the predictors. Visually, we can do this with the <code>pairs()</code> function, which plots all possible scatterplots between pairs of variables in the dataset.</p>
 <p>We can also do this numerically with the <code>cor()</code> function, which when applied to a dataset, returns all pairwise correlations. Notice this is a symmetric matrix. Recall that correlation measures strength and direction of the linear relationship between two variables. The correlation between <code>Ht</code> and <code>HtShoes</code> is extremely high. So high, that rounded to two decimal places, it appears to be 1!</p>
 <p>Unlike exact collinearity, here we can still fit a model with all of the predictors, but what effect does this have?</p>
-<div class="sourceCode" id="cb1230"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1230-1"><a href="collinearity.html#cb1230-1" aria-hidden="true" tabindex="-1"></a>hip_model <span class="ot">=</span> <span class="fu">lm</span>(hipcenter <span class="sc">~</span> ., <span class="at">data =</span> seatpos)</span>
-<span id="cb1230-2"><a href="collinearity.html#cb1230-2" aria-hidden="true" tabindex="-1"></a><span class="fu">summary</span>(hip_model)</span></code></pre></div>
+<div class="sourceCode" id="cb1230"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1230-1"><a href="collinearity.html#cb1230-1" tabindex="-1"></a>hip_model <span class="ot">=</span> <span class="fu">lm</span>(hipcenter <span class="sc">~</span> ., <span class="at">data =</span> seatpos)</span>
+<span id="cb1230-2"><a href="collinearity.html#cb1230-2" tabindex="-1"></a><span class="fu">summary</span>(hip_model)</span></code></pre></div>
 <pre><code>## 
 ## Call:
 ## lm(formula = hipcenter ~ ., data = seatpos)
@@ -596,8 +596,8 @@ <h2><span class="header-section-number">15.2</span> Collinearity<a href="colline
 <p>One of the first things we should notice is that the <span class="math inline">\(F\)</span>-test for the regression tells us that the regression is significant, however each individual predictor is not. Another interesting result is the opposite signs of the coefficients for <code>Ht</code> and <code>HtShoes</code>. This should seem rather counter-intuitive. Increasing <code>Ht</code> increases <code>hipcenter</code>, but increasing <code>HtShoes</code> decreases <code>hipcenter</code>?</p>
 <p>This happens as a result of the predictors being highly correlated. For example, the <code>HtShoe</code> variable explains a large amount of the variation in <code>Ht</code>. When they are both in the model, their effects on the response are lessened individually, but together they still explain a large portion of the variation of <code>hipcenter</code>.</p>
 <p>We define <span class="math inline">\(R_j^2\)</span> to be the proportion of observed variation in the <span class="math inline">\(j\)</span>-th predictor explained by the other predictors. In other words <span class="math inline">\(R_j^2\)</span> is the multiple R-Squared for the regression of <span class="math inline">\(x_j\)</span> on each of the other predictors.</p>
-<div class="sourceCode" id="cb1232"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1232-1"><a href="collinearity.html#cb1232-1" aria-hidden="true" tabindex="-1"></a>ht_shoes_model <span class="ot">=</span> <span class="fu">lm</span>(HtShoes <span class="sc">~</span> . <span class="sc">-</span> hipcenter, <span class="at">data =</span> seatpos)</span>
-<span id="cb1232-2"><a href="collinearity.html#cb1232-2" aria-hidden="true" tabindex="-1"></a><span class="fu">summary</span>(ht_shoes_model)<span class="sc">$</span>r.squared</span></code></pre></div>
+<div class="sourceCode" id="cb1232"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1232-1"><a href="collinearity.html#cb1232-1" tabindex="-1"></a>ht_shoes_model <span class="ot">=</span> <span class="fu">lm</span>(HtShoes <span class="sc">~</span> . <span class="sc">-</span> hipcenter, <span class="at">data =</span> seatpos)</span>
+<span id="cb1232-2"><a href="collinearity.html#cb1232-2" tabindex="-1"></a><span class="fu">summary</span>(ht_shoes_model)<span class="sc">$</span>r.squared</span></code></pre></div>
 <pre><code>## [1] 0.9967472</code></pre>
 <p>Here we see that the other predictors explain <span class="math inline">\(99.67\%\)</span> of the variation in <code>HtShoe</code>. When fitting this model, we removed <code>hipcenter</code> since it is not a predictor.</p>
 <div id="variance-inflation-factor." class="section level3 hasAnchor" number="15.2.1">
@@ -617,36 +617,36 @@ <h3><span class="header-section-number">15.2.1</span> Variance Inflation Factor.
 \]</span></p>
 <p>the <strong>variance inflation factor.</strong> The variance inflation factor quantifies the effect of collinearity on the variance of our regression estimates. When <span class="math inline">\(R_j^2\)</span> is large, that is close to 1, <span class="math inline">\(x_j\)</span> is well explained by the other predictors. With a large <span class="math inline">\(R_j^2\)</span> the variance inflation factor becomes large. This tells us that when <span class="math inline">\(x_j\)</span> is highly correlated with other predictors, our estimate of <span class="math inline">\(\beta_j\)</span> is highly variable.</p>
 <p>The <code>vif</code> function from the <code>faraway</code> package calculates the VIFs for each of the predictors of a model.</p>
-<div class="sourceCode" id="cb1234"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1234-1"><a href="collinearity.html#cb1234-1" aria-hidden="true" tabindex="-1"></a><span class="fu">vif</span>(hip_model)</span></code></pre></div>
+<div class="sourceCode" id="cb1234"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1234-1"><a href="collinearity.html#cb1234-1" tabindex="-1"></a><span class="fu">vif</span>(hip_model)</span></code></pre></div>
 <pre><code>##        Age     Weight    HtShoes         Ht     Seated        Arm      Thigh 
 ##   1.997931   3.647030 307.429378 333.137832   8.951054   4.496368   2.762886 
 ##        Leg 
 ##   6.694291</code></pre>
 <p>In practice it is common to say that any VIF greater than <span class="math inline">\(5\)</span> is cause for concern. So in this example we see there is a huge multicollinearity issue as many of the predictors have a VIF greater than 5.</p>
 <p>Let’s further investigate how the presence of collinearity actually affects a model. If we add a moderate amount of noise to the data, we see that the estimates of the coefficients change drastically. This is a rather undesirable effect. Adding random noise should not affect the coefficients of a model.</p>
-<div class="sourceCode" id="cb1236"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1236-1"><a href="collinearity.html#cb1236-1" aria-hidden="true" tabindex="-1"></a><span class="fu">set.seed</span>(<span class="dv">1337</span>)</span>
-<span id="cb1236-2"><a href="collinearity.html#cb1236-2" aria-hidden="true" tabindex="-1"></a>noise <span class="ot">=</span> <span class="fu">rnorm</span>(<span class="at">n =</span> <span class="fu">nrow</span>(seatpos), <span class="at">mean =</span> <span class="dv">0</span>, <span class="at">sd =</span> <span class="dv">5</span>)</span>
-<span id="cb1236-3"><a href="collinearity.html#cb1236-3" aria-hidden="true" tabindex="-1"></a>hip_model_noise <span class="ot">=</span> <span class="fu">lm</span>(hipcenter <span class="sc">+</span> noise <span class="sc">~</span> ., <span class="at">data =</span> seatpos)</span></code></pre></div>
+<div class="sourceCode" id="cb1236"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1236-1"><a href="collinearity.html#cb1236-1" tabindex="-1"></a><span class="fu">set.seed</span>(<span class="dv">1337</span>)</span>
+<span id="cb1236-2"><a href="collinearity.html#cb1236-2" tabindex="-1"></a>noise <span class="ot">=</span> <span class="fu">rnorm</span>(<span class="at">n =</span> <span class="fu">nrow</span>(seatpos), <span class="at">mean =</span> <span class="dv">0</span>, <span class="at">sd =</span> <span class="dv">5</span>)</span>
+<span id="cb1236-3"><a href="collinearity.html#cb1236-3" tabindex="-1"></a>hip_model_noise <span class="ot">=</span> <span class="fu">lm</span>(hipcenter <span class="sc">+</span> noise <span class="sc">~</span> ., <span class="at">data =</span> seatpos)</span></code></pre></div>
 <p>Adding the noise had such a large effect, the sign of the coefficient for <code>Ht</code> has changed.</p>
-<div class="sourceCode" id="cb1237"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1237-1"><a href="collinearity.html#cb1237-1" aria-hidden="true" tabindex="-1"></a><span class="fu">coef</span>(hip_model)</span></code></pre></div>
+<div class="sourceCode" id="cb1237"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1237-1"><a href="collinearity.html#cb1237-1" tabindex="-1"></a><span class="fu">coef</span>(hip_model)</span></code></pre></div>
 <pre><code>##  (Intercept)          Age       Weight      HtShoes           Ht       Seated 
 ## 436.43212823   0.77571620   0.02631308  -2.69240774   0.60134458   0.53375170 
 ##          Arm        Thigh          Leg 
 ##  -1.32806864  -1.14311888  -6.43904627</code></pre>
-<div class="sourceCode" id="cb1239"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1239-1"><a href="collinearity.html#cb1239-1" aria-hidden="true" tabindex="-1"></a><span class="fu">coef</span>(hip_model_noise)</span></code></pre></div>
+<div class="sourceCode" id="cb1239"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1239-1"><a href="collinearity.html#cb1239-1" tabindex="-1"></a><span class="fu">coef</span>(hip_model_noise)</span></code></pre></div>
 <pre><code>##  (Intercept)          Age       Weight      HtShoes           Ht       Seated 
 ## 415.32909380   0.76578240   0.01910958  -2.90377584  -0.12068122   2.03241638 
 ##          Arm        Thigh          Leg 
 ##  -1.02127944  -0.89034509  -5.61777220</code></pre>
 <p>This tells us that a model with collinearity is bad at explaining the relationship between the response and the predictors. We cannot even be confident in the direction of the relationship. However, does collinearity affect prediction?</p>
-<div class="sourceCode" id="cb1241"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1241-1"><a href="collinearity.html#cb1241-1" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(<span class="fu">fitted</span>(hip_model), <span class="fu">fitted</span>(hip_model_noise), <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>,</span>
-<span id="cb1241-2"><a href="collinearity.html#cb1241-2" aria-hidden="true" tabindex="-1"></a>     <span class="at">xlab =</span> <span class="st">&quot;Predicted, Without Noise&quot;</span>, <span class="at">ylab =</span> <span class="st">&quot;Predicted, With Noise&quot;</span>, <span class="at">cex =</span> <span class="fl">1.5</span>)</span>
-<span id="cb1241-3"><a href="collinearity.html#cb1241-3" aria-hidden="true" tabindex="-1"></a><span class="fu">abline</span>(<span class="at">a =</span> <span class="dv">0</span>, <span class="at">b =</span> <span class="dv">1</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1241"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1241-1"><a href="collinearity.html#cb1241-1" tabindex="-1"></a><span class="fu">plot</span>(<span class="fu">fitted</span>(hip_model), <span class="fu">fitted</span>(hip_model_noise), <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>,</span>
+<span id="cb1241-2"><a href="collinearity.html#cb1241-2" tabindex="-1"></a>     <span class="at">xlab =</span> <span class="st">&quot;Predicted, Without Noise&quot;</span>, <span class="at">ylab =</span> <span class="st">&quot;Predicted, With Noise&quot;</span>, <span class="at">cex =</span> <span class="fl">1.5</span>)</span>
+<span id="cb1241-3"><a href="collinearity.html#cb1241-3" tabindex="-1"></a><span class="fu">abline</span>(<span class="at">a =</span> <span class="dv">0</span>, <span class="at">b =</span> <span class="dv">1</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
 <p><img src="collinearity_files/figure-html/unnamed-chunk-15-1.png" width="672" style="display: block; margin: auto;" /></p>
 <p>We see that by plotting the predicted values using both models against each other, they are actually rather similar.</p>
 <p>Let’s now look at a smaller model,</p>
-<div class="sourceCode" id="cb1242"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1242-1"><a href="collinearity.html#cb1242-1" aria-hidden="true" tabindex="-1"></a>hip_model_small <span class="ot">=</span> <span class="fu">lm</span>(hipcenter <span class="sc">~</span> Age <span class="sc">+</span> Arm <span class="sc">+</span> Ht, <span class="at">data =</span> seatpos)</span>
-<span id="cb1242-2"><a href="collinearity.html#cb1242-2" aria-hidden="true" tabindex="-1"></a><span class="fu">summary</span>(hip_model_small)</span></code></pre></div>
+<div class="sourceCode" id="cb1242"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1242-1"><a href="collinearity.html#cb1242-1" tabindex="-1"></a>hip_model_small <span class="ot">=</span> <span class="fu">lm</span>(hipcenter <span class="sc">~</span> Age <span class="sc">+</span> Arm <span class="sc">+</span> Ht, <span class="at">data =</span> seatpos)</span>
+<span id="cb1242-2"><a href="collinearity.html#cb1242-2" tabindex="-1"></a><span class="fu">summary</span>(hip_model_small)</span></code></pre></div>
 <pre><code>## 
 ## Call:
 ## lm(formula = hipcenter ~ Age + Arm + Ht, data = seatpos)
@@ -667,11 +667,11 @@ <h3><span class="header-section-number">15.2.1</span> Variance Inflation Factor.
 ## Residual standard error: 36.12 on 34 degrees of freedom
 ## Multiple R-squared:  0.6631, Adjusted R-squared:  0.6333 
 ## F-statistic:  22.3 on 3 and 34 DF,  p-value: 3.649e-08</code></pre>
-<div class="sourceCode" id="cb1244"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1244-1"><a href="collinearity.html#cb1244-1" aria-hidden="true" tabindex="-1"></a><span class="fu">vif</span>(hip_model_small)</span></code></pre></div>
+<div class="sourceCode" id="cb1244"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1244-1"><a href="collinearity.html#cb1244-1" tabindex="-1"></a><span class="fu">vif</span>(hip_model_small)</span></code></pre></div>
 <pre><code>##      Age      Arm       Ht 
 ## 1.749943 3.996766 3.508693</code></pre>
 <p>Immediately we see that multicollinearity isn’t an issue here.</p>
-<div class="sourceCode" id="cb1246"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1246-1"><a href="collinearity.html#cb1246-1" aria-hidden="true" tabindex="-1"></a><span class="fu">anova</span>(hip_model_small, hip_model)</span></code></pre></div>
+<div class="sourceCode" id="cb1246"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1246-1"><a href="collinearity.html#cb1246-1" tabindex="-1"></a><span class="fu">anova</span>(hip_model_small, hip_model)</span></code></pre></div>
 <pre><code>## Analysis of Variance Table
 ## 
 ## Model 1: hipcenter ~ Age + Arm + Ht
@@ -687,22 +687,22 @@ <h3><span class="header-section-number">15.2.1</span> Variance Inflation Factor.
 <li>Regressing the response (<code>hipcenter</code>) against all of the predictors except the predictor of interest (<code>HtShoes</code>).</li>
 <li>Regressing the predictor of interest (<code>HtShoes</code>) against the other predictors (<code>Age</code>, <code>Arm</code>, and <code>Ht</code>).</li>
 </ul>
-<div class="sourceCode" id="cb1248"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1248-1"><a href="collinearity.html#cb1248-1" aria-hidden="true" tabindex="-1"></a>ht_shoes_model_small <span class="ot">=</span> <span class="fu">lm</span>(HtShoes <span class="sc">~</span> Age <span class="sc">+</span> Arm <span class="sc">+</span> Ht, <span class="at">data =</span> seatpos)</span></code></pre></div>
+<div class="sourceCode" id="cb1248"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1248-1"><a href="collinearity.html#cb1248-1" tabindex="-1"></a>ht_shoes_model_small <span class="ot">=</span> <span class="fu">lm</span>(HtShoes <span class="sc">~</span> Age <span class="sc">+</span> Arm <span class="sc">+</span> Ht, <span class="at">data =</span> seatpos)</span></code></pre></div>
 <p>So now, the residuals of <code>hip_model_small</code> give us the variation of <code>hipcenter</code> that is <em>unexplained</em> by <code>Age</code>, <code>Arm</code>, and <code>Ht</code>. Similarly, the residuals of <code>ht_shoes_model_small</code> give us the variation of <code>HtShoes</code> unexplained by <code>Age</code>, <code>Arm</code>, and <code>Ht</code>.</p>
 <p>The correlation of these two residuals gives us the <strong>partial correlation coefficient</strong> of <code>HtShoes</code> and <code>hipcenter</code> with the effects of <code>Age</code>, <code>Arm</code>, and <code>Ht</code> removed.</p>
-<div class="sourceCode" id="cb1249"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1249-1"><a href="collinearity.html#cb1249-1" aria-hidden="true" tabindex="-1"></a><span class="fu">cor</span>(<span class="fu">resid</span>(ht_shoes_model_small), <span class="fu">resid</span>(hip_model_small))</span></code></pre></div>
+<div class="sourceCode" id="cb1249"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1249-1"><a href="collinearity.html#cb1249-1" tabindex="-1"></a><span class="fu">cor</span>(<span class="fu">resid</span>(ht_shoes_model_small), <span class="fu">resid</span>(hip_model_small))</span></code></pre></div>
 <pre><code>## [1] -0.03311061</code></pre>
 <p>Since this value is small, close to zero, it means that the variation of <code>hipcenter</code> that is unexplained by <code>Age</code>, <code>Arm</code>, and <code>Ht</code> shows very little correlation with the variation of <code>HtShoes</code> that is not explained by <code>Age</code>, <code>Arm</code>, and <code>Ht</code>. Thus adding <code>HtShoes</code> to the model would likely be of little benefit.</p>
 <p>Similarly a <strong>variable added plot</strong> visualizes these residuals against each other. It is also helpful to regress the residuals of the response against the residuals of the predictor and add the regression line to the plot.</p>
-<div class="sourceCode" id="cb1251"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1251-1"><a href="collinearity.html#cb1251-1" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(<span class="fu">resid</span>(hip_model_small) <span class="sc">~</span> <span class="fu">resid</span>(ht_shoes_model_small), </span>
-<span id="cb1251-2"><a href="collinearity.html#cb1251-2" aria-hidden="true" tabindex="-1"></a>     <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>,</span>
-<span id="cb1251-3"><a href="collinearity.html#cb1251-3" aria-hidden="true" tabindex="-1"></a>     <span class="at">xlab =</span> <span class="st">&quot;Residuals, Added Predictor&quot;</span>, </span>
-<span id="cb1251-4"><a href="collinearity.html#cb1251-4" aria-hidden="true" tabindex="-1"></a>     <span class="at">ylab =</span> <span class="st">&quot;Residuals, Original Model&quot;</span>,</span>
-<span id="cb1251-5"><a href="collinearity.html#cb1251-5" aria-hidden="true" tabindex="-1"></a>     <span class="at">main =</span> <span class="st">&quot;Variable Added Plot&quot;</span>)</span>
-<span id="cb1251-6"><a href="collinearity.html#cb1251-6" aria-hidden="true" tabindex="-1"></a><span class="fu">abline</span>(<span class="at">h =</span> <span class="dv">0</span>, <span class="at">lty =</span> <span class="dv">2</span>)</span>
-<span id="cb1251-7"><a href="collinearity.html#cb1251-7" aria-hidden="true" tabindex="-1"></a><span class="fu">abline</span>(<span class="at">v =</span> <span class="dv">0</span>, <span class="at">lty =</span> <span class="dv">2</span>)</span>
-<span id="cb1251-8"><a href="collinearity.html#cb1251-8" aria-hidden="true" tabindex="-1"></a><span class="fu">abline</span>(<span class="fu">lm</span>(<span class="fu">resid</span>(hip_model_small) <span class="sc">~</span> <span class="fu">resid</span>(ht_shoes_model_small)),</span>
-<span id="cb1251-9"><a href="collinearity.html#cb1251-9" aria-hidden="true" tabindex="-1"></a>       <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1251"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1251-1"><a href="collinearity.html#cb1251-1" tabindex="-1"></a><span class="fu">plot</span>(<span class="fu">resid</span>(hip_model_small) <span class="sc">~</span> <span class="fu">resid</span>(ht_shoes_model_small), </span>
+<span id="cb1251-2"><a href="collinearity.html#cb1251-2" tabindex="-1"></a>     <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>,</span>
+<span id="cb1251-3"><a href="collinearity.html#cb1251-3" tabindex="-1"></a>     <span class="at">xlab =</span> <span class="st">&quot;Residuals, Added Predictor&quot;</span>, </span>
+<span id="cb1251-4"><a href="collinearity.html#cb1251-4" tabindex="-1"></a>     <span class="at">ylab =</span> <span class="st">&quot;Residuals, Original Model&quot;</span>,</span>
+<span id="cb1251-5"><a href="collinearity.html#cb1251-5" tabindex="-1"></a>     <span class="at">main =</span> <span class="st">&quot;Variable Added Plot&quot;</span>)</span>
+<span id="cb1251-6"><a href="collinearity.html#cb1251-6" tabindex="-1"></a><span class="fu">abline</span>(<span class="at">h =</span> <span class="dv">0</span>, <span class="at">lty =</span> <span class="dv">2</span>)</span>
+<span id="cb1251-7"><a href="collinearity.html#cb1251-7" tabindex="-1"></a><span class="fu">abline</span>(<span class="at">v =</span> <span class="dv">0</span>, <span class="at">lty =</span> <span class="dv">2</span>)</span>
+<span id="cb1251-8"><a href="collinearity.html#cb1251-8" tabindex="-1"></a><span class="fu">abline</span>(<span class="fu">lm</span>(<span class="fu">resid</span>(hip_model_small) <span class="sc">~</span> <span class="fu">resid</span>(ht_shoes_model_small)),</span>
+<span id="cb1251-9"><a href="collinearity.html#cb1251-9" tabindex="-1"></a>       <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
 <p><img src="collinearity_files/figure-html/unnamed-chunk-20-1.png" width="672" style="display: block; margin: auto;" /></p>
 <p>Here the variable added plot shows almost no linear relationship. This tells us that adding <code>HtShoes</code> to the model would probably not be worthwhile. Since its variation is largely explained by the other predictors, adding it to the model will not do much to improve the model. However it will increase the variation of the estimates and make the model much harder to interpret.</p>
 <p>Had there been a strong linear relationship here, thus a large partial correlation coefficient, it would likely have been useful to add the additional predictor to the model.</p>
@@ -718,114 +718,114 @@ <h2><span class="header-section-number">15.3</span> Simulation<a href="collinear
 Y = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \epsilon
 \]</span></p>
 <p>where <span class="math inline">\(\epsilon \sim N(\mu = 0, \sigma^2 = 25)\)</span> and the <span class="math inline">\(\beta\)</span> coefficients defined below.</p>
-<div class="sourceCode" id="cb1252"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1252-1"><a href="collinearity.html#cb1252-1" aria-hidden="true" tabindex="-1"></a><span class="fu">set.seed</span>(<span class="dv">42</span>)</span>
-<span id="cb1252-2"><a href="collinearity.html#cb1252-2" aria-hidden="true" tabindex="-1"></a>beta_0 <span class="ot">=</span> <span class="dv">7</span></span>
-<span id="cb1252-3"><a href="collinearity.html#cb1252-3" aria-hidden="true" tabindex="-1"></a>beta_1 <span class="ot">=</span> <span class="dv">3</span></span>
-<span id="cb1252-4"><a href="collinearity.html#cb1252-4" aria-hidden="true" tabindex="-1"></a>beta_2 <span class="ot">=</span> <span class="dv">4</span></span>
-<span id="cb1252-5"><a href="collinearity.html#cb1252-5" aria-hidden="true" tabindex="-1"></a>sigma  <span class="ot">=</span> <span class="dv">5</span></span></code></pre></div>
+<div class="sourceCode" id="cb1252"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1252-1"><a href="collinearity.html#cb1252-1" tabindex="-1"></a><span class="fu">set.seed</span>(<span class="dv">42</span>)</span>
+<span id="cb1252-2"><a href="collinearity.html#cb1252-2" tabindex="-1"></a>beta_0 <span class="ot">=</span> <span class="dv">7</span></span>
+<span id="cb1252-3"><a href="collinearity.html#cb1252-3" tabindex="-1"></a>beta_1 <span class="ot">=</span> <span class="dv">3</span></span>
+<span id="cb1252-4"><a href="collinearity.html#cb1252-4" tabindex="-1"></a>beta_2 <span class="ot">=</span> <span class="dv">4</span></span>
+<span id="cb1252-5"><a href="collinearity.html#cb1252-5" tabindex="-1"></a>sigma  <span class="ot">=</span> <span class="dv">5</span></span></code></pre></div>
 <p>We will use a sample size of 10, and 2500 simulations for both situations.</p>
-<div class="sourceCode" id="cb1253"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1253-1"><a href="collinearity.html#cb1253-1" aria-hidden="true" tabindex="-1"></a>sample_size <span class="ot">=</span> <span class="dv">10</span></span>
-<span id="cb1253-2"><a href="collinearity.html#cb1253-2" aria-hidden="true" tabindex="-1"></a>num_sim     <span class="ot">=</span> <span class="dv">2500</span></span></code></pre></div>
+<div class="sourceCode" id="cb1253"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1253-1"><a href="collinearity.html#cb1253-1" tabindex="-1"></a>sample_size <span class="ot">=</span> <span class="dv">10</span></span>
+<span id="cb1253-2"><a href="collinearity.html#cb1253-2" tabindex="-1"></a>num_sim     <span class="ot">=</span> <span class="dv">2500</span></span></code></pre></div>
 <p>We’ll first consider the situation with a collinearity issue, so we manually create the two predictor variables.</p>
-<div class="sourceCode" id="cb1254"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1254-1"><a href="collinearity.html#cb1254-1" aria-hidden="true" tabindex="-1"></a>x1 <span class="ot">=</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>, <span class="dv">3</span>, <span class="dv">4</span>, <span class="dv">5</span>, <span class="dv">6</span>, <span class="dv">7</span>, <span class="dv">8</span>, <span class="dv">9</span>, <span class="dv">10</span>)</span>
-<span id="cb1254-2"><a href="collinearity.html#cb1254-2" aria-hidden="true" tabindex="-1"></a>x2 <span class="ot">=</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>, <span class="dv">3</span>, <span class="dv">4</span>, <span class="dv">5</span>, <span class="dv">7</span>, <span class="dv">6</span>, <span class="dv">10</span>, <span class="dv">9</span>, <span class="dv">8</span>)</span></code></pre></div>
-<div class="sourceCode" id="cb1255"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1255-1"><a href="collinearity.html#cb1255-1" aria-hidden="true" tabindex="-1"></a><span class="fu">c</span>(<span class="fu">sd</span>(x1), <span class="fu">sd</span>(x2))</span></code></pre></div>
+<div class="sourceCode" id="cb1254"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1254-1"><a href="collinearity.html#cb1254-1" tabindex="-1"></a>x1 <span class="ot">=</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>, <span class="dv">3</span>, <span class="dv">4</span>, <span class="dv">5</span>, <span class="dv">6</span>, <span class="dv">7</span>, <span class="dv">8</span>, <span class="dv">9</span>, <span class="dv">10</span>)</span>
+<span id="cb1254-2"><a href="collinearity.html#cb1254-2" tabindex="-1"></a>x2 <span class="ot">=</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>, <span class="dv">3</span>, <span class="dv">4</span>, <span class="dv">5</span>, <span class="dv">7</span>, <span class="dv">6</span>, <span class="dv">10</span>, <span class="dv">9</span>, <span class="dv">8</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1255"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1255-1"><a href="collinearity.html#cb1255-1" tabindex="-1"></a><span class="fu">c</span>(<span class="fu">sd</span>(x1), <span class="fu">sd</span>(x2))</span></code></pre></div>
 <pre><code>## [1] 3.02765 3.02765</code></pre>
-<div class="sourceCode" id="cb1257"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1257-1"><a href="collinearity.html#cb1257-1" aria-hidden="true" tabindex="-1"></a><span class="fu">cor</span>(x1, x2)</span></code></pre></div>
+<div class="sourceCode" id="cb1257"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1257-1"><a href="collinearity.html#cb1257-1" tabindex="-1"></a><span class="fu">cor</span>(x1, x2)</span></code></pre></div>
 <pre><code>## [1] 0.9393939</code></pre>
 <p>Notice that they have extremely high correlation.</p>
-<div class="sourceCode" id="cb1259"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1259-1"><a href="collinearity.html#cb1259-1" aria-hidden="true" tabindex="-1"></a>true_line_bad <span class="ot">=</span> beta_0 <span class="sc">+</span> beta_1 <span class="sc">*</span> x1 <span class="sc">+</span> beta_2 <span class="sc">*</span> x2</span>
-<span id="cb1259-2"><a href="collinearity.html#cb1259-2" aria-hidden="true" tabindex="-1"></a>beta_hat_bad  <span class="ot">=</span> <span class="fu">matrix</span>(<span class="dv">0</span>, num_sim, <span class="dv">2</span>)</span>
-<span id="cb1259-3"><a href="collinearity.html#cb1259-3" aria-hidden="true" tabindex="-1"></a>mse_bad       <span class="ot">=</span> <span class="fu">rep</span>(<span class="dv">0</span>, num_sim)</span></code></pre></div>
+<div class="sourceCode" id="cb1259"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1259-1"><a href="collinearity.html#cb1259-1" tabindex="-1"></a>true_line_bad <span class="ot">=</span> beta_0 <span class="sc">+</span> beta_1 <span class="sc">*</span> x1 <span class="sc">+</span> beta_2 <span class="sc">*</span> x2</span>
+<span id="cb1259-2"><a href="collinearity.html#cb1259-2" tabindex="-1"></a>beta_hat_bad  <span class="ot">=</span> <span class="fu">matrix</span>(<span class="dv">0</span>, num_sim, <span class="dv">2</span>)</span>
+<span id="cb1259-3"><a href="collinearity.html#cb1259-3" tabindex="-1"></a>mse_bad       <span class="ot">=</span> <span class="fu">rep</span>(<span class="dv">0</span>, num_sim)</span></code></pre></div>
 <p>We perform the simulation 2500 times, each time fitting a regression model, and storing the estimated coefficients and the MSE.</p>
-<div class="sourceCode" id="cb1260"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1260-1"><a href="collinearity.html#cb1260-1" aria-hidden="true" tabindex="-1"></a><span class="cf">for</span> (s <span class="cf">in</span> <span class="dv">1</span><span class="sc">:</span>num_sim) {</span>
-<span id="cb1260-2"><a href="collinearity.html#cb1260-2" aria-hidden="true" tabindex="-1"></a>  y <span class="ot">=</span> true_line_bad <span class="sc">+</span> <span class="fu">rnorm</span>(<span class="at">n =</span> sample_size, <span class="at">mean =</span> <span class="dv">0</span>, <span class="at">sd =</span> sigma)</span>
-<span id="cb1260-3"><a href="collinearity.html#cb1260-3" aria-hidden="true" tabindex="-1"></a>  reg_out <span class="ot">=</span> <span class="fu">lm</span>(y <span class="sc">~</span> x1 <span class="sc">+</span> x2)</span>
-<span id="cb1260-4"><a href="collinearity.html#cb1260-4" aria-hidden="true" tabindex="-1"></a>  beta_hat_bad[s, ] <span class="ot">=</span> <span class="fu">coef</span>(reg_out)[<span class="sc">-</span><span class="dv">1</span>]</span>
-<span id="cb1260-5"><a href="collinearity.html#cb1260-5" aria-hidden="true" tabindex="-1"></a>  mse_bad[s] <span class="ot">=</span> <span class="fu">mean</span>(<span class="fu">resid</span>(reg_out) <span class="sc">^</span> <span class="dv">2</span>)</span>
-<span id="cb1260-6"><a href="collinearity.html#cb1260-6" aria-hidden="true" tabindex="-1"></a>}</span></code></pre></div>
+<div class="sourceCode" id="cb1260"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1260-1"><a href="collinearity.html#cb1260-1" tabindex="-1"></a><span class="cf">for</span> (s <span class="cf">in</span> <span class="dv">1</span><span class="sc">:</span>num_sim) {</span>
+<span id="cb1260-2"><a href="collinearity.html#cb1260-2" tabindex="-1"></a>  y <span class="ot">=</span> true_line_bad <span class="sc">+</span> <span class="fu">rnorm</span>(<span class="at">n =</span> sample_size, <span class="at">mean =</span> <span class="dv">0</span>, <span class="at">sd =</span> sigma)</span>
+<span id="cb1260-3"><a href="collinearity.html#cb1260-3" tabindex="-1"></a>  reg_out <span class="ot">=</span> <span class="fu">lm</span>(y <span class="sc">~</span> x1 <span class="sc">+</span> x2)</span>
+<span id="cb1260-4"><a href="collinearity.html#cb1260-4" tabindex="-1"></a>  beta_hat_bad[s, ] <span class="ot">=</span> <span class="fu">coef</span>(reg_out)[<span class="sc">-</span><span class="dv">1</span>]</span>
+<span id="cb1260-5"><a href="collinearity.html#cb1260-5" tabindex="-1"></a>  mse_bad[s] <span class="ot">=</span> <span class="fu">mean</span>(<span class="fu">resid</span>(reg_out) <span class="sc">^</span> <span class="dv">2</span>)</span>
+<span id="cb1260-6"><a href="collinearity.html#cb1260-6" tabindex="-1"></a>}</span></code></pre></div>
 <p>Now we move to the situation without a collinearity issue, so we again manually create the two predictor variables.</p>
-<div class="sourceCode" id="cb1261"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1261-1"><a href="collinearity.html#cb1261-1" aria-hidden="true" tabindex="-1"></a>z1 <span class="ot">=</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>, <span class="dv">3</span>, <span class="dv">4</span>, <span class="dv">5</span>, <span class="dv">6</span>, <span class="dv">7</span>, <span class="dv">8</span>, <span class="dv">9</span>, <span class="dv">10</span>)</span>
-<span id="cb1261-2"><a href="collinearity.html#cb1261-2" aria-hidden="true" tabindex="-1"></a>z2 <span class="ot">=</span> <span class="fu">c</span>(<span class="dv">9</span>, <span class="dv">2</span>, <span class="dv">7</span>, <span class="dv">4</span>, <span class="dv">5</span>, <span class="dv">6</span>, <span class="dv">3</span>, <span class="dv">8</span>, <span class="dv">1</span>, <span class="dv">10</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1261"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1261-1"><a href="collinearity.html#cb1261-1" tabindex="-1"></a>z1 <span class="ot">=</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>, <span class="dv">3</span>, <span class="dv">4</span>, <span class="dv">5</span>, <span class="dv">6</span>, <span class="dv">7</span>, <span class="dv">8</span>, <span class="dv">9</span>, <span class="dv">10</span>)</span>
+<span id="cb1261-2"><a href="collinearity.html#cb1261-2" tabindex="-1"></a>z2 <span class="ot">=</span> <span class="fu">c</span>(<span class="dv">9</span>, <span class="dv">2</span>, <span class="dv">7</span>, <span class="dv">4</span>, <span class="dv">5</span>, <span class="dv">6</span>, <span class="dv">3</span>, <span class="dv">8</span>, <span class="dv">1</span>, <span class="dv">10</span>)</span></code></pre></div>
 <p>Notice that the standard deviations of each are the same as before, however, now the correlation is extremely close to 0.</p>
-<div class="sourceCode" id="cb1262"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1262-1"><a href="collinearity.html#cb1262-1" aria-hidden="true" tabindex="-1"></a><span class="fu">c</span>(<span class="fu">sd</span>(z1), <span class="fu">sd</span>(z2))</span></code></pre></div>
+<div class="sourceCode" id="cb1262"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1262-1"><a href="collinearity.html#cb1262-1" tabindex="-1"></a><span class="fu">c</span>(<span class="fu">sd</span>(z1), <span class="fu">sd</span>(z2))</span></code></pre></div>
 <pre><code>## [1] 3.02765 3.02765</code></pre>
-<div class="sourceCode" id="cb1264"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1264-1"><a href="collinearity.html#cb1264-1" aria-hidden="true" tabindex="-1"></a><span class="fu">cor</span>(z1, z2)</span></code></pre></div>
+<div class="sourceCode" id="cb1264"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1264-1"><a href="collinearity.html#cb1264-1" tabindex="-1"></a><span class="fu">cor</span>(z1, z2)</span></code></pre></div>
 <pre><code>## [1] 0.03030303</code></pre>
-<div class="sourceCode" id="cb1266"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1266-1"><a href="collinearity.html#cb1266-1" aria-hidden="true" tabindex="-1"></a>true_line_good <span class="ot">=</span> beta_0 <span class="sc">+</span> beta_1 <span class="sc">*</span> z1 <span class="sc">+</span> beta_2 <span class="sc">*</span> z2</span>
-<span id="cb1266-2"><a href="collinearity.html#cb1266-2" aria-hidden="true" tabindex="-1"></a>beta_hat_good  <span class="ot">=</span> <span class="fu">matrix</span>(<span class="dv">0</span>, num_sim, <span class="dv">2</span>)</span>
-<span id="cb1266-3"><a href="collinearity.html#cb1266-3" aria-hidden="true" tabindex="-1"></a>mse_good       <span class="ot">=</span> <span class="fu">rep</span>(<span class="dv">0</span>, num_sim)</span></code></pre></div>
+<div class="sourceCode" id="cb1266"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1266-1"><a href="collinearity.html#cb1266-1" tabindex="-1"></a>true_line_good <span class="ot">=</span> beta_0 <span class="sc">+</span> beta_1 <span class="sc">*</span> z1 <span class="sc">+</span> beta_2 <span class="sc">*</span> z2</span>
+<span id="cb1266-2"><a href="collinearity.html#cb1266-2" tabindex="-1"></a>beta_hat_good  <span class="ot">=</span> <span class="fu">matrix</span>(<span class="dv">0</span>, num_sim, <span class="dv">2</span>)</span>
+<span id="cb1266-3"><a href="collinearity.html#cb1266-3" tabindex="-1"></a>mse_good       <span class="ot">=</span> <span class="fu">rep</span>(<span class="dv">0</span>, num_sim)</span></code></pre></div>
 <p>We then perform simulations and store the same results.</p>
-<div class="sourceCode" id="cb1267"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1267-1"><a href="collinearity.html#cb1267-1" aria-hidden="true" tabindex="-1"></a><span class="cf">for</span> (s <span class="cf">in</span> <span class="dv">1</span><span class="sc">:</span>num_sim) {</span>
-<span id="cb1267-2"><a href="collinearity.html#cb1267-2" aria-hidden="true" tabindex="-1"></a>  y <span class="ot">=</span> true_line_good <span class="sc">+</span> <span class="fu">rnorm</span>(<span class="at">n =</span> sample_size, <span class="at">mean =</span> <span class="dv">0</span>, <span class="at">sd =</span> sigma)</span>
-<span id="cb1267-3"><a href="collinearity.html#cb1267-3" aria-hidden="true" tabindex="-1"></a>  reg_out <span class="ot">=</span> <span class="fu">lm</span>(y <span class="sc">~</span> z1 <span class="sc">+</span> z2)</span>
-<span id="cb1267-4"><a href="collinearity.html#cb1267-4" aria-hidden="true" tabindex="-1"></a>  beta_hat_good[s, ] <span class="ot">=</span> <span class="fu">coef</span>(reg_out)[<span class="sc">-</span><span class="dv">1</span>]</span>
-<span id="cb1267-5"><a href="collinearity.html#cb1267-5" aria-hidden="true" tabindex="-1"></a>  mse_good[s] <span class="ot">=</span> <span class="fu">mean</span>(<span class="fu">resid</span>(reg_out) <span class="sc">^</span> <span class="dv">2</span>)</span>
-<span id="cb1267-6"><a href="collinearity.html#cb1267-6" aria-hidden="true" tabindex="-1"></a>}</span></code></pre></div>
+<div class="sourceCode" id="cb1267"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1267-1"><a href="collinearity.html#cb1267-1" tabindex="-1"></a><span class="cf">for</span> (s <span class="cf">in</span> <span class="dv">1</span><span class="sc">:</span>num_sim) {</span>
+<span id="cb1267-2"><a href="collinearity.html#cb1267-2" tabindex="-1"></a>  y <span class="ot">=</span> true_line_good <span class="sc">+</span> <span class="fu">rnorm</span>(<span class="at">n =</span> sample_size, <span class="at">mean =</span> <span class="dv">0</span>, <span class="at">sd =</span> sigma)</span>
+<span id="cb1267-3"><a href="collinearity.html#cb1267-3" tabindex="-1"></a>  reg_out <span class="ot">=</span> <span class="fu">lm</span>(y <span class="sc">~</span> z1 <span class="sc">+</span> z2)</span>
+<span id="cb1267-4"><a href="collinearity.html#cb1267-4" tabindex="-1"></a>  beta_hat_good[s, ] <span class="ot">=</span> <span class="fu">coef</span>(reg_out)[<span class="sc">-</span><span class="dv">1</span>]</span>
+<span id="cb1267-5"><a href="collinearity.html#cb1267-5" tabindex="-1"></a>  mse_good[s] <span class="ot">=</span> <span class="fu">mean</span>(<span class="fu">resid</span>(reg_out) <span class="sc">^</span> <span class="dv">2</span>)</span>
+<span id="cb1267-6"><a href="collinearity.html#cb1267-6" tabindex="-1"></a>}</span></code></pre></div>
 <p>We’ll now investigate the differences.</p>
-<div class="sourceCode" id="cb1268"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1268-1"><a href="collinearity.html#cb1268-1" aria-hidden="true" tabindex="-1"></a><span class="fu">par</span>(<span class="at">mfrow =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>))</span>
-<span id="cb1268-2"><a href="collinearity.html#cb1268-2" aria-hidden="true" tabindex="-1"></a><span class="fu">hist</span>(beta_hat_bad[, <span class="dv">1</span>],</span>
-<span id="cb1268-3"><a href="collinearity.html#cb1268-3" aria-hidden="true" tabindex="-1"></a>     <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>,</span>
-<span id="cb1268-4"><a href="collinearity.html#cb1268-4" aria-hidden="true" tabindex="-1"></a>     <span class="at">border =</span> <span class="st">&quot;dodgerblue&quot;</span>,</span>
-<span id="cb1268-5"><a href="collinearity.html#cb1268-5" aria-hidden="true" tabindex="-1"></a>     <span class="at">main =</span> <span class="fu">expression</span>(<span class="st">&quot;Histogram of &quot;</span> <span class="sc">*</span><span class="fu">hat</span>(beta)[<span class="dv">1</span>]<span class="sc">*</span> <span class="st">&quot; with Collinearity&quot;</span>),</span>
-<span id="cb1268-6"><a href="collinearity.html#cb1268-6" aria-hidden="true" tabindex="-1"></a>     <span class="at">xlab =</span> <span class="fu">expression</span>(<span class="fu">hat</span>(beta)[<span class="dv">1</span>]),</span>
-<span id="cb1268-7"><a href="collinearity.html#cb1268-7" aria-hidden="true" tabindex="-1"></a>     <span class="at">breaks =</span> <span class="dv">20</span>)</span>
-<span id="cb1268-8"><a href="collinearity.html#cb1268-8" aria-hidden="true" tabindex="-1"></a><span class="fu">hist</span>(beta_hat_good[, <span class="dv">1</span>],</span>
-<span id="cb1268-9"><a href="collinearity.html#cb1268-9" aria-hidden="true" tabindex="-1"></a>     <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>,</span>
-<span id="cb1268-10"><a href="collinearity.html#cb1268-10" aria-hidden="true" tabindex="-1"></a>     <span class="at">border =</span> <span class="st">&quot;dodgerblue&quot;</span>,</span>
-<span id="cb1268-11"><a href="collinearity.html#cb1268-11" aria-hidden="true" tabindex="-1"></a>     <span class="at">main =</span> <span class="fu">expression</span>(<span class="st">&quot;Histogram of &quot;</span> <span class="sc">*</span><span class="fu">hat</span>(beta)[<span class="dv">1</span>]<span class="sc">*</span> <span class="st">&quot; without Collinearity&quot;</span>),</span>
-<span id="cb1268-12"><a href="collinearity.html#cb1268-12" aria-hidden="true" tabindex="-1"></a>     <span class="at">xlab =</span> <span class="fu">expression</span>(<span class="fu">hat</span>(beta)[<span class="dv">1</span>]),</span>
-<span id="cb1268-13"><a href="collinearity.html#cb1268-13" aria-hidden="true" tabindex="-1"></a>     <span class="at">breaks =</span> <span class="dv">20</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1268"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1268-1"><a href="collinearity.html#cb1268-1" tabindex="-1"></a><span class="fu">par</span>(<span class="at">mfrow =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>))</span>
+<span id="cb1268-2"><a href="collinearity.html#cb1268-2" tabindex="-1"></a><span class="fu">hist</span>(beta_hat_bad[, <span class="dv">1</span>],</span>
+<span id="cb1268-3"><a href="collinearity.html#cb1268-3" tabindex="-1"></a>     <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>,</span>
+<span id="cb1268-4"><a href="collinearity.html#cb1268-4" tabindex="-1"></a>     <span class="at">border =</span> <span class="st">&quot;dodgerblue&quot;</span>,</span>
+<span id="cb1268-5"><a href="collinearity.html#cb1268-5" tabindex="-1"></a>     <span class="at">main =</span> <span class="fu">expression</span>(<span class="st">&quot;Histogram of &quot;</span> <span class="sc">*</span><span class="fu">hat</span>(beta)[<span class="dv">1</span>]<span class="sc">*</span> <span class="st">&quot; with Collinearity&quot;</span>),</span>
+<span id="cb1268-6"><a href="collinearity.html#cb1268-6" tabindex="-1"></a>     <span class="at">xlab =</span> <span class="fu">expression</span>(<span class="fu">hat</span>(beta)[<span class="dv">1</span>]),</span>
+<span id="cb1268-7"><a href="collinearity.html#cb1268-7" tabindex="-1"></a>     <span class="at">breaks =</span> <span class="dv">20</span>)</span>
+<span id="cb1268-8"><a href="collinearity.html#cb1268-8" tabindex="-1"></a><span class="fu">hist</span>(beta_hat_good[, <span class="dv">1</span>],</span>
+<span id="cb1268-9"><a href="collinearity.html#cb1268-9" tabindex="-1"></a>     <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>,</span>
+<span id="cb1268-10"><a href="collinearity.html#cb1268-10" tabindex="-1"></a>     <span class="at">border =</span> <span class="st">&quot;dodgerblue&quot;</span>,</span>
+<span id="cb1268-11"><a href="collinearity.html#cb1268-11" tabindex="-1"></a>     <span class="at">main =</span> <span class="fu">expression</span>(<span class="st">&quot;Histogram of &quot;</span> <span class="sc">*</span><span class="fu">hat</span>(beta)[<span class="dv">1</span>]<span class="sc">*</span> <span class="st">&quot; without Collinearity&quot;</span>),</span>
+<span id="cb1268-12"><a href="collinearity.html#cb1268-12" tabindex="-1"></a>     <span class="at">xlab =</span> <span class="fu">expression</span>(<span class="fu">hat</span>(beta)[<span class="dv">1</span>]),</span>
+<span id="cb1268-13"><a href="collinearity.html#cb1268-13" tabindex="-1"></a>     <span class="at">breaks =</span> <span class="dv">20</span>)</span></code></pre></div>
 <p><img src="collinearity_files/figure-html/unnamed-chunk-31-1.png" width="1152" style="display: block; margin: auto;" /></p>
 <p>First, for <span class="math inline">\(\beta_1\)</span>, which has a true value of <span class="math inline">\(3\)</span>, we see that both with and without collinearity, the simulated values are centered near <span class="math inline">\(3\)</span>.</p>
-<div class="sourceCode" id="cb1269"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1269-1"><a href="collinearity.html#cb1269-1" aria-hidden="true" tabindex="-1"></a><span class="fu">mean</span>(beta_hat_bad[, <span class="dv">1</span>])</span></code></pre></div>
+<div class="sourceCode" id="cb1269"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1269-1"><a href="collinearity.html#cb1269-1" tabindex="-1"></a><span class="fu">mean</span>(beta_hat_bad[, <span class="dv">1</span>])</span></code></pre></div>
 <pre><code>## [1] 2.963325</code></pre>
-<div class="sourceCode" id="cb1271"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1271-1"><a href="collinearity.html#cb1271-1" aria-hidden="true" tabindex="-1"></a><span class="fu">mean</span>(beta_hat_good[, <span class="dv">1</span>])</span></code></pre></div>
+<div class="sourceCode" id="cb1271"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1271-1"><a href="collinearity.html#cb1271-1" tabindex="-1"></a><span class="fu">mean</span>(beta_hat_good[, <span class="dv">1</span>])</span></code></pre></div>
 <pre><code>## [1] 3.013414</code></pre>
 <p>The way the predictors were created, the <span class="math inline">\(S_{x_j x_j}\)</span> portion of the variance is the same for the predictors in both cases, but the variance is still much larger in the simulations performed with collinearity. The variance is so large in the collinear case, that sometimes the estimated coefficient for <span class="math inline">\(\beta_1\)</span> is negative!</p>
-<div class="sourceCode" id="cb1273"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1273-1"><a href="collinearity.html#cb1273-1" aria-hidden="true" tabindex="-1"></a><span class="fu">sd</span>(beta_hat_bad[, <span class="dv">1</span>])</span></code></pre></div>
+<div class="sourceCode" id="cb1273"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1273-1"><a href="collinearity.html#cb1273-1" tabindex="-1"></a><span class="fu">sd</span>(beta_hat_bad[, <span class="dv">1</span>])</span></code></pre></div>
 <pre><code>## [1] 1.633294</code></pre>
-<div class="sourceCode" id="cb1275"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1275-1"><a href="collinearity.html#cb1275-1" aria-hidden="true" tabindex="-1"></a><span class="fu">sd</span>(beta_hat_good[, <span class="dv">1</span>])</span></code></pre></div>
+<div class="sourceCode" id="cb1275"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1275-1"><a href="collinearity.html#cb1275-1" tabindex="-1"></a><span class="fu">sd</span>(beta_hat_good[, <span class="dv">1</span>])</span></code></pre></div>
 <pre><code>## [1] 0.5484684</code></pre>
-<div class="sourceCode" id="cb1277"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1277-1"><a href="collinearity.html#cb1277-1" aria-hidden="true" tabindex="-1"></a><span class="fu">par</span>(<span class="at">mfrow =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>))</span>
-<span id="cb1277-2"><a href="collinearity.html#cb1277-2" aria-hidden="true" tabindex="-1"></a><span class="fu">hist</span>(beta_hat_bad[, <span class="dv">2</span>],</span>
-<span id="cb1277-3"><a href="collinearity.html#cb1277-3" aria-hidden="true" tabindex="-1"></a>     <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>,</span>
-<span id="cb1277-4"><a href="collinearity.html#cb1277-4" aria-hidden="true" tabindex="-1"></a>     <span class="at">border =</span> <span class="st">&quot;dodgerblue&quot;</span>,</span>
-<span id="cb1277-5"><a href="collinearity.html#cb1277-5" aria-hidden="true" tabindex="-1"></a>     <span class="at">main =</span> <span class="fu">expression</span>(<span class="st">&quot;Histogram of &quot;</span> <span class="sc">*</span><span class="fu">hat</span>(beta)[<span class="dv">2</span>]<span class="sc">*</span> <span class="st">&quot; with Collinearity&quot;</span>),</span>
-<span id="cb1277-6"><a href="collinearity.html#cb1277-6" aria-hidden="true" tabindex="-1"></a>     <span class="at">xlab =</span> <span class="fu">expression</span>(<span class="fu">hat</span>(beta)[<span class="dv">2</span>]),</span>
-<span id="cb1277-7"><a href="collinearity.html#cb1277-7" aria-hidden="true" tabindex="-1"></a>     <span class="at">breaks =</span> <span class="dv">20</span>)</span>
-<span id="cb1277-8"><a href="collinearity.html#cb1277-8" aria-hidden="true" tabindex="-1"></a><span class="fu">hist</span>(beta_hat_good[, <span class="dv">2</span>],</span>
-<span id="cb1277-9"><a href="collinearity.html#cb1277-9" aria-hidden="true" tabindex="-1"></a>     <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>,</span>
-<span id="cb1277-10"><a href="collinearity.html#cb1277-10" aria-hidden="true" tabindex="-1"></a>     <span class="at">border =</span> <span class="st">&quot;dodgerblue&quot;</span>,</span>
-<span id="cb1277-11"><a href="collinearity.html#cb1277-11" aria-hidden="true" tabindex="-1"></a>     <span class="at">main =</span> <span class="fu">expression</span>(<span class="st">&quot;Histogram of &quot;</span> <span class="sc">*</span><span class="fu">hat</span>(beta)[<span class="dv">2</span>]<span class="sc">*</span> <span class="st">&quot; without Collinearity&quot;</span>),</span>
-<span id="cb1277-12"><a href="collinearity.html#cb1277-12" aria-hidden="true" tabindex="-1"></a>     <span class="at">xlab =</span> <span class="fu">expression</span>(<span class="fu">hat</span>(beta)[<span class="dv">2</span>]),</span>
-<span id="cb1277-13"><a href="collinearity.html#cb1277-13" aria-hidden="true" tabindex="-1"></a>     <span class="at">breaks =</span> <span class="dv">20</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1277"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1277-1"><a href="collinearity.html#cb1277-1" tabindex="-1"></a><span class="fu">par</span>(<span class="at">mfrow =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>))</span>
+<span id="cb1277-2"><a href="collinearity.html#cb1277-2" tabindex="-1"></a><span class="fu">hist</span>(beta_hat_bad[, <span class="dv">2</span>],</span>
+<span id="cb1277-3"><a href="collinearity.html#cb1277-3" tabindex="-1"></a>     <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>,</span>
+<span id="cb1277-4"><a href="collinearity.html#cb1277-4" tabindex="-1"></a>     <span class="at">border =</span> <span class="st">&quot;dodgerblue&quot;</span>,</span>
+<span id="cb1277-5"><a href="collinearity.html#cb1277-5" tabindex="-1"></a>     <span class="at">main =</span> <span class="fu">expression</span>(<span class="st">&quot;Histogram of &quot;</span> <span class="sc">*</span><span class="fu">hat</span>(beta)[<span class="dv">2</span>]<span class="sc">*</span> <span class="st">&quot; with Collinearity&quot;</span>),</span>
+<span id="cb1277-6"><a href="collinearity.html#cb1277-6" tabindex="-1"></a>     <span class="at">xlab =</span> <span class="fu">expression</span>(<span class="fu">hat</span>(beta)[<span class="dv">2</span>]),</span>
+<span id="cb1277-7"><a href="collinearity.html#cb1277-7" tabindex="-1"></a>     <span class="at">breaks =</span> <span class="dv">20</span>)</span>
+<span id="cb1277-8"><a href="collinearity.html#cb1277-8" tabindex="-1"></a><span class="fu">hist</span>(beta_hat_good[, <span class="dv">2</span>],</span>
+<span id="cb1277-9"><a href="collinearity.html#cb1277-9" tabindex="-1"></a>     <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>,</span>
+<span id="cb1277-10"><a href="collinearity.html#cb1277-10" tabindex="-1"></a>     <span class="at">border =</span> <span class="st">&quot;dodgerblue&quot;</span>,</span>
+<span id="cb1277-11"><a href="collinearity.html#cb1277-11" tabindex="-1"></a>     <span class="at">main =</span> <span class="fu">expression</span>(<span class="st">&quot;Histogram of &quot;</span> <span class="sc">*</span><span class="fu">hat</span>(beta)[<span class="dv">2</span>]<span class="sc">*</span> <span class="st">&quot; without Collinearity&quot;</span>),</span>
+<span id="cb1277-12"><a href="collinearity.html#cb1277-12" tabindex="-1"></a>     <span class="at">xlab =</span> <span class="fu">expression</span>(<span class="fu">hat</span>(beta)[<span class="dv">2</span>]),</span>
+<span id="cb1277-13"><a href="collinearity.html#cb1277-13" tabindex="-1"></a>     <span class="at">breaks =</span> <span class="dv">20</span>)</span></code></pre></div>
 <p><img src="collinearity_files/figure-html/unnamed-chunk-34-1.png" width="1152" style="display: block; margin: auto;" /></p>
 <p>We see the same issues with <span class="math inline">\(\beta_2\)</span>. On average the estimates are correct, but the variance is again much larger with collinearity.</p>
-<div class="sourceCode" id="cb1278"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1278-1"><a href="collinearity.html#cb1278-1" aria-hidden="true" tabindex="-1"></a><span class="fu">mean</span>(beta_hat_bad[, <span class="dv">2</span>])</span></code></pre></div>
+<div class="sourceCode" id="cb1278"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1278-1"><a href="collinearity.html#cb1278-1" tabindex="-1"></a><span class="fu">mean</span>(beta_hat_bad[, <span class="dv">2</span>])</span></code></pre></div>
 <pre><code>## [1] 4.025059</code></pre>
-<div class="sourceCode" id="cb1280"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1280-1"><a href="collinearity.html#cb1280-1" aria-hidden="true" tabindex="-1"></a><span class="fu">mean</span>(beta_hat_good[, <span class="dv">2</span>])</span></code></pre></div>
+<div class="sourceCode" id="cb1280"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1280-1"><a href="collinearity.html#cb1280-1" tabindex="-1"></a><span class="fu">mean</span>(beta_hat_good[, <span class="dv">2</span>])</span></code></pre></div>
 <pre><code>## [1] 4.004913</code></pre>
-<div class="sourceCode" id="cb1282"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1282-1"><a href="collinearity.html#cb1282-1" aria-hidden="true" tabindex="-1"></a><span class="fu">sd</span>(beta_hat_bad[, <span class="dv">2</span>])</span></code></pre></div>
+<div class="sourceCode" id="cb1282"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1282-1"><a href="collinearity.html#cb1282-1" tabindex="-1"></a><span class="fu">sd</span>(beta_hat_bad[, <span class="dv">2</span>])</span></code></pre></div>
 <pre><code>## [1] 1.642592</code></pre>
-<div class="sourceCode" id="cb1284"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1284-1"><a href="collinearity.html#cb1284-1" aria-hidden="true" tabindex="-1"></a><span class="fu">sd</span>(beta_hat_good[, <span class="dv">2</span>])</span></code></pre></div>
+<div class="sourceCode" id="cb1284"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1284-1"><a href="collinearity.html#cb1284-1" tabindex="-1"></a><span class="fu">sd</span>(beta_hat_good[, <span class="dv">2</span>])</span></code></pre></div>
 <pre><code>## [1] 0.5470381</code></pre>
-<div class="sourceCode" id="cb1286"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1286-1"><a href="collinearity.html#cb1286-1" aria-hidden="true" tabindex="-1"></a><span class="fu">par</span>(<span class="at">mfrow =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>))</span>
-<span id="cb1286-2"><a href="collinearity.html#cb1286-2" aria-hidden="true" tabindex="-1"></a><span class="fu">hist</span>(mse_bad,</span>
-<span id="cb1286-3"><a href="collinearity.html#cb1286-3" aria-hidden="true" tabindex="-1"></a>     <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>,</span>
-<span id="cb1286-4"><a href="collinearity.html#cb1286-4" aria-hidden="true" tabindex="-1"></a>     <span class="at">border =</span> <span class="st">&quot;dodgerblue&quot;</span>,</span>
-<span id="cb1286-5"><a href="collinearity.html#cb1286-5" aria-hidden="true" tabindex="-1"></a>     <span class="at">main =</span> <span class="st">&quot;MSE, with Collinearity&quot;</span>,</span>
-<span id="cb1286-6"><a href="collinearity.html#cb1286-6" aria-hidden="true" tabindex="-1"></a>     <span class="at">xlab =</span> <span class="st">&quot;MSE&quot;</span>)</span>
-<span id="cb1286-7"><a href="collinearity.html#cb1286-7" aria-hidden="true" tabindex="-1"></a><span class="fu">hist</span>(mse_good,</span>
-<span id="cb1286-8"><a href="collinearity.html#cb1286-8" aria-hidden="true" tabindex="-1"></a>     <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>,</span>
-<span id="cb1286-9"><a href="collinearity.html#cb1286-9" aria-hidden="true" tabindex="-1"></a>     <span class="at">border =</span> <span class="st">&quot;dodgerblue&quot;</span>,</span>
-<span id="cb1286-10"><a href="collinearity.html#cb1286-10" aria-hidden="true" tabindex="-1"></a>     <span class="at">main =</span> <span class="st">&quot;MSE, without Collinearity&quot;</span>,</span>
-<span id="cb1286-11"><a href="collinearity.html#cb1286-11" aria-hidden="true" tabindex="-1"></a>     <span class="at">xlab =</span> <span class="st">&quot;MSE&quot;</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1286"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1286-1"><a href="collinearity.html#cb1286-1" tabindex="-1"></a><span class="fu">par</span>(<span class="at">mfrow =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>))</span>
+<span id="cb1286-2"><a href="collinearity.html#cb1286-2" tabindex="-1"></a><span class="fu">hist</span>(mse_bad,</span>
+<span id="cb1286-3"><a href="collinearity.html#cb1286-3" tabindex="-1"></a>     <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>,</span>
+<span id="cb1286-4"><a href="collinearity.html#cb1286-4" tabindex="-1"></a>     <span class="at">border =</span> <span class="st">&quot;dodgerblue&quot;</span>,</span>
+<span id="cb1286-5"><a href="collinearity.html#cb1286-5" tabindex="-1"></a>     <span class="at">main =</span> <span class="st">&quot;MSE, with Collinearity&quot;</span>,</span>
+<span id="cb1286-6"><a href="collinearity.html#cb1286-6" tabindex="-1"></a>     <span class="at">xlab =</span> <span class="st">&quot;MSE&quot;</span>)</span>
+<span id="cb1286-7"><a href="collinearity.html#cb1286-7" tabindex="-1"></a><span class="fu">hist</span>(mse_good,</span>
+<span id="cb1286-8"><a href="collinearity.html#cb1286-8" tabindex="-1"></a>     <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>,</span>
+<span id="cb1286-9"><a href="collinearity.html#cb1286-9" tabindex="-1"></a>     <span class="at">border =</span> <span class="st">&quot;dodgerblue&quot;</span>,</span>
+<span id="cb1286-10"><a href="collinearity.html#cb1286-10" tabindex="-1"></a>     <span class="at">main =</span> <span class="st">&quot;MSE, without Collinearity&quot;</span>,</span>
+<span id="cb1286-11"><a href="collinearity.html#cb1286-11" tabindex="-1"></a>     <span class="at">xlab =</span> <span class="st">&quot;MSE&quot;</span>)</span></code></pre></div>
 <p><img src="collinearity_files/figure-html/unnamed-chunk-37-1.png" width="1152" style="display: block; margin: auto;" /></p>
 <p>Interestingly, in both cases, the MSE is roughly the same on average. Again, this is because collinearity affects a model’s ability to <em>explain</em>, but not predict.</p>
-<div class="sourceCode" id="cb1287"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1287-1"><a href="collinearity.html#cb1287-1" aria-hidden="true" tabindex="-1"></a><span class="fu">mean</span>(mse_bad)</span></code></pre></div>
+<div class="sourceCode" id="cb1287"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1287-1"><a href="collinearity.html#cb1287-1" tabindex="-1"></a><span class="fu">mean</span>(mse_bad)</span></code></pre></div>
 <pre><code>## [1] 17.7186</code></pre>
-<div class="sourceCode" id="cb1289"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1289-1"><a href="collinearity.html#cb1289-1" aria-hidden="true" tabindex="-1"></a><span class="fu">mean</span>(mse_good)</span></code></pre></div>
+<div class="sourceCode" id="cb1289"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1289-1"><a href="collinearity.html#cb1289-1" tabindex="-1"></a><span class="fu">mean</span>(mse_good)</span></code></pre></div>
 <pre><code>## [1] 17.70513</code></pre>
 </div>
 <div id="r-markdown-8" class="section level2 hasAnchor" number="15.4">
@@ -834,7 +834,7 @@ <h2><span class="header-section-number">15.4</span> <code>R</code> Markdown<a hr
 <ul>
 <li><a href="collinearity.Rmd" target="_blank"><code>collinearity.Rmd</code></a></li>
 </ul>
-<p>The file was created using <code>R</code> version <code>4.3.2</code>.</p>
+<p>The file was created using <code>R</code> version <code>4.4.1</code>.</p>
 
 </div>
 </div>
diff --git a/collinearity.md b/collinearity.md
index db1bd39..2d618bb 100644
--- a/collinearity.md
+++ b/collinearity.md
@@ -16,7 +16,7 @@ After reading this chapter you will be able to:
 Let's create a dataset where one of the predictors, $x_3$, is a linear combination of the other predictors.
 
 
-```r
+``` r
 gen_exact_collin_data = function(num_samples = 100) {
   x1 = rnorm(n = num_samples, mean = 80, sd = 10)
   x2 = rnorm(n = num_samples, mean = 70, sd = 5)
@@ -29,7 +29,7 @@ gen_exact_collin_data = function(num_samples = 100) {
 Notice that the way we are generating this data, the response $y$ only really depends on $x_1$ and $x_2$.
 
 
-```r
+``` r
 set.seed(42)
 exact_collin_data = gen_exact_collin_data()
 head(exact_collin_data)
@@ -48,7 +48,7 @@ head(exact_collin_data)
 What happens when we attempt to fit a regression model in `R` using all of the predictors?
 
 
-```r
+``` r
 exact_collin_fit = lm(y ~ x1 + x2 + x3, data = exact_collin_data)
 summary(exact_collin_fit)
 ```
@@ -79,7 +79,7 @@ summary(exact_collin_fit)
 We see that `R` simply decides to exclude a variable. Why is this happening?
 
 
-```r
+``` r
 X = cbind(1, as.matrix(exact_collin_data[,-1]))
 solve(t(X) %*% X)
 ```
@@ -91,7 +91,7 @@ When this happens, we say there is **exact collinearity** in the dataset.
 As a result of this issue, `R` essentially chose to fit the model `y ~ x1 + x2`. However notice that two other models would accomplish exactly the same fit.
 
 
-```r
+``` r
 fit1 = lm(y ~ x1 + x2, data = exact_collin_data)
 fit2 = lm(y ~ x1 + x3, data = exact_collin_data)
 fit3 = lm(y ~ x2 + x3, data = exact_collin_data)
@@ -100,7 +100,7 @@ fit3 = lm(y ~ x2 + x3, data = exact_collin_data)
 We see that the fitted values for each of the three models are exactly the same. This is a result of $x_3$ containing all of the information from $x_1$ and $x_2$. As long as one of $x_1$ or $x_2$ is included in the model, $x_3$ can be used to recover the information from the variable not included.
 
 
-```r
+``` r
 all.equal(fitted(fit1), fitted(fit2))
 ```
 
@@ -108,7 +108,7 @@ all.equal(fitted(fit1), fitted(fit2))
 ## [1] TRUE
 ```
 
-```r
+``` r
 all.equal(fitted(fit2), fitted(fit3))
 ```
 
@@ -119,7 +119,7 @@ all.equal(fitted(fit2), fitted(fit3))
 While their fitted values are all the same, their estimated coefficients are wildly different. The sign of $x_2$ is switched in two of the models! So only `fit1` properly *explains* the relationship between the variables, `fit2` and `fit3` still *predict* as well as `fit1`, despite the coefficients having little to no meaning, a concept we will return to later.
 
 
-```r
+``` r
 coef(fit1)
 ```
 
@@ -128,7 +128,7 @@ coef(fit1)
 ##   2.9573357   0.9856291   1.0170586
 ```
 
-```r
+``` r
 coef(fit2)
 ```
 
@@ -137,7 +137,7 @@ coef(fit2)
 ##   2.1945418   0.4770998   0.2542647
 ```
 
-```r
+``` r
 coef(fit3)
 ```
 
@@ -155,7 +155,7 @@ Looking at the `seatpos` dataset from the `faraway` package, we will see an exam
 We will attempt to fit a model that predicts `hipcenter`. Two predictor variables are immediately interesting to us: `HtShoes` and `Ht`. We certainly expect a person's height to be highly correlated to their height when wearing shoes. We'll pay special attention to these two variables when fitting models.
 
 
-```r
+``` r
 library(faraway)
 pairs(seatpos, col = "dodgerblue")
 ```
@@ -164,7 +164,7 @@ pairs(seatpos, col = "dodgerblue")
 
 \begin{center}\includegraphics{collinearity_files/figure-latex/unnamed-chunk-9-1} \end{center}
 
-```r
+``` r
 round(cor(seatpos), 2)
 ```
 
@@ -188,7 +188,7 @@ We can also do this numerically with the `cor()` function, which when applied to
 Unlike exact collinearity, here we can still fit a model with all of the predictors, but what effect does this have?
 
 
-```r
+``` r
 hip_model = lm(hipcenter ~ ., data = seatpos)
 summary(hip_model)
 ```
@@ -228,7 +228,7 @@ This happens as a result of the predictors being highly correlated. For example,
 We define $R_j^2$ to be the proportion of observed variation in the $j$-th predictor explained by the other predictors. In other words $R_j^2$ is the multiple R-Squared for the regression of $x_j$ on each of the other predictors.
 
 
-```r
+``` r
 ht_shoes_model = lm(HtShoes ~ . - hipcenter, data = seatpos)
 summary(ht_shoes_model)$r.squared
 ```
@@ -266,7 +266,7 @@ the **variance inflation factor.** The variance inflation factor quantifies the
 The `vif` function from the `faraway` package calculates the VIFs for each of the predictors of a model.
 
 
-```r
+``` r
 vif(hip_model)
 ```
 
@@ -282,7 +282,7 @@ In practice it is common to say that any VIF greater than $5$ is cause for conce
 Let's further investigate how the presence of collinearity actually affects a model. If we add a moderate amount of noise to the data, we see that the estimates of the coefficients change drastically. This is a rather undesirable effect. Adding random noise should not affect the coefficients of a model.
 
 
-```r
+``` r
 set.seed(1337)
 noise = rnorm(n = nrow(seatpos), mean = 0, sd = 5)
 hip_model_noise = lm(hipcenter + noise ~ ., data = seatpos)
@@ -291,7 +291,7 @@ hip_model_noise = lm(hipcenter + noise ~ ., data = seatpos)
 Adding the noise had such a large effect, the sign of the coefficient for `Ht` has changed.
 
 
-```r
+``` r
 coef(hip_model)
 ```
 
@@ -302,7 +302,7 @@ coef(hip_model)
 ##  -1.32806864  -1.14311888  -6.43904627
 ```
 
-```r
+``` r
 coef(hip_model_noise)
 ```
 
@@ -316,7 +316,7 @@ coef(hip_model_noise)
 This tells us that a model with collinearity is bad at explaining the relationship between the response and the predictors. We cannot even be confident in the direction of the relationship. However, does collinearity affect prediction?
 
 
-```r
+``` r
 plot(fitted(hip_model), fitted(hip_model_noise), col = "dodgerblue", pch = 20,
      xlab = "Predicted, Without Noise", ylab = "Predicted, With Noise", cex = 1.5)
 abline(a = 0, b = 1, col = "darkorange", lwd = 2)
@@ -331,7 +331,7 @@ We see that by plotting the predicted values using both models against each othe
 Let's now look at a smaller model,
 
 
-```r
+``` r
 hip_model_small = lm(hipcenter ~ Age + Arm + Ht, data = seatpos)
 summary(hip_model_small)
 ```
@@ -359,7 +359,7 @@ summary(hip_model_small)
 ## F-statistic:  22.3 on 3 and 34 DF,  p-value: 3.649e-08
 ```
 
-```r
+``` r
 vif(hip_model_small)
 ```
 
@@ -371,7 +371,7 @@ vif(hip_model_small)
 Immediately we see that multicollinearity isn't an issue here.
 
 
-```r
+``` r
 anova(hip_model_small, hip_model)
 ```
 
@@ -396,7 +396,7 @@ To quantify this effect we will look at a **variable added plot** and a **partia
 - Regressing the predictor of interest (`HtShoes`) against the other predictors (`Age`, `Arm`, and `Ht`).
 
 
-```r
+``` r
 ht_shoes_model_small = lm(HtShoes ~ Age + Arm + Ht, data = seatpos)
 ```
 
@@ -405,7 +405,7 @@ So now, the residuals of `hip_model_small` give us the variation of `hipcenter`
 The correlation of these two residuals gives us the **partial correlation coefficient** of `HtShoes` and `hipcenter` with the effects of `Age`, `Arm`, and `Ht` removed.
 
 
-```r
+``` r
 cor(resid(ht_shoes_model_small), resid(hip_model_small))
 ```
 
@@ -418,7 +418,7 @@ Since this value is small, close to zero, it means that the variation of `hipcen
 Similarly a **variable added plot** visualizes these residuals against each other. It is also helpful to regress the residuals of the response against the residuals of the predictor and add the regression line to the plot.
 
 
-```r
+``` r
 plot(resid(hip_model_small) ~ resid(ht_shoes_model_small), 
      col = "dodgerblue", pch = 20,
      xlab = "Residuals, Added Predictor", 
@@ -455,7 +455,7 @@ Y = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \epsilon
 where $\epsilon \sim N(\mu = 0, \sigma^2 = 25)$ and the $\beta$ coefficients defined below.
 
 
-```r
+``` r
 set.seed(42)
 beta_0 = 7
 beta_1 = 3
@@ -466,7 +466,7 @@ sigma  = 5
 We will use a sample size of 10, and 2500 simulations for both situations.
 
 
-```r
+``` r
 sample_size = 10
 num_sim     = 2500
 ```
@@ -474,13 +474,13 @@ num_sim     = 2500
 We'll first consider the situation with a collinearity issue, so we manually create the two predictor variables.
 
 
-```r
+``` r
 x1 = c(1, 2, 3, 4, 5, 6, 7, 8, 9, 10)
 x2 = c(1, 2, 3, 4, 5, 7, 6, 10, 9, 8)
 ```
 
 
-```r
+``` r
 c(sd(x1), sd(x2))
 ```
 
@@ -488,7 +488,7 @@ c(sd(x1), sd(x2))
 ## [1] 3.02765 3.02765
 ```
 
-```r
+``` r
 cor(x1, x2)
 ```
 
@@ -499,7 +499,7 @@ cor(x1, x2)
 Notice that they have extremely high correlation.
 
 
-```r
+``` r
 true_line_bad = beta_0 + beta_1 * x1 + beta_2 * x2
 beta_hat_bad  = matrix(0, num_sim, 2)
 mse_bad       = rep(0, num_sim)
@@ -508,7 +508,7 @@ mse_bad       = rep(0, num_sim)
 We perform the simulation 2500 times, each time fitting a regression model, and storing the estimated coefficients and the MSE.
 
 
-```r
+``` r
 for (s in 1:num_sim) {
   y = true_line_bad + rnorm(n = sample_size, mean = 0, sd = sigma)
   reg_out = lm(y ~ x1 + x2)
@@ -520,7 +520,7 @@ for (s in 1:num_sim) {
 Now we move to  the situation without a collinearity issue, so we again manually create the two predictor variables.
 
 
-```r
+``` r
 z1 = c(1, 2, 3, 4, 5, 6, 7, 8, 9, 10)
 z2 = c(9, 2, 7, 4, 5, 6, 3, 8, 1, 10)
 ```
@@ -528,7 +528,7 @@ z2 = c(9, 2, 7, 4, 5, 6, 3, 8, 1, 10)
 Notice that the standard deviations of each are the same as before, however, now the correlation is extremely close to 0.
 
 
-```r
+``` r
 c(sd(z1), sd(z2))
 ```
 
@@ -536,7 +536,7 @@ c(sd(z1), sd(z2))
 ## [1] 3.02765 3.02765
 ```
 
-```r
+``` r
 cor(z1, z2)
 ```
 
@@ -545,7 +545,7 @@ cor(z1, z2)
 ```
 
 
-```r
+``` r
 true_line_good = beta_0 + beta_1 * z1 + beta_2 * z2
 beta_hat_good  = matrix(0, num_sim, 2)
 mse_good       = rep(0, num_sim)
@@ -554,7 +554,7 @@ mse_good       = rep(0, num_sim)
 We then perform simulations and store the same results.
 
 
-```r
+``` r
 for (s in 1:num_sim) {
   y = true_line_good + rnorm(n = sample_size, mean = 0, sd = sigma)
   reg_out = lm(y ~ z1 + z2)
@@ -566,7 +566,7 @@ for (s in 1:num_sim) {
 We'll now investigate the differences.
 
 
-```r
+``` r
 par(mfrow = c(1, 2))
 hist(beta_hat_bad[, 1],
      col = "darkorange",
@@ -589,7 +589,7 @@ hist(beta_hat_good[, 1],
 First, for $\beta_1$, which has a true value of $3$, we see that both with and without collinearity, the simulated values are centered near $3$.
 
 
-```r
+``` r
 mean(beta_hat_bad[, 1])
 ```
 
@@ -597,7 +597,7 @@ mean(beta_hat_bad[, 1])
 ## [1] 2.963325
 ```
 
-```r
+``` r
 mean(beta_hat_good[, 1])
 ```
 
@@ -608,7 +608,7 @@ mean(beta_hat_good[, 1])
 The way the predictors were created, the $S_{x_j x_j}$ portion of the variance is the same for the predictors in both cases, but the variance is still much larger in the simulations performed with collinearity. The variance is so large in the collinear case, that sometimes the estimated coefficient for $\beta_1$ is negative! 
 
 
-```r
+``` r
 sd(beta_hat_bad[, 1])
 ```
 
@@ -616,7 +616,7 @@ sd(beta_hat_bad[, 1])
 ## [1] 1.633294
 ```
 
-```r
+``` r
 sd(beta_hat_good[, 1])
 ```
 
@@ -625,7 +625,7 @@ sd(beta_hat_good[, 1])
 ```
 
 
-```r
+``` r
 par(mfrow = c(1, 2))
 hist(beta_hat_bad[, 2],
      col = "darkorange",
@@ -648,7 +648,7 @@ hist(beta_hat_good[, 2],
 We see the same issues with $\beta_2$. On average the estimates are correct, but the variance is again much larger with collinearity.
 
 
-```r
+``` r
 mean(beta_hat_bad[, 2])
 ```
 
@@ -656,7 +656,7 @@ mean(beta_hat_bad[, 2])
 ## [1] 4.025059
 ```
 
-```r
+``` r
 mean(beta_hat_good[, 2])
 ```
 
@@ -665,7 +665,7 @@ mean(beta_hat_good[, 2])
 ```
 
 
-```r
+``` r
 sd(beta_hat_bad[, 2])
 ```
 
@@ -673,7 +673,7 @@ sd(beta_hat_bad[, 2])
 ## [1] 1.642592
 ```
 
-```r
+``` r
 sd(beta_hat_good[, 2])
 ```
 
@@ -682,7 +682,7 @@ sd(beta_hat_good[, 2])
 ```
 
 
-```r
+``` r
 par(mfrow = c(1, 2))
 hist(mse_bad,
      col = "darkorange",
@@ -703,7 +703,7 @@ hist(mse_good,
 Interestingly, in both cases, the MSE is roughly the same on average. Again, this is because collinearity affects a model's ability to *explain*, but not predict.
 
 
-```r
+``` r
 mean(mse_bad)
 ```
 
@@ -711,7 +711,7 @@ mean(mse_bad)
 ## [1] 17.7186
 ```
 
-```r
+``` r
 mean(mse_good)
 ```
 
@@ -725,4 +725,4 @@ The `R` Markdown file for this chapter can be found here:
 
 - [`collinearity.Rmd`](collinearity.Rmd){target="_blank"}
 
-The file was created using `R` version `4.3.2`.
+The file was created using `R` version `4.4.1`.
diff --git a/data-and-programming.html b/data-and-programming.html
index cee203a..9c26b11 100644
--- a/data-and-programming.html
+++ b/data-and-programming.html
@@ -6,7 +6,7 @@
   <meta http-equiv="X-UA-Compatible" content="IE=edge" />
   <title>Chapter 3 Data and Programming | Applied Statistics with R</title>
   <meta name="description" content="Chapter 3 Data and Programming | Applied Statistics with R" />
-  <meta name="generator" content="bookdown 0.37 and GitBook 2.6.7" />
+  <meta name="generator" content="bookdown 0.40 and GitBook 2.6.7" />
 
   <meta property="og:title" content="Chapter 3 Data and Programming | Applied Statistics with R" />
   <meta property="og:type" content="book" />
@@ -55,7 +55,7 @@
 
 <style type="text/css">
 pre > code.sourceCode { white-space: pre; position: relative; }
-pre > code.sourceCode > span { display: inline-block; line-height: 1.25; }
+pre > code.sourceCode > span { line-height: 1.25; }
 pre > code.sourceCode > span:empty { height: 1.2em; }
 .sourceCode { overflow: visible; }
 code.sourceCode > span { color: inherit; text-decoration: inherit; }
@@ -528,22 +528,22 @@ <h2><span class="header-section-number">3.2</span> Data Structures<a href="data-
 <h3><span class="header-section-number">3.2.1</span> Vectors<a href="data-and-programming.html#vectors" class="anchor-section" aria-label="Anchor link to header"></a></h3>
 <p>Many operations in <code>R</code> make heavy use of <strong>vectors</strong>. Vectors in <code>R</code> are indexed starting at <code>1</code>. That is what the <code>[1]</code> in the output is indicating, that the first element of the row being displayed is the first element of the vector. Larger vectors will start additional rows with <code>[*]</code> where <code>*</code> is the index of the first element of the row.</p>
 <p>Possibly the most common way to create a vector in <code>R</code> is using the <code>c()</code> function, which is short for “combine.” As the name suggests, it combines a list of elements separated by commas.</p>
-<div class="sourceCode" id="cb6"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb6-1"><a href="data-and-programming.html#cb6-1" aria-hidden="true" tabindex="-1"></a><span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">3</span>, <span class="dv">5</span>, <span class="dv">7</span>, <span class="dv">8</span>, <span class="dv">9</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb6"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb6-1"><a href="data-and-programming.html#cb6-1" tabindex="-1"></a><span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">3</span>, <span class="dv">5</span>, <span class="dv">7</span>, <span class="dv">8</span>, <span class="dv">9</span>)</span></code></pre></div>
 <pre><code>## [1] 1 3 5 7 8 9</code></pre>
 <p>Here <code>R</code> simply outputs this vector. If we would like to store this vector in a <strong>variable</strong>, we can do so with the <strong>assignment</strong> operator <code>=</code>. In this case the variable <code>x</code> now holds the vector we just created, and we can access the vector by typing <code>x</code>.</p>
-<div class="sourceCode" id="cb8"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb8-1"><a href="data-and-programming.html#cb8-1" aria-hidden="true" tabindex="-1"></a>x <span class="ot">=</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">3</span>, <span class="dv">5</span>, <span class="dv">7</span>, <span class="dv">8</span>, <span class="dv">9</span>)</span>
-<span id="cb8-2"><a href="data-and-programming.html#cb8-2" aria-hidden="true" tabindex="-1"></a>x</span></code></pre></div>
+<div class="sourceCode" id="cb8"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb8-1"><a href="data-and-programming.html#cb8-1" tabindex="-1"></a>x <span class="ot">=</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">3</span>, <span class="dv">5</span>, <span class="dv">7</span>, <span class="dv">8</span>, <span class="dv">9</span>)</span>
+<span id="cb8-2"><a href="data-and-programming.html#cb8-2" tabindex="-1"></a>x</span></code></pre></div>
 <pre><code>## [1] 1 3 5 7 8 9</code></pre>
 <p>As an aside, there is a long history of the assignment operator in <code>R</code>, partially due to the keys available on the <a href="https://twitter.com/kwbroman/status/747829864091127809" target="_blank">keyboards of the creators of the <code>S</code> language.</a> (Which preceded <code>R</code>.) For simplicity we will use <code>=</code>, but know that often you will see <code>&lt;-</code> as the assignment operator.</p>
 <p>The pros and cons of these two are well beyond the scope of this book, but know that for our purposes you will have no issue if you simply use <code>=</code>. If you are interested in the weird cases where the difference matters, check out <a href="http://www.burns-stat.com/documents/books/the-r-inferno/" target="_blank">The R Inferno</a>.</p>
 <p>If you wish to use <code>&lt;-</code>, you will still need to use <code>=</code>, however only for argument passing. Some users like to keep assignment (<code>&lt;-</code>) and argument passing (<code>=</code>) separate. No matter what you choose, the more important thing is that you <strong>stay consistent</strong>. Also, if working on a larger collaborative project, you should use whatever style is already in place.</p>
 <p>Because vectors must contain elements that are all the same type, <code>R</code> will automatically coerce to a single type when attempting to create a vector that combines multiple types.</p>
-<div class="sourceCode" id="cb10"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb10-1"><a href="data-and-programming.html#cb10-1" aria-hidden="true" tabindex="-1"></a><span class="fu">c</span>(<span class="dv">42</span>, <span class="st">&quot;Statistics&quot;</span>, <span class="cn">TRUE</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb10"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb10-1"><a href="data-and-programming.html#cb10-1" tabindex="-1"></a><span class="fu">c</span>(<span class="dv">42</span>, <span class="st">&quot;Statistics&quot;</span>, <span class="cn">TRUE</span>)</span></code></pre></div>
 <pre><code>## [1] &quot;42&quot;         &quot;Statistics&quot; &quot;TRUE&quot;</code></pre>
-<div class="sourceCode" id="cb12"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb12-1"><a href="data-and-programming.html#cb12-1" aria-hidden="true" tabindex="-1"></a><span class="fu">c</span>(<span class="dv">42</span>, <span class="cn">TRUE</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb12"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb12-1"><a href="data-and-programming.html#cb12-1" tabindex="-1"></a><span class="fu">c</span>(<span class="dv">42</span>, <span class="cn">TRUE</span>)</span></code></pre></div>
 <pre><code>## [1] 42  1</code></pre>
 <p>Frequently you may wish to create a vector based on a sequence of numbers. The quickest and easiest way to do this is with the <code>:</code> operator, which creates a sequence of integers between two specified integers.</p>
-<div class="sourceCode" id="cb14"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb14-1"><a href="data-and-programming.html#cb14-1" aria-hidden="true" tabindex="-1"></a>(<span class="at">y =</span> <span class="dv">1</span><span class="sc">:</span><span class="dv">100</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb14"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb14-1"><a href="data-and-programming.html#cb14-1" tabindex="-1"></a>(<span class="at">y =</span> <span class="dv">1</span><span class="sc">:</span><span class="dv">100</span>)</span></code></pre></div>
 <pre><code>##   [1]   1   2   3   4   5   6   7   8   9  10  11  12  13  14  15  16  17  18
 ##  [19]  19  20  21  22  23  24  25  26  27  28  29  30  31  32  33  34  35  36
 ##  [37]  37  38  39  40  41  42  43  44  45  46  47  48  49  50  51  52  53  54
@@ -552,21 +552,21 @@ <h3><span class="header-section-number">3.2.1</span> Vectors<a href="data-and-pr
 ##  [91]  91  92  93  94  95  96  97  98  99 100</code></pre>
 <p>Here we see <code>R</code> labeling the rows after the first since this is a large vector. Also, we see that by putting parentheses around the assignment, <code>R</code> both stores the vector in a variable called <code>y</code> and automatically outputs <code>y</code> to the console.</p>
 <p>Note that scalars do not exist in <code>R</code>. They are simply vectors of length <code>1</code>.</p>
-<div class="sourceCode" id="cb16"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb16-1"><a href="data-and-programming.html#cb16-1" aria-hidden="true" tabindex="-1"></a><span class="dv">2</span></span></code></pre></div>
+<div class="sourceCode" id="cb16"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb16-1"><a href="data-and-programming.html#cb16-1" tabindex="-1"></a><span class="dv">2</span></span></code></pre></div>
 <pre><code>## [1] 2</code></pre>
 <p>To create a sequence that is not limited to consecutive integers, use the <code>seq()</code> function to define a sequence by its start, end, and increment.</p>
-<div class="sourceCode" id="cb18"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb18-1"><a href="data-and-programming.html#cb18-1" aria-hidden="true" tabindex="-1"></a><span class="fu">seq</span>(<span class="at">from =</span> <span class="fl">1.5</span>, <span class="at">to =</span> <span class="fl">4.2</span>, <span class="at">by =</span> <span class="fl">0.1</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb18"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb18-1"><a href="data-and-programming.html#cb18-1" tabindex="-1"></a><span class="fu">seq</span>(<span class="at">from =</span> <span class="fl">1.5</span>, <span class="at">to =</span> <span class="fl">4.2</span>, <span class="at">by =</span> <span class="fl">0.1</span>)</span></code></pre></div>
 <pre><code>##  [1] 1.5 1.6 1.7 1.8 1.9 2.0 2.1 2.2 2.3 2.4 2.5 2.6 2.7 2.8 2.9 3.0 3.1 3.2 3.3
 ## [20] 3.4 3.5 3.6 3.7 3.8 3.9 4.0 4.1 4.2</code></pre>
 <p>We will discuss functions in detail later, but note here that the input labels <code>from</code>, <code>to</code>, and <code>by</code> are optional.</p>
-<div class="sourceCode" id="cb20"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb20-1"><a href="data-and-programming.html#cb20-1" aria-hidden="true" tabindex="-1"></a><span class="fu">seq</span>(<span class="fl">1.5</span>, <span class="fl">4.2</span>, <span class="fl">0.1</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb20"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb20-1"><a href="data-and-programming.html#cb20-1" tabindex="-1"></a><span class="fu">seq</span>(<span class="fl">1.5</span>, <span class="fl">4.2</span>, <span class="fl">0.1</span>)</span></code></pre></div>
 <pre><code>##  [1] 1.5 1.6 1.7 1.8 1.9 2.0 2.1 2.2 2.3 2.4 2.5 2.6 2.7 2.8 2.9 3.0 3.1 3.2 3.3
 ## [20] 3.4 3.5 3.6 3.7 3.8 3.9 4.0 4.1 4.2</code></pre>
 <p>Another common operation to create a vector is <code>rep()</code>, which can repeat a single value a number of times.</p>
-<div class="sourceCode" id="cb22"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb22-1"><a href="data-and-programming.html#cb22-1" aria-hidden="true" tabindex="-1"></a><span class="fu">rep</span>(<span class="st">&quot;A&quot;</span>, <span class="at">times =</span> <span class="dv">10</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb22"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb22-1"><a href="data-and-programming.html#cb22-1" tabindex="-1"></a><span class="fu">rep</span>(<span class="st">&quot;A&quot;</span>, <span class="at">times =</span> <span class="dv">10</span>)</span></code></pre></div>
 <pre><code>##  [1] &quot;A&quot; &quot;A&quot; &quot;A&quot; &quot;A&quot; &quot;A&quot; &quot;A&quot; &quot;A&quot; &quot;A&quot; &quot;A&quot; &quot;A&quot;</code></pre>
 <p>The <code>rep()</code> function can be used to repeat a vector some number of times.</p>
-<div class="sourceCode" id="cb24"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb24-1"><a href="data-and-programming.html#cb24-1" aria-hidden="true" tabindex="-1"></a><span class="fu">rep</span>(x, <span class="at">times =</span> <span class="dv">3</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb24"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb24-1"><a href="data-and-programming.html#cb24-1" tabindex="-1"></a><span class="fu">rep</span>(x, <span class="at">times =</span> <span class="dv">3</span>)</span></code></pre></div>
 <pre><code>##  [1] 1 3 5 7 8 9 1 3 5 7 8 9 1 3 5 7 8 9</code></pre>
 <p>We have now seen four different ways to create vectors:</p>
 <ul>
@@ -576,54 +576,54 @@ <h3><span class="header-section-number">3.2.1</span> Vectors<a href="data-and-pr
 <li><code>rep()</code></li>
 </ul>
 <p>So far we have mostly used them in isolation, but they are often used together.</p>
-<div class="sourceCode" id="cb26"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb26-1"><a href="data-and-programming.html#cb26-1" aria-hidden="true" tabindex="-1"></a><span class="fu">c</span>(x, <span class="fu">rep</span>(<span class="fu">seq</span>(<span class="dv">1</span>, <span class="dv">9</span>, <span class="dv">2</span>), <span class="dv">3</span>), <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>, <span class="dv">3</span>), <span class="dv">42</span>, <span class="dv">2</span><span class="sc">:</span><span class="dv">4</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb26"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb26-1"><a href="data-and-programming.html#cb26-1" tabindex="-1"></a><span class="fu">c</span>(x, <span class="fu">rep</span>(<span class="fu">seq</span>(<span class="dv">1</span>, <span class="dv">9</span>, <span class="dv">2</span>), <span class="dv">3</span>), <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>, <span class="dv">3</span>), <span class="dv">42</span>, <span class="dv">2</span><span class="sc">:</span><span class="dv">4</span>)</span></code></pre></div>
 <pre><code>##  [1]  1  3  5  7  8  9  1  3  5  7  9  1  3  5  7  9  1  3  5  7  9  1  2  3 42
 ## [26]  2  3  4</code></pre>
 <p>The length of a vector can be obtained with the <code>length()</code> function.</p>
-<div class="sourceCode" id="cb28"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb28-1"><a href="data-and-programming.html#cb28-1" aria-hidden="true" tabindex="-1"></a><span class="fu">length</span>(x)</span></code></pre></div>
+<div class="sourceCode" id="cb28"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb28-1"><a href="data-and-programming.html#cb28-1" tabindex="-1"></a><span class="fu">length</span>(x)</span></code></pre></div>
 <pre><code>## [1] 6</code></pre>
-<div class="sourceCode" id="cb30"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb30-1"><a href="data-and-programming.html#cb30-1" aria-hidden="true" tabindex="-1"></a><span class="fu">length</span>(y)</span></code></pre></div>
+<div class="sourceCode" id="cb30"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb30-1"><a href="data-and-programming.html#cb30-1" tabindex="-1"></a><span class="fu">length</span>(y)</span></code></pre></div>
 <pre><code>## [1] 100</code></pre>
 <div id="subsetting" class="section level4 hasAnchor" number="3.2.1.1">
 <h4><span class="header-section-number">3.2.1.1</span> Subsetting<a href="data-and-programming.html#subsetting" class="anchor-section" aria-label="Anchor link to header"></a></h4>
 <p>To subset a vector, we use square brackets, <code>[]</code>.</p>
-<div class="sourceCode" id="cb32"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb32-1"><a href="data-and-programming.html#cb32-1" aria-hidden="true" tabindex="-1"></a>x</span></code></pre></div>
+<div class="sourceCode" id="cb32"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb32-1"><a href="data-and-programming.html#cb32-1" tabindex="-1"></a>x</span></code></pre></div>
 <pre><code>## [1] 1 3 5 7 8 9</code></pre>
-<div class="sourceCode" id="cb34"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb34-1"><a href="data-and-programming.html#cb34-1" aria-hidden="true" tabindex="-1"></a>x[<span class="dv">1</span>]</span></code></pre></div>
+<div class="sourceCode" id="cb34"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb34-1"><a href="data-and-programming.html#cb34-1" tabindex="-1"></a>x[<span class="dv">1</span>]</span></code></pre></div>
 <pre><code>## [1] 1</code></pre>
-<div class="sourceCode" id="cb36"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb36-1"><a href="data-and-programming.html#cb36-1" aria-hidden="true" tabindex="-1"></a>x[<span class="dv">3</span>]</span></code></pre></div>
+<div class="sourceCode" id="cb36"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb36-1"><a href="data-and-programming.html#cb36-1" tabindex="-1"></a>x[<span class="dv">3</span>]</span></code></pre></div>
 <pre><code>## [1] 5</code></pre>
 <p>We see that <code>x[1]</code> returns the first element, and <code>x[3]</code> returns the third element.</p>
-<div class="sourceCode" id="cb38"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb38-1"><a href="data-and-programming.html#cb38-1" aria-hidden="true" tabindex="-1"></a>x[<span class="sc">-</span><span class="dv">2</span>]</span></code></pre></div>
+<div class="sourceCode" id="cb38"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb38-1"><a href="data-and-programming.html#cb38-1" tabindex="-1"></a>x[<span class="sc">-</span><span class="dv">2</span>]</span></code></pre></div>
 <pre><code>## [1] 1 5 7 8 9</code></pre>
 <p>We can also exclude certain indexes, in this case the second element.</p>
-<div class="sourceCode" id="cb40"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb40-1"><a href="data-and-programming.html#cb40-1" aria-hidden="true" tabindex="-1"></a>x[<span class="dv">1</span><span class="sc">:</span><span class="dv">3</span>]</span></code></pre></div>
+<div class="sourceCode" id="cb40"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb40-1"><a href="data-and-programming.html#cb40-1" tabindex="-1"></a>x[<span class="dv">1</span><span class="sc">:</span><span class="dv">3</span>]</span></code></pre></div>
 <pre><code>## [1] 1 3 5</code></pre>
-<div class="sourceCode" id="cb42"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb42-1"><a href="data-and-programming.html#cb42-1" aria-hidden="true" tabindex="-1"></a>x[<span class="fu">c</span>(<span class="dv">1</span>,<span class="dv">3</span>,<span class="dv">4</span>)]</span></code></pre></div>
+<div class="sourceCode" id="cb42"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb42-1"><a href="data-and-programming.html#cb42-1" tabindex="-1"></a>x[<span class="fu">c</span>(<span class="dv">1</span>,<span class="dv">3</span>,<span class="dv">4</span>)]</span></code></pre></div>
 <pre><code>## [1] 1 5 7</code></pre>
 <p>Lastly we see that we can subset based on a vector of indices.</p>
 <p>All of the above are subsetting a vector using a vector of indexes. (Remember a single number is still a vector.) We could instead use a vector of logical values.</p>
-<div class="sourceCode" id="cb44"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb44-1"><a href="data-and-programming.html#cb44-1" aria-hidden="true" tabindex="-1"></a>z <span class="ot">=</span> <span class="fu">c</span>(<span class="cn">TRUE</span>, <span class="cn">TRUE</span>, <span class="cn">FALSE</span>, <span class="cn">TRUE</span>, <span class="cn">TRUE</span>, <span class="cn">FALSE</span>)</span>
-<span id="cb44-2"><a href="data-and-programming.html#cb44-2" aria-hidden="true" tabindex="-1"></a>z</span></code></pre></div>
+<div class="sourceCode" id="cb44"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb44-1"><a href="data-and-programming.html#cb44-1" tabindex="-1"></a>z <span class="ot">=</span> <span class="fu">c</span>(<span class="cn">TRUE</span>, <span class="cn">TRUE</span>, <span class="cn">FALSE</span>, <span class="cn">TRUE</span>, <span class="cn">TRUE</span>, <span class="cn">FALSE</span>)</span>
+<span id="cb44-2"><a href="data-and-programming.html#cb44-2" tabindex="-1"></a>z</span></code></pre></div>
 <pre><code>## [1]  TRUE  TRUE FALSE  TRUE  TRUE FALSE</code></pre>
-<div class="sourceCode" id="cb46"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb46-1"><a href="data-and-programming.html#cb46-1" aria-hidden="true" tabindex="-1"></a>x[z]</span></code></pre></div>
+<div class="sourceCode" id="cb46"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb46-1"><a href="data-and-programming.html#cb46-1" tabindex="-1"></a>x[z]</span></code></pre></div>
 <pre><code>## [1] 1 3 7 8</code></pre>
 </div>
 </div>
 <div id="vectorization" class="section level3 hasAnchor" number="3.2.2">
 <h3><span class="header-section-number">3.2.2</span> Vectorization<a href="data-and-programming.html#vectorization" class="anchor-section" aria-label="Anchor link to header"></a></h3>
 <p>One of the biggest strengths of <code>R</code> is its use of vectorized operations. (Frequently the lack of understanding of this concept leads to a belief that <code>R</code> is <em>slow</em>. <code>R</code> is not the fastest language, but it has a reputation for being slower than it really is.)</p>
-<div class="sourceCode" id="cb48"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb48-1"><a href="data-and-programming.html#cb48-1" aria-hidden="true" tabindex="-1"></a>x <span class="ot">=</span> <span class="dv">1</span><span class="sc">:</span><span class="dv">10</span></span>
-<span id="cb48-2"><a href="data-and-programming.html#cb48-2" aria-hidden="true" tabindex="-1"></a>x <span class="sc">+</span> <span class="dv">1</span></span></code></pre></div>
+<div class="sourceCode" id="cb48"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb48-1"><a href="data-and-programming.html#cb48-1" tabindex="-1"></a>x <span class="ot">=</span> <span class="dv">1</span><span class="sc">:</span><span class="dv">10</span></span>
+<span id="cb48-2"><a href="data-and-programming.html#cb48-2" tabindex="-1"></a>x <span class="sc">+</span> <span class="dv">1</span></span></code></pre></div>
 <pre><code>##  [1]  2  3  4  5  6  7  8  9 10 11</code></pre>
-<div class="sourceCode" id="cb50"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb50-1"><a href="data-and-programming.html#cb50-1" aria-hidden="true" tabindex="-1"></a><span class="dv">2</span> <span class="sc">*</span> x</span></code></pre></div>
+<div class="sourceCode" id="cb50"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb50-1"><a href="data-and-programming.html#cb50-1" tabindex="-1"></a><span class="dv">2</span> <span class="sc">*</span> x</span></code></pre></div>
 <pre><code>##  [1]  2  4  6  8 10 12 14 16 18 20</code></pre>
-<div class="sourceCode" id="cb52"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb52-1"><a href="data-and-programming.html#cb52-1" aria-hidden="true" tabindex="-1"></a><span class="dv">2</span> <span class="sc">^</span> x</span></code></pre></div>
+<div class="sourceCode" id="cb52"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb52-1"><a href="data-and-programming.html#cb52-1" tabindex="-1"></a><span class="dv">2</span> <span class="sc">^</span> x</span></code></pre></div>
 <pre><code>##  [1]    2    4    8   16   32   64  128  256  512 1024</code></pre>
-<div class="sourceCode" id="cb54"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb54-1"><a href="data-and-programming.html#cb54-1" aria-hidden="true" tabindex="-1"></a><span class="fu">sqrt</span>(x)</span></code></pre></div>
+<div class="sourceCode" id="cb54"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb54-1"><a href="data-and-programming.html#cb54-1" tabindex="-1"></a><span class="fu">sqrt</span>(x)</span></code></pre></div>
 <pre><code>##  [1] 1.000000 1.414214 1.732051 2.000000 2.236068 2.449490 2.645751 2.828427
 ##  [9] 3.000000 3.162278</code></pre>
-<div class="sourceCode" id="cb56"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb56-1"><a href="data-and-programming.html#cb56-1" aria-hidden="true" tabindex="-1"></a><span class="fu">log</span>(x)</span></code></pre></div>
+<div class="sourceCode" id="cb56"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb56-1"><a href="data-and-programming.html#cb56-1" tabindex="-1"></a><span class="fu">log</span>(x)</span></code></pre></div>
 <pre><code>##  [1] 0.0000000 0.6931472 1.0986123 1.3862944 1.6094379 1.7917595 1.9459101
 ##  [8] 2.0794415 2.1972246 2.3025851</code></pre>
 <p>We see that when a function like <code>log()</code> is called on a vector <code>x</code>, a vector is returned which has applied the function to each element of the vector <code>x</code>.</p>
@@ -703,53 +703,53 @@ <h3><span class="header-section-number">3.2.3</span> Logical Operators<a href="d
 </tbody>
 </table>
 <p>In <code>R</code>, logical operators are vectorized.</p>
-<div class="sourceCode" id="cb58"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb58-1"><a href="data-and-programming.html#cb58-1" aria-hidden="true" tabindex="-1"></a>x <span class="ot">=</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">3</span>, <span class="dv">5</span>, <span class="dv">7</span>, <span class="dv">8</span>, <span class="dv">9</span>)</span></code></pre></div>
-<div class="sourceCode" id="cb59"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb59-1"><a href="data-and-programming.html#cb59-1" aria-hidden="true" tabindex="-1"></a>x <span class="sc">&gt;</span> <span class="dv">3</span></span></code></pre></div>
+<div class="sourceCode" id="cb58"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb58-1"><a href="data-and-programming.html#cb58-1" tabindex="-1"></a>x <span class="ot">=</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">3</span>, <span class="dv">5</span>, <span class="dv">7</span>, <span class="dv">8</span>, <span class="dv">9</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb59"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb59-1"><a href="data-and-programming.html#cb59-1" tabindex="-1"></a>x <span class="sc">&gt;</span> <span class="dv">3</span></span></code></pre></div>
 <pre><code>## [1] FALSE FALSE  TRUE  TRUE  TRUE  TRUE</code></pre>
-<div class="sourceCode" id="cb61"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb61-1"><a href="data-and-programming.html#cb61-1" aria-hidden="true" tabindex="-1"></a>x <span class="sc">&lt;</span> <span class="dv">3</span></span></code></pre></div>
+<div class="sourceCode" id="cb61"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb61-1"><a href="data-and-programming.html#cb61-1" tabindex="-1"></a>x <span class="sc">&lt;</span> <span class="dv">3</span></span></code></pre></div>
 <pre><code>## [1]  TRUE FALSE FALSE FALSE FALSE FALSE</code></pre>
-<div class="sourceCode" id="cb63"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb63-1"><a href="data-and-programming.html#cb63-1" aria-hidden="true" tabindex="-1"></a>x <span class="sc">==</span> <span class="dv">3</span></span></code></pre></div>
+<div class="sourceCode" id="cb63"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb63-1"><a href="data-and-programming.html#cb63-1" tabindex="-1"></a>x <span class="sc">==</span> <span class="dv">3</span></span></code></pre></div>
 <pre><code>## [1] FALSE  TRUE FALSE FALSE FALSE FALSE</code></pre>
-<div class="sourceCode" id="cb65"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb65-1"><a href="data-and-programming.html#cb65-1" aria-hidden="true" tabindex="-1"></a>x <span class="sc">!=</span> <span class="dv">3</span></span></code></pre></div>
+<div class="sourceCode" id="cb65"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb65-1"><a href="data-and-programming.html#cb65-1" tabindex="-1"></a>x <span class="sc">!=</span> <span class="dv">3</span></span></code></pre></div>
 <pre><code>## [1]  TRUE FALSE  TRUE  TRUE  TRUE  TRUE</code></pre>
-<div class="sourceCode" id="cb67"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb67-1"><a href="data-and-programming.html#cb67-1" aria-hidden="true" tabindex="-1"></a>x <span class="sc">==</span> <span class="dv">3</span> <span class="sc">&amp;</span> x <span class="sc">!=</span> <span class="dv">3</span></span></code></pre></div>
+<div class="sourceCode" id="cb67"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb67-1"><a href="data-and-programming.html#cb67-1" tabindex="-1"></a>x <span class="sc">==</span> <span class="dv">3</span> <span class="sc">&amp;</span> x <span class="sc">!=</span> <span class="dv">3</span></span></code></pre></div>
 <pre><code>## [1] FALSE FALSE FALSE FALSE FALSE FALSE</code></pre>
-<div class="sourceCode" id="cb69"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb69-1"><a href="data-and-programming.html#cb69-1" aria-hidden="true" tabindex="-1"></a>x <span class="sc">==</span> <span class="dv">3</span> <span class="sc">|</span> x <span class="sc">!=</span> <span class="dv">3</span></span></code></pre></div>
+<div class="sourceCode" id="cb69"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb69-1"><a href="data-and-programming.html#cb69-1" tabindex="-1"></a>x <span class="sc">==</span> <span class="dv">3</span> <span class="sc">|</span> x <span class="sc">!=</span> <span class="dv">3</span></span></code></pre></div>
 <pre><code>## [1] TRUE TRUE TRUE TRUE TRUE TRUE</code></pre>
 <p>This is extremely useful for subsetting.</p>
-<div class="sourceCode" id="cb71"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb71-1"><a href="data-and-programming.html#cb71-1" aria-hidden="true" tabindex="-1"></a>x[x <span class="sc">&gt;</span> <span class="dv">3</span>]</span></code></pre></div>
+<div class="sourceCode" id="cb71"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb71-1"><a href="data-and-programming.html#cb71-1" tabindex="-1"></a>x[x <span class="sc">&gt;</span> <span class="dv">3</span>]</span></code></pre></div>
 <pre><code>## [1] 5 7 8 9</code></pre>
-<div class="sourceCode" id="cb73"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb73-1"><a href="data-and-programming.html#cb73-1" aria-hidden="true" tabindex="-1"></a>x[x <span class="sc">!=</span> <span class="dv">3</span>]</span></code></pre></div>
+<div class="sourceCode" id="cb73"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb73-1"><a href="data-and-programming.html#cb73-1" tabindex="-1"></a>x[x <span class="sc">!=</span> <span class="dv">3</span>]</span></code></pre></div>
 <pre><code>## [1] 1 5 7 8 9</code></pre>
-<div class="sourceCode" id="cb75"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb75-1"><a href="data-and-programming.html#cb75-1" aria-hidden="true" tabindex="-1"></a><span class="fu">sum</span>(x <span class="sc">&gt;</span> <span class="dv">3</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb75"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb75-1"><a href="data-and-programming.html#cb75-1" tabindex="-1"></a><span class="fu">sum</span>(x <span class="sc">&gt;</span> <span class="dv">3</span>)</span></code></pre></div>
 <pre><code>## [1] 4</code></pre>
-<div class="sourceCode" id="cb77"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb77-1"><a href="data-and-programming.html#cb77-1" aria-hidden="true" tabindex="-1"></a><span class="fu">as.numeric</span>(x <span class="sc">&gt;</span> <span class="dv">3</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb77"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb77-1"><a href="data-and-programming.html#cb77-1" tabindex="-1"></a><span class="fu">as.numeric</span>(x <span class="sc">&gt;</span> <span class="dv">3</span>)</span></code></pre></div>
 <pre><code>## [1] 0 0 1 1 1 1</code></pre>
 <p>Here we see that using the <code>sum()</code> function on a vector of logical <code>TRUE</code> and <code>FALSE</code> values that is the result of <code>x &gt; 3</code> results in a numeric result. <code>R</code> is first automatically coercing the logical to numeric where <code>TRUE</code> is <code>1</code> and <code>FALSE</code> is <code>0</code>. This coercion from logical to numeric happens for most mathematical operations. If you are interested in more detail, check out <a href="https://adv-r.hadley.nz/vectors-chap.html?q=coercion#testing-and-coercion">Advanced R</a>.</p>
-<div class="sourceCode" id="cb79"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb79-1"><a href="data-and-programming.html#cb79-1" aria-hidden="true" tabindex="-1"></a><span class="fu">which</span>(x <span class="sc">&gt;</span> <span class="dv">3</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb79"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb79-1"><a href="data-and-programming.html#cb79-1" tabindex="-1"></a><span class="fu">which</span>(x <span class="sc">&gt;</span> <span class="dv">3</span>)</span></code></pre></div>
 <pre><code>## [1] 3 4 5 6</code></pre>
-<div class="sourceCode" id="cb81"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb81-1"><a href="data-and-programming.html#cb81-1" aria-hidden="true" tabindex="-1"></a>x[<span class="fu">which</span>(x <span class="sc">&gt;</span> <span class="dv">3</span>)]</span></code></pre></div>
+<div class="sourceCode" id="cb81"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb81-1"><a href="data-and-programming.html#cb81-1" tabindex="-1"></a>x[<span class="fu">which</span>(x <span class="sc">&gt;</span> <span class="dv">3</span>)]</span></code></pre></div>
 <pre><code>## [1] 5 7 8 9</code></pre>
-<div class="sourceCode" id="cb83"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb83-1"><a href="data-and-programming.html#cb83-1" aria-hidden="true" tabindex="-1"></a><span class="fu">max</span>(x)</span></code></pre></div>
+<div class="sourceCode" id="cb83"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb83-1"><a href="data-and-programming.html#cb83-1" tabindex="-1"></a><span class="fu">max</span>(x)</span></code></pre></div>
 <pre><code>## [1] 9</code></pre>
-<div class="sourceCode" id="cb85"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb85-1"><a href="data-and-programming.html#cb85-1" aria-hidden="true" tabindex="-1"></a><span class="fu">which</span>(x <span class="sc">==</span> <span class="fu">max</span>(x))</span></code></pre></div>
+<div class="sourceCode" id="cb85"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb85-1"><a href="data-and-programming.html#cb85-1" tabindex="-1"></a><span class="fu">which</span>(x <span class="sc">==</span> <span class="fu">max</span>(x))</span></code></pre></div>
 <pre><code>## [1] 6</code></pre>
-<div class="sourceCode" id="cb87"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb87-1"><a href="data-and-programming.html#cb87-1" aria-hidden="true" tabindex="-1"></a><span class="fu">which.max</span>(x)</span></code></pre></div>
+<div class="sourceCode" id="cb87"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb87-1"><a href="data-and-programming.html#cb87-1" tabindex="-1"></a><span class="fu">which.max</span>(x)</span></code></pre></div>
 <pre><code>## [1] 6</code></pre>
 </div>
 <div id="more-vectorization" class="section level3 hasAnchor" number="3.2.4">
 <h3><span class="header-section-number">3.2.4</span> More Vectorization<a href="data-and-programming.html#more-vectorization" class="anchor-section" aria-label="Anchor link to header"></a></h3>
-<div class="sourceCode" id="cb89"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb89-1"><a href="data-and-programming.html#cb89-1" aria-hidden="true" tabindex="-1"></a>x <span class="ot">=</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">3</span>, <span class="dv">5</span>, <span class="dv">7</span>, <span class="dv">8</span>, <span class="dv">9</span>)</span>
-<span id="cb89-2"><a href="data-and-programming.html#cb89-2" aria-hidden="true" tabindex="-1"></a>y <span class="ot">=</span> <span class="dv">1</span><span class="sc">:</span><span class="dv">100</span></span></code></pre></div>
-<div class="sourceCode" id="cb90"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb90-1"><a href="data-and-programming.html#cb90-1" aria-hidden="true" tabindex="-1"></a>x <span class="sc">+</span> <span class="dv">2</span></span></code></pre></div>
+<div class="sourceCode" id="cb89"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb89-1"><a href="data-and-programming.html#cb89-1" tabindex="-1"></a>x <span class="ot">=</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">3</span>, <span class="dv">5</span>, <span class="dv">7</span>, <span class="dv">8</span>, <span class="dv">9</span>)</span>
+<span id="cb89-2"><a href="data-and-programming.html#cb89-2" tabindex="-1"></a>y <span class="ot">=</span> <span class="dv">1</span><span class="sc">:</span><span class="dv">100</span></span></code></pre></div>
+<div class="sourceCode" id="cb90"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb90-1"><a href="data-and-programming.html#cb90-1" tabindex="-1"></a>x <span class="sc">+</span> <span class="dv">2</span></span></code></pre></div>
 <pre><code>## [1]  3  5  7  9 10 11</code></pre>
-<div class="sourceCode" id="cb92"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb92-1"><a href="data-and-programming.html#cb92-1" aria-hidden="true" tabindex="-1"></a>x <span class="sc">+</span> <span class="fu">rep</span>(<span class="dv">2</span>, <span class="dv">6</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb92"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb92-1"><a href="data-and-programming.html#cb92-1" tabindex="-1"></a>x <span class="sc">+</span> <span class="fu">rep</span>(<span class="dv">2</span>, <span class="dv">6</span>)</span></code></pre></div>
 <pre><code>## [1]  3  5  7  9 10 11</code></pre>
-<div class="sourceCode" id="cb94"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb94-1"><a href="data-and-programming.html#cb94-1" aria-hidden="true" tabindex="-1"></a>x <span class="sc">&gt;</span> <span class="dv">3</span></span></code></pre></div>
+<div class="sourceCode" id="cb94"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb94-1"><a href="data-and-programming.html#cb94-1" tabindex="-1"></a>x <span class="sc">&gt;</span> <span class="dv">3</span></span></code></pre></div>
 <pre><code>## [1] FALSE FALSE  TRUE  TRUE  TRUE  TRUE</code></pre>
-<div class="sourceCode" id="cb96"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb96-1"><a href="data-and-programming.html#cb96-1" aria-hidden="true" tabindex="-1"></a>x <span class="sc">&gt;</span> <span class="fu">rep</span>(<span class="dv">3</span>, <span class="dv">6</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb96"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb96-1"><a href="data-and-programming.html#cb96-1" tabindex="-1"></a>x <span class="sc">&gt;</span> <span class="fu">rep</span>(<span class="dv">3</span>, <span class="dv">6</span>)</span></code></pre></div>
 <pre><code>## [1] FALSE FALSE  TRUE  TRUE  TRUE  TRUE</code></pre>
-<div class="sourceCode" id="cb98"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb98-1"><a href="data-and-programming.html#cb98-1" aria-hidden="true" tabindex="-1"></a>x <span class="sc">+</span> y</span></code></pre></div>
+<div class="sourceCode" id="cb98"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb98-1"><a href="data-and-programming.html#cb98-1" tabindex="-1"></a>x <span class="sc">+</span> y</span></code></pre></div>
 <pre><code>## Warning in x + y: longer object length is not a multiple of shorter object
 ## length</code></pre>
 <pre><code>##   [1]   2   5   8  11  13  15   8  11  14  17  19  21  14  17  20  23  25  27
@@ -758,49 +758,49 @@ <h3><span class="header-section-number">3.2.4</span> More Vectorization<a href="
 ##  [55]  56  59  62  65  67  69  62  65  68  71  73  75  68  71  74  77  79  81
 ##  [73]  74  77  80  83  85  87  80  83  86  89  91  93  86  89  92  95  97  99
 ##  [91]  92  95  98 101 103 105  98 101 104 107</code></pre>
-<div class="sourceCode" id="cb101"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb101-1"><a href="data-and-programming.html#cb101-1" aria-hidden="true" tabindex="-1"></a><span class="fu">length</span>(x)</span></code></pre></div>
+<div class="sourceCode" id="cb101"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb101-1"><a href="data-and-programming.html#cb101-1" tabindex="-1"></a><span class="fu">length</span>(x)</span></code></pre></div>
 <pre><code>## [1] 6</code></pre>
-<div class="sourceCode" id="cb103"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb103-1"><a href="data-and-programming.html#cb103-1" aria-hidden="true" tabindex="-1"></a><span class="fu">length</span>(y)</span></code></pre></div>
+<div class="sourceCode" id="cb103"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb103-1"><a href="data-and-programming.html#cb103-1" tabindex="-1"></a><span class="fu">length</span>(y)</span></code></pre></div>
 <pre><code>## [1] 100</code></pre>
-<div class="sourceCode" id="cb105"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb105-1"><a href="data-and-programming.html#cb105-1" aria-hidden="true" tabindex="-1"></a><span class="fu">length</span>(y) <span class="sc">/</span> <span class="fu">length</span>(x)</span></code></pre></div>
+<div class="sourceCode" id="cb105"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb105-1"><a href="data-and-programming.html#cb105-1" tabindex="-1"></a><span class="fu">length</span>(y) <span class="sc">/</span> <span class="fu">length</span>(x)</span></code></pre></div>
 <pre><code>## [1] 16.66667</code></pre>
-<div class="sourceCode" id="cb107"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb107-1"><a href="data-and-programming.html#cb107-1" aria-hidden="true" tabindex="-1"></a>(x <span class="sc">+</span> y) <span class="sc">-</span> y</span></code></pre></div>
+<div class="sourceCode" id="cb107"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb107-1"><a href="data-and-programming.html#cb107-1" tabindex="-1"></a>(x <span class="sc">+</span> y) <span class="sc">-</span> y</span></code></pre></div>
 <pre><code>## Warning in x + y: longer object length is not a multiple of shorter object
 ## length</code></pre>
 <pre><code>##   [1] 1 3 5 7 8 9 1 3 5 7 8 9 1 3 5 7 8 9 1 3 5 7 8 9 1 3 5 7 8 9 1 3 5 7 8 9 1
 ##  [38] 3 5 7 8 9 1 3 5 7 8 9 1 3 5 7 8 9 1 3 5 7 8 9 1 3 5 7 8 9 1 3 5 7 8 9 1 3
 ##  [75] 5 7 8 9 1 3 5 7 8 9 1 3 5 7 8 9 1 3 5 7 8 9 1 3 5 7</code></pre>
-<div class="sourceCode" id="cb110"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb110-1"><a href="data-and-programming.html#cb110-1" aria-hidden="true" tabindex="-1"></a>y <span class="ot">=</span> <span class="dv">1</span><span class="sc">:</span><span class="dv">60</span></span>
-<span id="cb110-2"><a href="data-and-programming.html#cb110-2" aria-hidden="true" tabindex="-1"></a>x <span class="sc">+</span> y</span></code></pre></div>
+<div class="sourceCode" id="cb110"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb110-1"><a href="data-and-programming.html#cb110-1" tabindex="-1"></a>y <span class="ot">=</span> <span class="dv">1</span><span class="sc">:</span><span class="dv">60</span></span>
+<span id="cb110-2"><a href="data-and-programming.html#cb110-2" tabindex="-1"></a>x <span class="sc">+</span> y</span></code></pre></div>
 <pre><code>##  [1]  2  5  8 11 13 15  8 11 14 17 19 21 14 17 20 23 25 27 20 23 26 29 31 33 26
 ## [26] 29 32 35 37 39 32 35 38 41 43 45 38 41 44 47 49 51 44 47 50 53 55 57 50 53
 ## [51] 56 59 61 63 56 59 62 65 67 69</code></pre>
-<div class="sourceCode" id="cb112"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb112-1"><a href="data-and-programming.html#cb112-1" aria-hidden="true" tabindex="-1"></a><span class="fu">length</span>(y) <span class="sc">/</span> <span class="fu">length</span>(x)</span></code></pre></div>
+<div class="sourceCode" id="cb112"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb112-1"><a href="data-and-programming.html#cb112-1" tabindex="-1"></a><span class="fu">length</span>(y) <span class="sc">/</span> <span class="fu">length</span>(x)</span></code></pre></div>
 <pre><code>## [1] 10</code></pre>
-<div class="sourceCode" id="cb114"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb114-1"><a href="data-and-programming.html#cb114-1" aria-hidden="true" tabindex="-1"></a><span class="fu">rep</span>(x, <span class="dv">10</span>) <span class="sc">+</span> y</span></code></pre></div>
+<div class="sourceCode" id="cb114"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb114-1"><a href="data-and-programming.html#cb114-1" tabindex="-1"></a><span class="fu">rep</span>(x, <span class="dv">10</span>) <span class="sc">+</span> y</span></code></pre></div>
 <pre><code>##  [1]  2  5  8 11 13 15  8 11 14 17 19 21 14 17 20 23 25 27 20 23 26 29 31 33 26
 ## [26] 29 32 35 37 39 32 35 38 41 43 45 38 41 44 47 49 51 44 47 50 53 55 57 50 53
 ## [51] 56 59 61 63 56 59 62 65 67 69</code></pre>
-<div class="sourceCode" id="cb116"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb116-1"><a href="data-and-programming.html#cb116-1" aria-hidden="true" tabindex="-1"></a><span class="fu">all</span>(x <span class="sc">+</span> y <span class="sc">==</span> <span class="fu">rep</span>(x, <span class="dv">10</span>) <span class="sc">+</span> y)</span></code></pre></div>
+<div class="sourceCode" id="cb116"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb116-1"><a href="data-and-programming.html#cb116-1" tabindex="-1"></a><span class="fu">all</span>(x <span class="sc">+</span> y <span class="sc">==</span> <span class="fu">rep</span>(x, <span class="dv">10</span>) <span class="sc">+</span> y)</span></code></pre></div>
 <pre><code>## [1] TRUE</code></pre>
-<div class="sourceCode" id="cb118"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb118-1"><a href="data-and-programming.html#cb118-1" aria-hidden="true" tabindex="-1"></a><span class="fu">identical</span>(x <span class="sc">+</span> y, <span class="fu">rep</span>(x, <span class="dv">10</span>) <span class="sc">+</span> y)</span></code></pre></div>
+<div class="sourceCode" id="cb118"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb118-1"><a href="data-and-programming.html#cb118-1" tabindex="-1"></a><span class="fu">identical</span>(x <span class="sc">+</span> y, <span class="fu">rep</span>(x, <span class="dv">10</span>) <span class="sc">+</span> y)</span></code></pre></div>
 <pre><code>## [1] TRUE</code></pre>
-<div class="sourceCode" id="cb120"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb120-1"><a href="data-and-programming.html#cb120-1" aria-hidden="true" tabindex="-1"></a><span class="co"># ?any</span></span>
-<span id="cb120-2"><a href="data-and-programming.html#cb120-2" aria-hidden="true" tabindex="-1"></a><span class="co"># ?all.equal</span></span></code></pre></div>
-<div class="sourceCode" id="cb121"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb121-1"><a href="data-and-programming.html#cb121-1" aria-hidden="true" tabindex="-1"></a>x <span class="ot">=</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">3</span>, <span class="dv">5</span>)</span>
-<span id="cb121-2"><a href="data-and-programming.html#cb121-2" aria-hidden="true" tabindex="-1"></a>y <span class="ot">=</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>, <span class="dv">4</span>)</span>
-<span id="cb121-3"><a href="data-and-programming.html#cb121-3" aria-hidden="true" tabindex="-1"></a>x <span class="sc">==</span> y</span></code></pre></div>
+<div class="sourceCode" id="cb120"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb120-1"><a href="data-and-programming.html#cb120-1" tabindex="-1"></a><span class="co"># ?any</span></span>
+<span id="cb120-2"><a href="data-and-programming.html#cb120-2" tabindex="-1"></a><span class="co"># ?all.equal</span></span></code></pre></div>
+<div class="sourceCode" id="cb121"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb121-1"><a href="data-and-programming.html#cb121-1" tabindex="-1"></a>x <span class="ot">=</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">3</span>, <span class="dv">5</span>)</span>
+<span id="cb121-2"><a href="data-and-programming.html#cb121-2" tabindex="-1"></a>y <span class="ot">=</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>, <span class="dv">4</span>)</span>
+<span id="cb121-3"><a href="data-and-programming.html#cb121-3" tabindex="-1"></a>x <span class="sc">==</span> y</span></code></pre></div>
 <pre><code>## [1]  TRUE FALSE FALSE</code></pre>
-<div class="sourceCode" id="cb123"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb123-1"><a href="data-and-programming.html#cb123-1" aria-hidden="true" tabindex="-1"></a><span class="fu">all</span>(x <span class="sc">==</span> y)</span></code></pre></div>
+<div class="sourceCode" id="cb123"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb123-1"><a href="data-and-programming.html#cb123-1" tabindex="-1"></a><span class="fu">all</span>(x <span class="sc">==</span> y)</span></code></pre></div>
 <pre><code>## [1] FALSE</code></pre>
-<div class="sourceCode" id="cb125"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb125-1"><a href="data-and-programming.html#cb125-1" aria-hidden="true" tabindex="-1"></a><span class="fu">any</span>(x <span class="sc">==</span> y)</span></code></pre></div>
+<div class="sourceCode" id="cb125"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb125-1"><a href="data-and-programming.html#cb125-1" tabindex="-1"></a><span class="fu">any</span>(x <span class="sc">==</span> y)</span></code></pre></div>
 <pre><code>## [1] TRUE</code></pre>
 <p>While <code>all</code> returns <code>TRUE</code> only when all of its arguments are <code>TRUE</code>, <code>any</code> returns <code>TRUE</code> when at least one of its arguments is <code>TRUE</code>.</p>
-<div class="sourceCode" id="cb127"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb127-1"><a href="data-and-programming.html#cb127-1" aria-hidden="true" tabindex="-1"></a>x <span class="ot">=</span> <span class="fu">c</span>(<span class="dv">10</span> <span class="sc">^</span> (<span class="sc">-</span><span class="dv">8</span>))</span>
-<span id="cb127-2"><a href="data-and-programming.html#cb127-2" aria-hidden="true" tabindex="-1"></a>y <span class="ot">=</span> <span class="fu">c</span>(<span class="dv">10</span> <span class="sc">^</span> (<span class="sc">-</span><span class="dv">9</span>))</span>
-<span id="cb127-3"><a href="data-and-programming.html#cb127-3" aria-hidden="true" tabindex="-1"></a><span class="fu">all</span>(x <span class="sc">==</span> y)</span></code></pre></div>
+<div class="sourceCode" id="cb127"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb127-1"><a href="data-and-programming.html#cb127-1" tabindex="-1"></a>x <span class="ot">=</span> <span class="fu">c</span>(<span class="dv">10</span> <span class="sc">^</span> (<span class="sc">-</span><span class="dv">8</span>))</span>
+<span id="cb127-2"><a href="data-and-programming.html#cb127-2" tabindex="-1"></a>y <span class="ot">=</span> <span class="fu">c</span>(<span class="dv">10</span> <span class="sc">^</span> (<span class="sc">-</span><span class="dv">9</span>))</span>
+<span id="cb127-3"><a href="data-and-programming.html#cb127-3" tabindex="-1"></a><span class="fu">all</span>(x <span class="sc">==</span> y)</span></code></pre></div>
 <pre><code>## [1] FALSE</code></pre>
-<div class="sourceCode" id="cb129"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb129-1"><a href="data-and-programming.html#cb129-1" aria-hidden="true" tabindex="-1"></a><span class="fu">all.equal</span>(x, y)</span></code></pre></div>
+<div class="sourceCode" id="cb129"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb129-1"><a href="data-and-programming.html#cb129-1" tabindex="-1"></a><span class="fu">all.equal</span>(x, y)</span></code></pre></div>
 <pre><code>## [1] TRUE</code></pre>
 <p>The <code>all.equal</code> function tests “near equality” with a default tolerance value around <code>1.5e-8</code> and returns <code>TRUE</code> if all of its arguments have differences smaller than the tolerance.</p>
 </div>
@@ -808,57 +808,57 @@ <h3><span class="header-section-number">3.2.4</span> More Vectorization<a href="
 <h3><span class="header-section-number">3.2.5</span> Matrices<a href="data-and-programming.html#matrices" class="anchor-section" aria-label="Anchor link to header"></a></h3>
 <p><code>R</code> can also be used for <strong>matrix</strong> calculations. Matrices have rows and columns containing a single data type. In a matrix, the order of rows and columns is important. (This is not true of <em>data frames</em>, which we will see later.)</p>
 <p>Matrices can be created using the <code>matrix</code> function.</p>
-<div class="sourceCode" id="cb131"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb131-1"><a href="data-and-programming.html#cb131-1" aria-hidden="true" tabindex="-1"></a>x <span class="ot">=</span> <span class="dv">1</span><span class="sc">:</span><span class="dv">9</span></span>
-<span id="cb131-2"><a href="data-and-programming.html#cb131-2" aria-hidden="true" tabindex="-1"></a>x</span></code></pre></div>
+<div class="sourceCode" id="cb131"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb131-1"><a href="data-and-programming.html#cb131-1" tabindex="-1"></a>x <span class="ot">=</span> <span class="dv">1</span><span class="sc">:</span><span class="dv">9</span></span>
+<span id="cb131-2"><a href="data-and-programming.html#cb131-2" tabindex="-1"></a>x</span></code></pre></div>
 <pre><code>## [1] 1 2 3 4 5 6 7 8 9</code></pre>
-<div class="sourceCode" id="cb133"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb133-1"><a href="data-and-programming.html#cb133-1" aria-hidden="true" tabindex="-1"></a>X <span class="ot">=</span> <span class="fu">matrix</span>(x, <span class="at">nrow =</span> <span class="dv">3</span>, <span class="at">ncol =</span> <span class="dv">3</span>)</span>
-<span id="cb133-2"><a href="data-and-programming.html#cb133-2" aria-hidden="true" tabindex="-1"></a>X</span></code></pre></div>
+<div class="sourceCode" id="cb133"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb133-1"><a href="data-and-programming.html#cb133-1" tabindex="-1"></a>X <span class="ot">=</span> <span class="fu">matrix</span>(x, <span class="at">nrow =</span> <span class="dv">3</span>, <span class="at">ncol =</span> <span class="dv">3</span>)</span>
+<span id="cb133-2"><a href="data-and-programming.html#cb133-2" tabindex="-1"></a>X</span></code></pre></div>
 <pre><code>##      [,1] [,2] [,3]
 ## [1,]    1    4    7
 ## [2,]    2    5    8
 ## [3,]    3    6    9</code></pre>
 <p>Note here that we are using two different variables: lower case <code>x</code>, which stores a vector and capital <code>X</code>, which stores a matrix. (Following the usual mathematical convention.) We can do this because <code>R</code> is case sensitive.</p>
 <p>By default the <code>matrix</code> function reorders a vector into columns, but we can also tell <code>R</code> to use rows instead.</p>
-<div class="sourceCode" id="cb135"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb135-1"><a href="data-and-programming.html#cb135-1" aria-hidden="true" tabindex="-1"></a>Y <span class="ot">=</span> <span class="fu">matrix</span>(x, <span class="at">nrow =</span> <span class="dv">3</span>, <span class="at">ncol =</span> <span class="dv">3</span>, <span class="at">byrow =</span> <span class="cn">TRUE</span>)</span>
-<span id="cb135-2"><a href="data-and-programming.html#cb135-2" aria-hidden="true" tabindex="-1"></a>Y</span></code></pre></div>
+<div class="sourceCode" id="cb135"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb135-1"><a href="data-and-programming.html#cb135-1" tabindex="-1"></a>Y <span class="ot">=</span> <span class="fu">matrix</span>(x, <span class="at">nrow =</span> <span class="dv">3</span>, <span class="at">ncol =</span> <span class="dv">3</span>, <span class="at">byrow =</span> <span class="cn">TRUE</span>)</span>
+<span id="cb135-2"><a href="data-and-programming.html#cb135-2" tabindex="-1"></a>Y</span></code></pre></div>
 <pre><code>##      [,1] [,2] [,3]
 ## [1,]    1    2    3
 ## [2,]    4    5    6
 ## [3,]    7    8    9</code></pre>
 <p>We can also create a matrix of a specified dimension where every element is the same, in this case <code>0</code>.</p>
-<div class="sourceCode" id="cb137"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb137-1"><a href="data-and-programming.html#cb137-1" aria-hidden="true" tabindex="-1"></a>Z <span class="ot">=</span> <span class="fu">matrix</span>(<span class="dv">0</span>, <span class="dv">2</span>, <span class="dv">4</span>)</span>
-<span id="cb137-2"><a href="data-and-programming.html#cb137-2" aria-hidden="true" tabindex="-1"></a>Z</span></code></pre></div>
+<div class="sourceCode" id="cb137"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb137-1"><a href="data-and-programming.html#cb137-1" tabindex="-1"></a>Z <span class="ot">=</span> <span class="fu">matrix</span>(<span class="dv">0</span>, <span class="dv">2</span>, <span class="dv">4</span>)</span>
+<span id="cb137-2"><a href="data-and-programming.html#cb137-2" tabindex="-1"></a>Z</span></code></pre></div>
 <pre><code>##      [,1] [,2] [,3] [,4]
 ## [1,]    0    0    0    0
 ## [2,]    0    0    0    0</code></pre>
 <p>Like vectors, matrices can be subsetted using square brackets, <code>[]</code>. However, since matrices are two-dimensional, we need to specify both a row and a column when subsetting.</p>
-<div class="sourceCode" id="cb139"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb139-1"><a href="data-and-programming.html#cb139-1" aria-hidden="true" tabindex="-1"></a>X</span></code></pre></div>
+<div class="sourceCode" id="cb139"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb139-1"><a href="data-and-programming.html#cb139-1" tabindex="-1"></a>X</span></code></pre></div>
 <pre><code>##      [,1] [,2] [,3]
 ## [1,]    1    4    7
 ## [2,]    2    5    8
 ## [3,]    3    6    9</code></pre>
-<div class="sourceCode" id="cb141"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb141-1"><a href="data-and-programming.html#cb141-1" aria-hidden="true" tabindex="-1"></a>X[<span class="dv">1</span>, <span class="dv">2</span>]</span></code></pre></div>
+<div class="sourceCode" id="cb141"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb141-1"><a href="data-and-programming.html#cb141-1" tabindex="-1"></a>X[<span class="dv">1</span>, <span class="dv">2</span>]</span></code></pre></div>
 <pre><code>## [1] 4</code></pre>
 <p>Here we accessed the element in the first row and the second column. We could also subset an entire row or column.</p>
-<div class="sourceCode" id="cb143"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb143-1"><a href="data-and-programming.html#cb143-1" aria-hidden="true" tabindex="-1"></a>X[<span class="dv">1</span>, ]</span></code></pre></div>
+<div class="sourceCode" id="cb143"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb143-1"><a href="data-and-programming.html#cb143-1" tabindex="-1"></a>X[<span class="dv">1</span>, ]</span></code></pre></div>
 <pre><code>## [1] 1 4 7</code></pre>
-<div class="sourceCode" id="cb145"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb145-1"><a href="data-and-programming.html#cb145-1" aria-hidden="true" tabindex="-1"></a>X[, <span class="dv">2</span>]</span></code></pre></div>
+<div class="sourceCode" id="cb145"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb145-1"><a href="data-and-programming.html#cb145-1" tabindex="-1"></a>X[, <span class="dv">2</span>]</span></code></pre></div>
 <pre><code>## [1] 4 5 6</code></pre>
 <p>We can also use vectors to subset more than one row or column at a time. Here we subset to the first and third column of the second row.</p>
-<div class="sourceCode" id="cb147"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb147-1"><a href="data-and-programming.html#cb147-1" aria-hidden="true" tabindex="-1"></a>X[<span class="dv">2</span>, <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">3</span>)]</span></code></pre></div>
+<div class="sourceCode" id="cb147"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb147-1"><a href="data-and-programming.html#cb147-1" tabindex="-1"></a>X[<span class="dv">2</span>, <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">3</span>)]</span></code></pre></div>
 <pre><code>## [1] 2 8</code></pre>
 <p>Matrices can also be created by combining vectors as columns, using <code>cbind</code>, or combining vectors as rows, using <code>rbind</code>.</p>
-<div class="sourceCode" id="cb149"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb149-1"><a href="data-and-programming.html#cb149-1" aria-hidden="true" tabindex="-1"></a>x <span class="ot">=</span> <span class="dv">1</span><span class="sc">:</span><span class="dv">9</span></span>
-<span id="cb149-2"><a href="data-and-programming.html#cb149-2" aria-hidden="true" tabindex="-1"></a><span class="fu">rev</span>(x)</span></code></pre></div>
+<div class="sourceCode" id="cb149"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb149-1"><a href="data-and-programming.html#cb149-1" tabindex="-1"></a>x <span class="ot">=</span> <span class="dv">1</span><span class="sc">:</span><span class="dv">9</span></span>
+<span id="cb149-2"><a href="data-and-programming.html#cb149-2" tabindex="-1"></a><span class="fu">rev</span>(x)</span></code></pre></div>
 <pre><code>## [1] 9 8 7 6 5 4 3 2 1</code></pre>
-<div class="sourceCode" id="cb151"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb151-1"><a href="data-and-programming.html#cb151-1" aria-hidden="true" tabindex="-1"></a><span class="fu">rep</span>(<span class="dv">1</span>, <span class="dv">9</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb151"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb151-1"><a href="data-and-programming.html#cb151-1" tabindex="-1"></a><span class="fu">rep</span>(<span class="dv">1</span>, <span class="dv">9</span>)</span></code></pre></div>
 <pre><code>## [1] 1 1 1 1 1 1 1 1 1</code></pre>
-<div class="sourceCode" id="cb153"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb153-1"><a href="data-and-programming.html#cb153-1" aria-hidden="true" tabindex="-1"></a><span class="fu">rbind</span>(x, <span class="fu">rev</span>(x), <span class="fu">rep</span>(<span class="dv">1</span>, <span class="dv">9</span>))</span></code></pre></div>
+<div class="sourceCode" id="cb153"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb153-1"><a href="data-and-programming.html#cb153-1" tabindex="-1"></a><span class="fu">rbind</span>(x, <span class="fu">rev</span>(x), <span class="fu">rep</span>(<span class="dv">1</span>, <span class="dv">9</span>))</span></code></pre></div>
 <pre><code>##   [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9]
 ## x    1    2    3    4    5    6    7    8    9
 ##      9    8    7    6    5    4    3    2    1
 ##      1    1    1    1    1    1    1    1    1</code></pre>
-<div class="sourceCode" id="cb155"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb155-1"><a href="data-and-programming.html#cb155-1" aria-hidden="true" tabindex="-1"></a><span class="fu">cbind</span>(<span class="at">col_1 =</span> x, <span class="at">col_2 =</span> <span class="fu">rev</span>(x), <span class="at">col_3 =</span> <span class="fu">rep</span>(<span class="dv">1</span>, <span class="dv">9</span>))</span></code></pre></div>
+<div class="sourceCode" id="cb155"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb155-1"><a href="data-and-programming.html#cb155-1" tabindex="-1"></a><span class="fu">cbind</span>(<span class="at">col_1 =</span> x, <span class="at">col_2 =</span> <span class="fu">rev</span>(x), <span class="at">col_3 =</span> <span class="fu">rep</span>(<span class="dv">1</span>, <span class="dv">9</span>))</span></code></pre></div>
 <pre><code>##       col_1 col_2 col_3
 ##  [1,]     1     9     1
 ##  [2,]     2     8     1
@@ -871,96 +871,96 @@ <h3><span class="header-section-number">3.2.5</span> Matrices<a href="data-and-p
 ##  [9,]     9     1     1</code></pre>
 <p>When using <code>rbind</code> and <code>cbind</code> you can specify “argument” names that will be used as column names.</p>
 <p><code>R</code> can then be used to perform matrix calculations.</p>
-<div class="sourceCode" id="cb157"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb157-1"><a href="data-and-programming.html#cb157-1" aria-hidden="true" tabindex="-1"></a>x <span class="ot">=</span> <span class="dv">1</span><span class="sc">:</span><span class="dv">9</span></span>
-<span id="cb157-2"><a href="data-and-programming.html#cb157-2" aria-hidden="true" tabindex="-1"></a>y <span class="ot">=</span> <span class="dv">9</span><span class="sc">:</span><span class="dv">1</span></span>
-<span id="cb157-3"><a href="data-and-programming.html#cb157-3" aria-hidden="true" tabindex="-1"></a>X <span class="ot">=</span> <span class="fu">matrix</span>(x, <span class="dv">3</span>, <span class="dv">3</span>)</span>
-<span id="cb157-4"><a href="data-and-programming.html#cb157-4" aria-hidden="true" tabindex="-1"></a>Y <span class="ot">=</span> <span class="fu">matrix</span>(y, <span class="dv">3</span>, <span class="dv">3</span>)</span>
-<span id="cb157-5"><a href="data-and-programming.html#cb157-5" aria-hidden="true" tabindex="-1"></a>X</span></code></pre></div>
+<div class="sourceCode" id="cb157"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb157-1"><a href="data-and-programming.html#cb157-1" tabindex="-1"></a>x <span class="ot">=</span> <span class="dv">1</span><span class="sc">:</span><span class="dv">9</span></span>
+<span id="cb157-2"><a href="data-and-programming.html#cb157-2" tabindex="-1"></a>y <span class="ot">=</span> <span class="dv">9</span><span class="sc">:</span><span class="dv">1</span></span>
+<span id="cb157-3"><a href="data-and-programming.html#cb157-3" tabindex="-1"></a>X <span class="ot">=</span> <span class="fu">matrix</span>(x, <span class="dv">3</span>, <span class="dv">3</span>)</span>
+<span id="cb157-4"><a href="data-and-programming.html#cb157-4" tabindex="-1"></a>Y <span class="ot">=</span> <span class="fu">matrix</span>(y, <span class="dv">3</span>, <span class="dv">3</span>)</span>
+<span id="cb157-5"><a href="data-and-programming.html#cb157-5" tabindex="-1"></a>X</span></code></pre></div>
 <pre><code>##      [,1] [,2] [,3]
 ## [1,]    1    4    7
 ## [2,]    2    5    8
 ## [3,]    3    6    9</code></pre>
-<div class="sourceCode" id="cb159"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb159-1"><a href="data-and-programming.html#cb159-1" aria-hidden="true" tabindex="-1"></a>Y</span></code></pre></div>
+<div class="sourceCode" id="cb159"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb159-1"><a href="data-and-programming.html#cb159-1" tabindex="-1"></a>Y</span></code></pre></div>
 <pre><code>##      [,1] [,2] [,3]
 ## [1,]    9    6    3
 ## [2,]    8    5    2
 ## [3,]    7    4    1</code></pre>
-<div class="sourceCode" id="cb161"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb161-1"><a href="data-and-programming.html#cb161-1" aria-hidden="true" tabindex="-1"></a>X <span class="sc">+</span> Y</span></code></pre></div>
+<div class="sourceCode" id="cb161"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb161-1"><a href="data-and-programming.html#cb161-1" tabindex="-1"></a>X <span class="sc">+</span> Y</span></code></pre></div>
 <pre><code>##      [,1] [,2] [,3]
 ## [1,]   10   10   10
 ## [2,]   10   10   10
 ## [3,]   10   10   10</code></pre>
-<div class="sourceCode" id="cb163"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb163-1"><a href="data-and-programming.html#cb163-1" aria-hidden="true" tabindex="-1"></a>X <span class="sc">-</span> Y</span></code></pre></div>
+<div class="sourceCode" id="cb163"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb163-1"><a href="data-and-programming.html#cb163-1" tabindex="-1"></a>X <span class="sc">-</span> Y</span></code></pre></div>
 <pre><code>##      [,1] [,2] [,3]
 ## [1,]   -8   -2    4
 ## [2,]   -6    0    6
 ## [3,]   -4    2    8</code></pre>
-<div class="sourceCode" id="cb165"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb165-1"><a href="data-and-programming.html#cb165-1" aria-hidden="true" tabindex="-1"></a>X <span class="sc">*</span> Y</span></code></pre></div>
+<div class="sourceCode" id="cb165"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb165-1"><a href="data-and-programming.html#cb165-1" tabindex="-1"></a>X <span class="sc">*</span> Y</span></code></pre></div>
 <pre><code>##      [,1] [,2] [,3]
 ## [1,]    9   24   21
 ## [2,]   16   25   16
 ## [3,]   21   24    9</code></pre>
-<div class="sourceCode" id="cb167"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb167-1"><a href="data-and-programming.html#cb167-1" aria-hidden="true" tabindex="-1"></a>X <span class="sc">/</span> Y</span></code></pre></div>
+<div class="sourceCode" id="cb167"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb167-1"><a href="data-and-programming.html#cb167-1" tabindex="-1"></a>X <span class="sc">/</span> Y</span></code></pre></div>
 <pre><code>##           [,1]      [,2]     [,3]
 ## [1,] 0.1111111 0.6666667 2.333333
 ## [2,] 0.2500000 1.0000000 4.000000
 ## [3,] 0.4285714 1.5000000 9.000000</code></pre>
 <p>Note that <code>X * Y</code> is not matrix multiplication. It is element by element multiplication. (Same for <code>X / Y</code>). Instead, matrix multiplication uses <code>%*%</code>. Other matrix functions include <code>t()</code> which gives the transpose of a matrix and <code>solve()</code> which returns the inverse of a square matrix if it is invertible.</p>
-<div class="sourceCode" id="cb169"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb169-1"><a href="data-and-programming.html#cb169-1" aria-hidden="true" tabindex="-1"></a>X <span class="sc">%*%</span> Y</span></code></pre></div>
+<div class="sourceCode" id="cb169"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb169-1"><a href="data-and-programming.html#cb169-1" tabindex="-1"></a>X <span class="sc">%*%</span> Y</span></code></pre></div>
 <pre><code>##      [,1] [,2] [,3]
 ## [1,]   90   54   18
 ## [2,]  114   69   24
 ## [3,]  138   84   30</code></pre>
-<div class="sourceCode" id="cb171"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb171-1"><a href="data-and-programming.html#cb171-1" aria-hidden="true" tabindex="-1"></a><span class="fu">t</span>(X)</span></code></pre></div>
+<div class="sourceCode" id="cb171"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb171-1"><a href="data-and-programming.html#cb171-1" tabindex="-1"></a><span class="fu">t</span>(X)</span></code></pre></div>
 <pre><code>##      [,1] [,2] [,3]
 ## [1,]    1    2    3
 ## [2,]    4    5    6
 ## [3,]    7    8    9</code></pre>
-<div class="sourceCode" id="cb173"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb173-1"><a href="data-and-programming.html#cb173-1" aria-hidden="true" tabindex="-1"></a>Z <span class="ot">=</span> <span class="fu">matrix</span>(<span class="fu">c</span>(<span class="dv">9</span>, <span class="dv">2</span>, <span class="sc">-</span><span class="dv">3</span>, <span class="dv">2</span>, <span class="dv">4</span>, <span class="sc">-</span><span class="dv">2</span>, <span class="sc">-</span><span class="dv">3</span>, <span class="sc">-</span><span class="dv">2</span>, <span class="dv">16</span>), <span class="dv">3</span>, <span class="at">byrow =</span> <span class="cn">TRUE</span>)</span>
-<span id="cb173-2"><a href="data-and-programming.html#cb173-2" aria-hidden="true" tabindex="-1"></a>Z</span></code></pre></div>
+<div class="sourceCode" id="cb173"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb173-1"><a href="data-and-programming.html#cb173-1" tabindex="-1"></a>Z <span class="ot">=</span> <span class="fu">matrix</span>(<span class="fu">c</span>(<span class="dv">9</span>, <span class="dv">2</span>, <span class="sc">-</span><span class="dv">3</span>, <span class="dv">2</span>, <span class="dv">4</span>, <span class="sc">-</span><span class="dv">2</span>, <span class="sc">-</span><span class="dv">3</span>, <span class="sc">-</span><span class="dv">2</span>, <span class="dv">16</span>), <span class="dv">3</span>, <span class="at">byrow =</span> <span class="cn">TRUE</span>)</span>
+<span id="cb173-2"><a href="data-and-programming.html#cb173-2" tabindex="-1"></a>Z</span></code></pre></div>
 <pre><code>##      [,1] [,2] [,3]
 ## [1,]    9    2   -3
 ## [2,]    2    4   -2
 ## [3,]   -3   -2   16</code></pre>
-<div class="sourceCode" id="cb175"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb175-1"><a href="data-and-programming.html#cb175-1" aria-hidden="true" tabindex="-1"></a><span class="fu">solve</span>(Z)</span></code></pre></div>
+<div class="sourceCode" id="cb175"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb175-1"><a href="data-and-programming.html#cb175-1" tabindex="-1"></a><span class="fu">solve</span>(Z)</span></code></pre></div>
 <pre><code>##             [,1]        [,2]       [,3]
 ## [1,]  0.12931034 -0.05603448 0.01724138
 ## [2,] -0.05603448  0.29094828 0.02586207
 ## [3,]  0.01724138  0.02586207 0.06896552</code></pre>
 <p>To verify that <code>solve(Z)</code> returns the inverse, we multiply it by <code>Z</code>. We would expect this to return the identity matrix, however we see that this is not the case due to some computational issues. However, <code>R</code> also has the <code>all.equal()</code> function which checks for equality, with some small tolerance which accounts for some computational issues. The <code>identical()</code> function is used to check for exact equality.</p>
-<div class="sourceCode" id="cb177"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb177-1"><a href="data-and-programming.html#cb177-1" aria-hidden="true" tabindex="-1"></a><span class="fu">solve</span>(Z) <span class="sc">%*%</span> Z</span></code></pre></div>
+<div class="sourceCode" id="cb177"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb177-1"><a href="data-and-programming.html#cb177-1" tabindex="-1"></a><span class="fu">solve</span>(Z) <span class="sc">%*%</span> Z</span></code></pre></div>
 <pre><code>##              [,1]          [,2]         [,3]
 ## [1,] 1.000000e+00 -6.245005e-17 0.000000e+00
-## [2,] 8.326673e-17  1.000000e+00 5.551115e-17
+## [2,] 7.979728e-17  1.000000e+00 5.551115e-17
 ## [3,] 2.775558e-17  0.000000e+00 1.000000e+00</code></pre>
-<div class="sourceCode" id="cb179"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb179-1"><a href="data-and-programming.html#cb179-1" aria-hidden="true" tabindex="-1"></a><span class="fu">diag</span>(<span class="dv">3</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb179"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb179-1"><a href="data-and-programming.html#cb179-1" tabindex="-1"></a><span class="fu">diag</span>(<span class="dv">3</span>)</span></code></pre></div>
 <pre><code>##      [,1] [,2] [,3]
 ## [1,]    1    0    0
 ## [2,]    0    1    0
 ## [3,]    0    0    1</code></pre>
-<div class="sourceCode" id="cb181"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb181-1"><a href="data-and-programming.html#cb181-1" aria-hidden="true" tabindex="-1"></a><span class="fu">all.equal</span>(<span class="fu">solve</span>(Z) <span class="sc">%*%</span> Z, <span class="fu">diag</span>(<span class="dv">3</span>))</span></code></pre></div>
+<div class="sourceCode" id="cb181"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb181-1"><a href="data-and-programming.html#cb181-1" tabindex="-1"></a><span class="fu">all.equal</span>(<span class="fu">solve</span>(Z) <span class="sc">%*%</span> Z, <span class="fu">diag</span>(<span class="dv">3</span>))</span></code></pre></div>
 <pre><code>## [1] TRUE</code></pre>
 <p><code>R</code> has a number of matrix specific functions for obtaining dimension and summary information.</p>
-<div class="sourceCode" id="cb183"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb183-1"><a href="data-and-programming.html#cb183-1" aria-hidden="true" tabindex="-1"></a>X <span class="ot">=</span> <span class="fu">matrix</span>(<span class="dv">1</span><span class="sc">:</span><span class="dv">6</span>, <span class="dv">2</span>, <span class="dv">3</span>)</span>
-<span id="cb183-2"><a href="data-and-programming.html#cb183-2" aria-hidden="true" tabindex="-1"></a>X</span></code></pre></div>
+<div class="sourceCode" id="cb183"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb183-1"><a href="data-and-programming.html#cb183-1" tabindex="-1"></a>X <span class="ot">=</span> <span class="fu">matrix</span>(<span class="dv">1</span><span class="sc">:</span><span class="dv">6</span>, <span class="dv">2</span>, <span class="dv">3</span>)</span>
+<span id="cb183-2"><a href="data-and-programming.html#cb183-2" tabindex="-1"></a>X</span></code></pre></div>
 <pre><code>##      [,1] [,2] [,3]
 ## [1,]    1    3    5
 ## [2,]    2    4    6</code></pre>
-<div class="sourceCode" id="cb185"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb185-1"><a href="data-and-programming.html#cb185-1" aria-hidden="true" tabindex="-1"></a><span class="fu">dim</span>(X)</span></code></pre></div>
+<div class="sourceCode" id="cb185"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb185-1"><a href="data-and-programming.html#cb185-1" tabindex="-1"></a><span class="fu">dim</span>(X)</span></code></pre></div>
 <pre><code>## [1] 2 3</code></pre>
-<div class="sourceCode" id="cb187"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb187-1"><a href="data-and-programming.html#cb187-1" aria-hidden="true" tabindex="-1"></a><span class="fu">rowSums</span>(X)</span></code></pre></div>
+<div class="sourceCode" id="cb187"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb187-1"><a href="data-and-programming.html#cb187-1" tabindex="-1"></a><span class="fu">rowSums</span>(X)</span></code></pre></div>
 <pre><code>## [1]  9 12</code></pre>
-<div class="sourceCode" id="cb189"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb189-1"><a href="data-and-programming.html#cb189-1" aria-hidden="true" tabindex="-1"></a><span class="fu">colSums</span>(X)</span></code></pre></div>
+<div class="sourceCode" id="cb189"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb189-1"><a href="data-and-programming.html#cb189-1" tabindex="-1"></a><span class="fu">colSums</span>(X)</span></code></pre></div>
 <pre><code>## [1]  3  7 11</code></pre>
-<div class="sourceCode" id="cb191"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb191-1"><a href="data-and-programming.html#cb191-1" aria-hidden="true" tabindex="-1"></a><span class="fu">rowMeans</span>(X)</span></code></pre></div>
+<div class="sourceCode" id="cb191"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb191-1"><a href="data-and-programming.html#cb191-1" tabindex="-1"></a><span class="fu">rowMeans</span>(X)</span></code></pre></div>
 <pre><code>## [1] 3 4</code></pre>
-<div class="sourceCode" id="cb193"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb193-1"><a href="data-and-programming.html#cb193-1" aria-hidden="true" tabindex="-1"></a><span class="fu">colMeans</span>(X)</span></code></pre></div>
+<div class="sourceCode" id="cb193"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb193-1"><a href="data-and-programming.html#cb193-1" tabindex="-1"></a><span class="fu">colMeans</span>(X)</span></code></pre></div>
 <pre><code>## [1] 1.5 3.5 5.5</code></pre>
 <p>The <code>diag()</code> function can be used in a number of ways. We can extract the diagonal of a matrix.</p>
-<div class="sourceCode" id="cb195"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb195-1"><a href="data-and-programming.html#cb195-1" aria-hidden="true" tabindex="-1"></a><span class="fu">diag</span>(Z)</span></code></pre></div>
+<div class="sourceCode" id="cb195"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb195-1"><a href="data-and-programming.html#cb195-1" tabindex="-1"></a><span class="fu">diag</span>(Z)</span></code></pre></div>
 <pre><code>## [1]  9  4 16</code></pre>
 <p>Or create a matrix with specified elements on the diagonal. (And <code>0</code> on the off-diagonals.)</p>
-<div class="sourceCode" id="cb197"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb197-1"><a href="data-and-programming.html#cb197-1" aria-hidden="true" tabindex="-1"></a><span class="fu">diag</span>(<span class="dv">1</span><span class="sc">:</span><span class="dv">5</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb197"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb197-1"><a href="data-and-programming.html#cb197-1" tabindex="-1"></a><span class="fu">diag</span>(<span class="dv">1</span><span class="sc">:</span><span class="dv">5</span>)</span></code></pre></div>
 <pre><code>##      [,1] [,2] [,3] [,4] [,5]
 ## [1,]    1    0    0    0    0
 ## [2,]    0    2    0    0    0
@@ -968,7 +968,7 @@ <h3><span class="header-section-number">3.2.5</span> Matrices<a href="data-and-p
 ## [4,]    0    0    0    4    0
 ## [5,]    0    0    0    0    5</code></pre>
 <p>Or, lastly, create a square matrix of a certain dimension with <code>1</code> for every element of the diagonal and <code>0</code> for the off-diagonals.</p>
-<div class="sourceCode" id="cb199"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb199-1"><a href="data-and-programming.html#cb199-1" aria-hidden="true" tabindex="-1"></a><span class="fu">diag</span>(<span class="dv">5</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb199"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb199-1"><a href="data-and-programming.html#cb199-1" tabindex="-1"></a><span class="fu">diag</span>(<span class="dv">5</span>)</span></code></pre></div>
 <pre><code>##      [,1] [,2] [,3] [,4] [,5]
 ## [1,]    1    0    0    0    0
 ## [2,]    0    1    0    0    0
@@ -978,47 +978,47 @@ <h3><span class="header-section-number">3.2.5</span> Matrices<a href="data-and-p
 <div id="calculations-with-vectors-and-matrices" class="section level4 unnumbered hasAnchor">
 <h4>Calculations with Vectors and Matrices<a href="data-and-programming.html#calculations-with-vectors-and-matrices" class="anchor-section" aria-label="Anchor link to header"></a></h4>
 <p>Certain operations in <code>R</code>, for example <code>%*%</code> have different behavior on vectors and matrices. To illustrate this, we will first create two vectors.</p>
-<div class="sourceCode" id="cb201"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb201-1"><a href="data-and-programming.html#cb201-1" aria-hidden="true" tabindex="-1"></a>a_vec <span class="ot">=</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>, <span class="dv">3</span>)</span>
-<span id="cb201-2"><a href="data-and-programming.html#cb201-2" aria-hidden="true" tabindex="-1"></a>b_vec <span class="ot">=</span> <span class="fu">c</span>(<span class="dv">2</span>, <span class="dv">2</span>, <span class="dv">2</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb201"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb201-1"><a href="data-and-programming.html#cb201-1" tabindex="-1"></a>a_vec <span class="ot">=</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>, <span class="dv">3</span>)</span>
+<span id="cb201-2"><a href="data-and-programming.html#cb201-2" tabindex="-1"></a>b_vec <span class="ot">=</span> <span class="fu">c</span>(<span class="dv">2</span>, <span class="dv">2</span>, <span class="dv">2</span>)</span></code></pre></div>
 <p>Note that these are indeed vectors. They are not matrices.</p>
-<div class="sourceCode" id="cb202"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb202-1"><a href="data-and-programming.html#cb202-1" aria-hidden="true" tabindex="-1"></a><span class="fu">c</span>(<span class="fu">is.vector</span>(a_vec), <span class="fu">is.vector</span>(b_vec))</span></code></pre></div>
+<div class="sourceCode" id="cb202"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb202-1"><a href="data-and-programming.html#cb202-1" tabindex="-1"></a><span class="fu">c</span>(<span class="fu">is.vector</span>(a_vec), <span class="fu">is.vector</span>(b_vec))</span></code></pre></div>
 <pre><code>## [1] TRUE TRUE</code></pre>
-<div class="sourceCode" id="cb204"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb204-1"><a href="data-and-programming.html#cb204-1" aria-hidden="true" tabindex="-1"></a><span class="fu">c</span>(<span class="fu">is.matrix</span>(a_vec), <span class="fu">is.matrix</span>(b_vec))</span></code></pre></div>
+<div class="sourceCode" id="cb204"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb204-1"><a href="data-and-programming.html#cb204-1" tabindex="-1"></a><span class="fu">c</span>(<span class="fu">is.matrix</span>(a_vec), <span class="fu">is.matrix</span>(b_vec))</span></code></pre></div>
 <pre><code>## [1] FALSE FALSE</code></pre>
 <p>When this is the case, the <code>%*%</code> operator is used to calculate the <strong>dot product</strong>, also known as the <strong>inner product</strong> of the two vectors.</p>
 <p>The dot product of vectors <span class="math inline">\(\boldsymbol{a} = \lbrack a_1, a_2, \cdots a_n \rbrack\)</span> and <span class="math inline">\(\boldsymbol{b} = \lbrack b_1, b_2, \cdots b_n \rbrack\)</span> is defined to be</p>
 <p><span class="math display">\[
 \boldsymbol{a} \cdot \boldsymbol{b} = \sum_{i = 1}^{n} a_i b_i = a_1 b_1 + a_2 b_2 + \cdots a_n b_n.
 \]</span></p>
-<div class="sourceCode" id="cb206"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb206-1"><a href="data-and-programming.html#cb206-1" aria-hidden="true" tabindex="-1"></a>a_vec <span class="sc">%*%</span> b_vec <span class="co"># inner product</span></span></code></pre></div>
+<div class="sourceCode" id="cb206"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb206-1"><a href="data-and-programming.html#cb206-1" tabindex="-1"></a>a_vec <span class="sc">%*%</span> b_vec <span class="co"># inner product</span></span></code></pre></div>
 <pre><code>##      [,1]
 ## [1,]   12</code></pre>
-<div class="sourceCode" id="cb208"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb208-1"><a href="data-and-programming.html#cb208-1" aria-hidden="true" tabindex="-1"></a>a_vec <span class="sc">%o%</span> b_vec <span class="co"># outer product</span></span></code></pre></div>
+<div class="sourceCode" id="cb208"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb208-1"><a href="data-and-programming.html#cb208-1" tabindex="-1"></a>a_vec <span class="sc">%o%</span> b_vec <span class="co"># outer product</span></span></code></pre></div>
 <pre><code>##      [,1] [,2] [,3]
 ## [1,]    2    2    2
 ## [2,]    4    4    4
 ## [3,]    6    6    6</code></pre>
 <p>The <code>%o%</code> operator is used to calculate the <strong>outer product</strong> of the two vectors.</p>
 <p>When vectors are coerced to become matrices, they are column vectors. So a vector of length <span class="math inline">\(n\)</span> becomes an <span class="math inline">\(n \times 1\)</span> matrix after coercion.</p>
-<div class="sourceCode" id="cb210"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb210-1"><a href="data-and-programming.html#cb210-1" aria-hidden="true" tabindex="-1"></a><span class="fu">as.matrix</span>(a_vec)</span></code></pre></div>
+<div class="sourceCode" id="cb210"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb210-1"><a href="data-and-programming.html#cb210-1" tabindex="-1"></a><span class="fu">as.matrix</span>(a_vec)</span></code></pre></div>
 <pre><code>##      [,1]
 ## [1,]    1
 ## [2,]    2
 ## [3,]    3</code></pre>
 <p>If we use the <code>%*%</code> operator on matrices, <code>%*%</code> again performs the expected matrix multiplication. So you might expect the following to produce an error, because the dimensions are incorrect.</p>
-<div class="sourceCode" id="cb212"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb212-1"><a href="data-and-programming.html#cb212-1" aria-hidden="true" tabindex="-1"></a><span class="fu">as.matrix</span>(a_vec) <span class="sc">%*%</span> b_vec</span></code></pre></div>
+<div class="sourceCode" id="cb212"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb212-1"><a href="data-and-programming.html#cb212-1" tabindex="-1"></a><span class="fu">as.matrix</span>(a_vec) <span class="sc">%*%</span> b_vec</span></code></pre></div>
 <pre><code>##      [,1] [,2] [,3]
 ## [1,]    2    2    2
 ## [2,]    4    4    4
 ## [3,]    6    6    6</code></pre>
 <p>At face value this is a <span class="math inline">\(3 \times 1\)</span> matrix, multiplied by a <span class="math inline">\(3 \times 1\)</span> matrix. However, when <code>b_vec</code> is automatically coerced to be a matrix, <code>R</code> decided to make it a “row vector,” a <span class="math inline">\(1 \times 3\)</span> matrix, so that the multiplication has conformable dimensions.</p>
 <p>If we had coerced both, then <code>R</code> would produce an error.</p>
-<div class="sourceCode" id="cb214"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb214-1"><a href="data-and-programming.html#cb214-1" aria-hidden="true" tabindex="-1"></a><span class="fu">as.matrix</span>(a_vec) <span class="sc">%*%</span> <span class="fu">as.matrix</span>(b_vec)</span></code></pre></div>
+<div class="sourceCode" id="cb214"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb214-1"><a href="data-and-programming.html#cb214-1" tabindex="-1"></a><span class="fu">as.matrix</span>(a_vec) <span class="sc">%*%</span> <span class="fu">as.matrix</span>(b_vec)</span></code></pre></div>
 <p>Another way to calculate a <em>dot product</em> is with the <code>crossprod()</code> function. Given two vectors, the <code>crossprod()</code> function calculates their dot product. The function has a rather misleading name.</p>
-<div class="sourceCode" id="cb215"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb215-1"><a href="data-and-programming.html#cb215-1" aria-hidden="true" tabindex="-1"></a><span class="fu">crossprod</span>(a_vec, b_vec)  <span class="co"># inner product</span></span></code></pre></div>
+<div class="sourceCode" id="cb215"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb215-1"><a href="data-and-programming.html#cb215-1" tabindex="-1"></a><span class="fu">crossprod</span>(a_vec, b_vec)  <span class="co"># inner product</span></span></code></pre></div>
 <pre><code>##      [,1]
 ## [1,]   12</code></pre>
-<div class="sourceCode" id="cb217"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb217-1"><a href="data-and-programming.html#cb217-1" aria-hidden="true" tabindex="-1"></a><span class="fu">tcrossprod</span>(a_vec, b_vec)  <span class="co"># outer product</span></span></code></pre></div>
+<div class="sourceCode" id="cb217"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb217-1"><a href="data-and-programming.html#cb217-1" tabindex="-1"></a><span class="fu">tcrossprod</span>(a_vec, b_vec)  <span class="co"># outer product</span></span></code></pre></div>
 <pre><code>##      [,1] [,2] [,3]
 ## [1,]    2    2    2
 ## [2,]    4    4    4
@@ -1032,40 +1032,40 @@ <h4>Calculations with Vectors and Matrices<a href="data-and-programming.html#cal
 X^\top X
 \]</span></p>
 <p>is used repeatedly.</p>
-<div class="sourceCode" id="cb219"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb219-1"><a href="data-and-programming.html#cb219-1" aria-hidden="true" tabindex="-1"></a>C_mat <span class="ot">=</span> <span class="fu">matrix</span>(<span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>, <span class="dv">3</span>, <span class="dv">4</span>, <span class="dv">5</span>, <span class="dv">6</span>), <span class="dv">2</span>, <span class="dv">3</span>)</span>
-<span id="cb219-2"><a href="data-and-programming.html#cb219-2" aria-hidden="true" tabindex="-1"></a>D_mat <span class="ot">=</span> <span class="fu">matrix</span>(<span class="fu">c</span>(<span class="dv">2</span>, <span class="dv">2</span>, <span class="dv">2</span>, <span class="dv">2</span>, <span class="dv">2</span>, <span class="dv">2</span>), <span class="dv">2</span>, <span class="dv">3</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb219"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb219-1"><a href="data-and-programming.html#cb219-1" tabindex="-1"></a>C_mat <span class="ot">=</span> <span class="fu">matrix</span>(<span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>, <span class="dv">3</span>, <span class="dv">4</span>, <span class="dv">5</span>, <span class="dv">6</span>), <span class="dv">2</span>, <span class="dv">3</span>)</span>
+<span id="cb219-2"><a href="data-and-programming.html#cb219-2" tabindex="-1"></a>D_mat <span class="ot">=</span> <span class="fu">matrix</span>(<span class="fu">c</span>(<span class="dv">2</span>, <span class="dv">2</span>, <span class="dv">2</span>, <span class="dv">2</span>, <span class="dv">2</span>, <span class="dv">2</span>), <span class="dv">2</span>, <span class="dv">3</span>)</span></code></pre></div>
 <p>This is useful both as a shortcut for a frequent calculation and as a more efficient implementation than using <code>t()</code> and <code>%*%</code>.</p>
-<div class="sourceCode" id="cb220"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb220-1"><a href="data-and-programming.html#cb220-1" aria-hidden="true" tabindex="-1"></a><span class="fu">crossprod</span>(C_mat, D_mat)</span></code></pre></div>
+<div class="sourceCode" id="cb220"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb220-1"><a href="data-and-programming.html#cb220-1" tabindex="-1"></a><span class="fu">crossprod</span>(C_mat, D_mat)</span></code></pre></div>
 <pre><code>##      [,1] [,2] [,3]
 ## [1,]    6    6    6
 ## [2,]   14   14   14
 ## [3,]   22   22   22</code></pre>
-<div class="sourceCode" id="cb222"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb222-1"><a href="data-and-programming.html#cb222-1" aria-hidden="true" tabindex="-1"></a><span class="fu">t</span>(C_mat) <span class="sc">%*%</span> D_mat</span></code></pre></div>
+<div class="sourceCode" id="cb222"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb222-1"><a href="data-and-programming.html#cb222-1" tabindex="-1"></a><span class="fu">t</span>(C_mat) <span class="sc">%*%</span> D_mat</span></code></pre></div>
 <pre><code>##      [,1] [,2] [,3]
 ## [1,]    6    6    6
 ## [2,]   14   14   14
 ## [3,]   22   22   22</code></pre>
-<div class="sourceCode" id="cb224"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb224-1"><a href="data-and-programming.html#cb224-1" aria-hidden="true" tabindex="-1"></a><span class="fu">all.equal</span>(<span class="fu">crossprod</span>(C_mat, D_mat), <span class="fu">t</span>(C_mat) <span class="sc">%*%</span> D_mat)</span></code></pre></div>
+<div class="sourceCode" id="cb224"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb224-1"><a href="data-and-programming.html#cb224-1" tabindex="-1"></a><span class="fu">all.equal</span>(<span class="fu">crossprod</span>(C_mat, D_mat), <span class="fu">t</span>(C_mat) <span class="sc">%*%</span> D_mat)</span></code></pre></div>
 <pre><code>## [1] TRUE</code></pre>
-<div class="sourceCode" id="cb226"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb226-1"><a href="data-and-programming.html#cb226-1" aria-hidden="true" tabindex="-1"></a><span class="fu">crossprod</span>(C_mat, C_mat)</span></code></pre></div>
+<div class="sourceCode" id="cb226"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb226-1"><a href="data-and-programming.html#cb226-1" tabindex="-1"></a><span class="fu">crossprod</span>(C_mat, C_mat)</span></code></pre></div>
 <pre><code>##      [,1] [,2] [,3]
 ## [1,]    5   11   17
 ## [2,]   11   25   39
 ## [3,]   17   39   61</code></pre>
-<div class="sourceCode" id="cb228"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb228-1"><a href="data-and-programming.html#cb228-1" aria-hidden="true" tabindex="-1"></a><span class="fu">t</span>(C_mat) <span class="sc">%*%</span> C_mat</span></code></pre></div>
+<div class="sourceCode" id="cb228"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb228-1"><a href="data-and-programming.html#cb228-1" tabindex="-1"></a><span class="fu">t</span>(C_mat) <span class="sc">%*%</span> C_mat</span></code></pre></div>
 <pre><code>##      [,1] [,2] [,3]
 ## [1,]    5   11   17
 ## [2,]   11   25   39
 ## [3,]   17   39   61</code></pre>
-<div class="sourceCode" id="cb230"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb230-1"><a href="data-and-programming.html#cb230-1" aria-hidden="true" tabindex="-1"></a><span class="fu">all.equal</span>(<span class="fu">crossprod</span>(C_mat, C_mat), <span class="fu">t</span>(C_mat) <span class="sc">%*%</span> C_mat)</span></code></pre></div>
+<div class="sourceCode" id="cb230"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb230-1"><a href="data-and-programming.html#cb230-1" tabindex="-1"></a><span class="fu">all.equal</span>(<span class="fu">crossprod</span>(C_mat, C_mat), <span class="fu">t</span>(C_mat) <span class="sc">%*%</span> C_mat)</span></code></pre></div>
 <pre><code>## [1] TRUE</code></pre>
 </div>
 </div>
 <div id="lists" class="section level3 hasAnchor" number="3.2.6">
 <h3><span class="header-section-number">3.2.6</span> Lists<a href="data-and-programming.html#lists" class="anchor-section" aria-label="Anchor link to header"></a></h3>
 <p>A list is a one-dimensional heterogeneous data structure. So it is indexed like a vector with a single integer value, but each element can contain an element of any type.</p>
-<div class="sourceCode" id="cb232"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb232-1"><a href="data-and-programming.html#cb232-1" aria-hidden="true" tabindex="-1"></a><span class="co"># creation</span></span>
-<span id="cb232-2"><a href="data-and-programming.html#cb232-2" aria-hidden="true" tabindex="-1"></a><span class="fu">list</span>(<span class="dv">42</span>, <span class="st">&quot;Hello&quot;</span>, <span class="cn">TRUE</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb232"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb232-1"><a href="data-and-programming.html#cb232-1" tabindex="-1"></a><span class="co"># creation</span></span>
+<span id="cb232-2"><a href="data-and-programming.html#cb232-2" tabindex="-1"></a><span class="fu">list</span>(<span class="dv">42</span>, <span class="st">&quot;Hello&quot;</span>, <span class="cn">TRUE</span>)</span></code></pre></div>
 <pre><code>## [[1]]
 ## [1] 42
 ## 
@@ -1074,38 +1074,38 @@ <h3><span class="header-section-number">3.2.6</span> Lists<a href="data-and-prog
 ## 
 ## [[3]]
 ## [1] TRUE</code></pre>
-<div class="sourceCode" id="cb234"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb234-1"><a href="data-and-programming.html#cb234-1" aria-hidden="true" tabindex="-1"></a>ex_list <span class="ot">=</span> <span class="fu">list</span>(</span>
-<span id="cb234-2"><a href="data-and-programming.html#cb234-2" aria-hidden="true" tabindex="-1"></a>  <span class="at">a =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>, <span class="dv">3</span>, <span class="dv">4</span>),</span>
-<span id="cb234-3"><a href="data-and-programming.html#cb234-3" aria-hidden="true" tabindex="-1"></a>  <span class="at">b =</span> <span class="cn">TRUE</span>,</span>
-<span id="cb234-4"><a href="data-and-programming.html#cb234-4" aria-hidden="true" tabindex="-1"></a>  <span class="at">c =</span> <span class="st">&quot;Hello!&quot;</span>,</span>
-<span id="cb234-5"><a href="data-and-programming.html#cb234-5" aria-hidden="true" tabindex="-1"></a>  <span class="at">d =</span> <span class="cf">function</span>(<span class="at">arg =</span> <span class="dv">42</span>) {<span class="fu">print</span>(<span class="st">&quot;Hello World!&quot;</span>)},</span>
-<span id="cb234-6"><a href="data-and-programming.html#cb234-6" aria-hidden="true" tabindex="-1"></a>  <span class="at">e =</span> <span class="fu">diag</span>(<span class="dv">5</span>)</span>
-<span id="cb234-7"><a href="data-and-programming.html#cb234-7" aria-hidden="true" tabindex="-1"></a>)</span></code></pre></div>
+<div class="sourceCode" id="cb234"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb234-1"><a href="data-and-programming.html#cb234-1" tabindex="-1"></a>ex_list <span class="ot">=</span> <span class="fu">list</span>(</span>
+<span id="cb234-2"><a href="data-and-programming.html#cb234-2" tabindex="-1"></a>  <span class="at">a =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>, <span class="dv">3</span>, <span class="dv">4</span>),</span>
+<span id="cb234-3"><a href="data-and-programming.html#cb234-3" tabindex="-1"></a>  <span class="at">b =</span> <span class="cn">TRUE</span>,</span>
+<span id="cb234-4"><a href="data-and-programming.html#cb234-4" tabindex="-1"></a>  <span class="at">c =</span> <span class="st">&quot;Hello!&quot;</span>,</span>
+<span id="cb234-5"><a href="data-and-programming.html#cb234-5" tabindex="-1"></a>  <span class="at">d =</span> <span class="cf">function</span>(<span class="at">arg =</span> <span class="dv">42</span>) {<span class="fu">print</span>(<span class="st">&quot;Hello World!&quot;</span>)},</span>
+<span id="cb234-6"><a href="data-and-programming.html#cb234-6" tabindex="-1"></a>  <span class="at">e =</span> <span class="fu">diag</span>(<span class="dv">5</span>)</span>
+<span id="cb234-7"><a href="data-and-programming.html#cb234-7" tabindex="-1"></a>)</span></code></pre></div>
 <p>Lists can be subset using two syntaxes, the <code>$</code> operator, and square brackets <code>[]</code>. The <code>$</code> operator returns a named <strong>element</strong> of a list. The <code>[]</code> syntax returns a <strong>list</strong>, while the <code>[[]]</code> returns an <strong>element</strong> of a list.</p>
 <ul>
 <li><code>ex_list[1]</code> returns a list containing the first element.</li>
 <li><code>ex_list[[1]]</code> returns the first element of the list, in this case, a vector.</li>
 </ul>
-<div class="sourceCode" id="cb235"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb235-1"><a href="data-and-programming.html#cb235-1" aria-hidden="true" tabindex="-1"></a><span class="co"># subsetting</span></span>
-<span id="cb235-2"><a href="data-and-programming.html#cb235-2" aria-hidden="true" tabindex="-1"></a>ex_list<span class="sc">$</span>e</span></code></pre></div>
+<div class="sourceCode" id="cb235"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb235-1"><a href="data-and-programming.html#cb235-1" tabindex="-1"></a><span class="co"># subsetting</span></span>
+<span id="cb235-2"><a href="data-and-programming.html#cb235-2" tabindex="-1"></a>ex_list<span class="sc">$</span>e</span></code></pre></div>
 <pre><code>##      [,1] [,2] [,3] [,4] [,5]
 ## [1,]    1    0    0    0    0
 ## [2,]    0    1    0    0    0
 ## [3,]    0    0    1    0    0
 ## [4,]    0    0    0    1    0
 ## [5,]    0    0    0    0    1</code></pre>
-<div class="sourceCode" id="cb237"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb237-1"><a href="data-and-programming.html#cb237-1" aria-hidden="true" tabindex="-1"></a>ex_list[<span class="dv">1</span><span class="sc">:</span><span class="dv">2</span>]</span></code></pre></div>
+<div class="sourceCode" id="cb237"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb237-1"><a href="data-and-programming.html#cb237-1" tabindex="-1"></a>ex_list[<span class="dv">1</span><span class="sc">:</span><span class="dv">2</span>]</span></code></pre></div>
 <pre><code>## $a
 ## [1] 1 2 3 4
 ## 
 ## $b
 ## [1] TRUE</code></pre>
-<div class="sourceCode" id="cb239"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb239-1"><a href="data-and-programming.html#cb239-1" aria-hidden="true" tabindex="-1"></a>ex_list[<span class="dv">1</span>]</span></code></pre></div>
+<div class="sourceCode" id="cb239"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb239-1"><a href="data-and-programming.html#cb239-1" tabindex="-1"></a>ex_list[<span class="dv">1</span>]</span></code></pre></div>
 <pre><code>## $a
 ## [1] 1 2 3 4</code></pre>
-<div class="sourceCode" id="cb241"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb241-1"><a href="data-and-programming.html#cb241-1" aria-hidden="true" tabindex="-1"></a>ex_list[[<span class="dv">1</span>]]</span></code></pre></div>
+<div class="sourceCode" id="cb241"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb241-1"><a href="data-and-programming.html#cb241-1" tabindex="-1"></a>ex_list[[<span class="dv">1</span>]]</span></code></pre></div>
 <pre><code>## [1] 1 2 3 4</code></pre>
-<div class="sourceCode" id="cb243"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb243-1"><a href="data-and-programming.html#cb243-1" aria-hidden="true" tabindex="-1"></a>ex_list[<span class="fu">c</span>(<span class="st">&quot;e&quot;</span>, <span class="st">&quot;a&quot;</span>)]</span></code></pre></div>
+<div class="sourceCode" id="cb243"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb243-1"><a href="data-and-programming.html#cb243-1" tabindex="-1"></a>ex_list[<span class="fu">c</span>(<span class="st">&quot;e&quot;</span>, <span class="st">&quot;a&quot;</span>)]</span></code></pre></div>
 <pre><code>## $e
 ##      [,1] [,2] [,3] [,4] [,5]
 ## [1,]    1    0    0    0    0
@@ -1116,7 +1116,7 @@ <h3><span class="header-section-number">3.2.6</span> Lists<a href="data-and-prog
 ## 
 ## $a
 ## [1] 1 2 3 4</code></pre>
-<div class="sourceCode" id="cb245"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb245-1"><a href="data-and-programming.html#cb245-1" aria-hidden="true" tabindex="-1"></a>ex_list[<span class="st">&quot;e&quot;</span>]</span></code></pre></div>
+<div class="sourceCode" id="cb245"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb245-1"><a href="data-and-programming.html#cb245-1" tabindex="-1"></a>ex_list[<span class="st">&quot;e&quot;</span>]</span></code></pre></div>
 <pre><code>## $e
 ##      [,1] [,2] [,3] [,4] [,5]
 ## [1,]    1    0    0    0    0
@@ -1124,26 +1124,26 @@ <h3><span class="header-section-number">3.2.6</span> Lists<a href="data-and-prog
 ## [3,]    0    0    1    0    0
 ## [4,]    0    0    0    1    0
 ## [5,]    0    0    0    0    1</code></pre>
-<div class="sourceCode" id="cb247"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb247-1"><a href="data-and-programming.html#cb247-1" aria-hidden="true" tabindex="-1"></a>ex_list[[<span class="st">&quot;e&quot;</span>]]</span></code></pre></div>
+<div class="sourceCode" id="cb247"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb247-1"><a href="data-and-programming.html#cb247-1" tabindex="-1"></a>ex_list[[<span class="st">&quot;e&quot;</span>]]</span></code></pre></div>
 <pre><code>##      [,1] [,2] [,3] [,4] [,5]
 ## [1,]    1    0    0    0    0
 ## [2,]    0    1    0    0    0
 ## [3,]    0    0    1    0    0
 ## [4,]    0    0    0    1    0
 ## [5,]    0    0    0    0    1</code></pre>
-<div class="sourceCode" id="cb249"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb249-1"><a href="data-and-programming.html#cb249-1" aria-hidden="true" tabindex="-1"></a>ex_list<span class="sc">$</span>d</span></code></pre></div>
+<div class="sourceCode" id="cb249"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb249-1"><a href="data-and-programming.html#cb249-1" tabindex="-1"></a>ex_list<span class="sc">$</span>d</span></code></pre></div>
 <pre><code>## function(arg = 42) {print(&quot;Hello World!&quot;)}</code></pre>
-<div class="sourceCode" id="cb251"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb251-1"><a href="data-and-programming.html#cb251-1" aria-hidden="true" tabindex="-1"></a>ex_list<span class="sc">$</span><span class="fu">d</span>(<span class="at">arg =</span> <span class="dv">1</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb251"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb251-1"><a href="data-and-programming.html#cb251-1" tabindex="-1"></a>ex_list<span class="sc">$</span><span class="fu">d</span>(<span class="at">arg =</span> <span class="dv">1</span>)</span></code></pre></div>
 <pre><code>## [1] &quot;Hello World!&quot;</code></pre>
 </div>
 <div id="data-frames" class="section level3 hasAnchor" number="3.2.7">
 <h3><span class="header-section-number">3.2.7</span> Data Frames<a href="data-and-programming.html#data-frames" class="anchor-section" aria-label="Anchor link to header"></a></h3>
 <p>We have previously seen vectors and matrices for storing data as we introduced <code>R</code>. We will now introduce a <strong>data frame</strong> which will be the most common way that we store and interact with data in this course.</p>
-<div class="sourceCode" id="cb253"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb253-1"><a href="data-and-programming.html#cb253-1" aria-hidden="true" tabindex="-1"></a>example_data <span class="ot">=</span> <span class="fu">data.frame</span>(<span class="at">x =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">3</span>, <span class="dv">5</span>, <span class="dv">7</span>, <span class="dv">9</span>, <span class="dv">1</span>, <span class="dv">3</span>, <span class="dv">5</span>, <span class="dv">7</span>, <span class="dv">9</span>),</span>
-<span id="cb253-2"><a href="data-and-programming.html#cb253-2" aria-hidden="true" tabindex="-1"></a>                          <span class="at">y =</span> <span class="fu">c</span>(<span class="fu">rep</span>(<span class="st">&quot;Hello&quot;</span>, <span class="dv">9</span>), <span class="st">&quot;Goodbye&quot;</span>),</span>
-<span id="cb253-3"><a href="data-and-programming.html#cb253-3" aria-hidden="true" tabindex="-1"></a>                          <span class="at">z =</span> <span class="fu">rep</span>(<span class="fu">c</span>(<span class="cn">TRUE</span>, <span class="cn">FALSE</span>), <span class="dv">5</span>))</span></code></pre></div>
+<div class="sourceCode" id="cb253"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb253-1"><a href="data-and-programming.html#cb253-1" tabindex="-1"></a>example_data <span class="ot">=</span> <span class="fu">data.frame</span>(<span class="at">x =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">3</span>, <span class="dv">5</span>, <span class="dv">7</span>, <span class="dv">9</span>, <span class="dv">1</span>, <span class="dv">3</span>, <span class="dv">5</span>, <span class="dv">7</span>, <span class="dv">9</span>),</span>
+<span id="cb253-2"><a href="data-and-programming.html#cb253-2" tabindex="-1"></a>                          <span class="at">y =</span> <span class="fu">c</span>(<span class="fu">rep</span>(<span class="st">&quot;Hello&quot;</span>, <span class="dv">9</span>), <span class="st">&quot;Goodbye&quot;</span>),</span>
+<span id="cb253-3"><a href="data-and-programming.html#cb253-3" tabindex="-1"></a>                          <span class="at">z =</span> <span class="fu">rep</span>(<span class="fu">c</span>(<span class="cn">TRUE</span>, <span class="cn">FALSE</span>), <span class="dv">5</span>))</span></code></pre></div>
 <p>Unlike a matrix, which can be thought of as a vector rearranged into rows and columns, a data frame is not required to have the same data type for each element. A data frame is a <strong>list</strong> of vectors. So, each vector must contain the same data type, but the different vectors can store different data types.</p>
-<div class="sourceCode" id="cb254"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb254-1"><a href="data-and-programming.html#cb254-1" aria-hidden="true" tabindex="-1"></a>example_data</span></code></pre></div>
+<div class="sourceCode" id="cb254"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb254-1"><a href="data-and-programming.html#cb254-1" tabindex="-1"></a>example_data</span></code></pre></div>
 <pre><code>##    x       y     z
 ## 1  1   Hello  TRUE
 ## 2  3   Hello FALSE
@@ -1156,29 +1156,29 @@ <h3><span class="header-section-number">3.2.7</span> Data Frames<a href="data-an
 ## 9  7   Hello  TRUE
 ## 10 9 Goodbye FALSE</code></pre>
 <p>Unlike a list which has more flexibility, the elements of a data frame must all be vectors, and have the same length.</p>
-<div class="sourceCode" id="cb256"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb256-1"><a href="data-and-programming.html#cb256-1" aria-hidden="true" tabindex="-1"></a>example_data<span class="sc">$</span>x</span></code></pre></div>
+<div class="sourceCode" id="cb256"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb256-1"><a href="data-and-programming.html#cb256-1" tabindex="-1"></a>example_data<span class="sc">$</span>x</span></code></pre></div>
 <pre><code>##  [1] 1 3 5 7 9 1 3 5 7 9</code></pre>
-<div class="sourceCode" id="cb258"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb258-1"><a href="data-and-programming.html#cb258-1" aria-hidden="true" tabindex="-1"></a><span class="fu">all.equal</span>(<span class="fu">length</span>(example_data<span class="sc">$</span>x),</span>
-<span id="cb258-2"><a href="data-and-programming.html#cb258-2" aria-hidden="true" tabindex="-1"></a>          <span class="fu">length</span>(example_data<span class="sc">$</span>y),</span>
-<span id="cb258-3"><a href="data-and-programming.html#cb258-3" aria-hidden="true" tabindex="-1"></a>          <span class="fu">length</span>(example_data<span class="sc">$</span>z))</span></code></pre></div>
+<div class="sourceCode" id="cb258"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb258-1"><a href="data-and-programming.html#cb258-1" tabindex="-1"></a><span class="fu">all.equal</span>(<span class="fu">length</span>(example_data<span class="sc">$</span>x),</span>
+<span id="cb258-2"><a href="data-and-programming.html#cb258-2" tabindex="-1"></a>          <span class="fu">length</span>(example_data<span class="sc">$</span>y),</span>
+<span id="cb258-3"><a href="data-and-programming.html#cb258-3" tabindex="-1"></a>          <span class="fu">length</span>(example_data<span class="sc">$</span>z))</span></code></pre></div>
 <pre><code>## [1] TRUE</code></pre>
-<div class="sourceCode" id="cb260"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb260-1"><a href="data-and-programming.html#cb260-1" aria-hidden="true" tabindex="-1"></a><span class="fu">str</span>(example_data)</span></code></pre></div>
+<div class="sourceCode" id="cb260"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb260-1"><a href="data-and-programming.html#cb260-1" tabindex="-1"></a><span class="fu">str</span>(example_data)</span></code></pre></div>
 <pre><code>## &#39;data.frame&#39;:    10 obs. of  3 variables:
 ##  $ x: num  1 3 5 7 9 1 3 5 7 9
 ##  $ y: chr  &quot;Hello&quot; &quot;Hello&quot; &quot;Hello&quot; &quot;Hello&quot; ...
 ##  $ z: logi  TRUE FALSE TRUE FALSE TRUE FALSE ...</code></pre>
-<div class="sourceCode" id="cb262"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb262-1"><a href="data-and-programming.html#cb262-1" aria-hidden="true" tabindex="-1"></a><span class="fu">nrow</span>(example_data)</span></code></pre></div>
+<div class="sourceCode" id="cb262"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb262-1"><a href="data-and-programming.html#cb262-1" tabindex="-1"></a><span class="fu">nrow</span>(example_data)</span></code></pre></div>
 <pre><code>## [1] 10</code></pre>
-<div class="sourceCode" id="cb264"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb264-1"><a href="data-and-programming.html#cb264-1" aria-hidden="true" tabindex="-1"></a><span class="fu">ncol</span>(example_data)</span></code></pre></div>
+<div class="sourceCode" id="cb264"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb264-1"><a href="data-and-programming.html#cb264-1" tabindex="-1"></a><span class="fu">ncol</span>(example_data)</span></code></pre></div>
 <pre><code>## [1] 3</code></pre>
-<div class="sourceCode" id="cb266"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb266-1"><a href="data-and-programming.html#cb266-1" aria-hidden="true" tabindex="-1"></a><span class="fu">dim</span>(example_data)</span></code></pre></div>
+<div class="sourceCode" id="cb266"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb266-1"><a href="data-and-programming.html#cb266-1" tabindex="-1"></a><span class="fu">dim</span>(example_data)</span></code></pre></div>
 <pre><code>## [1] 10  3</code></pre>
 <p>The <code>data.frame()</code> function above is one way to create a data frame. We can also import data from various file types into <code>R</code>, as well as use data stored in packages.</p>
 <p><a href="data/example-data.csv">The example data above can also be found here as a .csv file.</a> To read this data into <code>R</code>, we would use the <code>read_csv()</code> function from the <code>readr</code> package. Note that <code>R</code> has a built-in function <code>read.csv()</code> that operates very similarly. The <code>readr</code> function <code>read_csv()</code> has a number of advantages. For example, it is much faster reading larger data. <a href="https://cran.r-project.org/web/packages/tibble/vignettes/tibble.html" target="_blank">It also uses the <code>tibble</code> package to read the data as a tibble.</a></p>
-<div class="sourceCode" id="cb268"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb268-1"><a href="data-and-programming.html#cb268-1" aria-hidden="true" tabindex="-1"></a><span class="fu">library</span>(readr)</span>
-<span id="cb268-2"><a href="data-and-programming.html#cb268-2" aria-hidden="true" tabindex="-1"></a>example_data_from_csv <span class="ot">=</span> <span class="fu">read_csv</span>(<span class="st">&quot;data/example-data.csv&quot;</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb268"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb268-1"><a href="data-and-programming.html#cb268-1" tabindex="-1"></a><span class="fu">library</span>(readr)</span>
+<span id="cb268-2"><a href="data-and-programming.html#cb268-2" tabindex="-1"></a>example_data_from_csv <span class="ot">=</span> <span class="fu">read_csv</span>(<span class="st">&quot;data/example-data.csv&quot;</span>)</span></code></pre></div>
 <p>This particular line of code assumes that the file <code>example_data.csv</code> exists in a folder called <code>data</code> in your current working directory.</p>
-<div class="sourceCode" id="cb269"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb269-1"><a href="data-and-programming.html#cb269-1" aria-hidden="true" tabindex="-1"></a>example_data_from_csv</span></code></pre></div>
+<div class="sourceCode" id="cb269"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb269-1"><a href="data-and-programming.html#cb269-1" tabindex="-1"></a>example_data_from_csv</span></code></pre></div>
 <pre><code>## # A tibble: 10 × 3
 ##        x y       z    
 ##    &lt;dbl&gt; &lt;chr&gt;   &lt;lgl&gt;
@@ -1194,9 +1194,9 @@ <h3><span class="header-section-number">3.2.7</span> Data Frames<a href="data-an
 ## 10     9 Goodbye FALSE</code></pre>
 <p>A tibble is simply a data frame that prints with sanity. Notice in the output above that we are given additional information such as dimension and variable type.</p>
 <p>The <code>as_tibble()</code> function can be used to coerce a regular data frame to a tibble.</p>
-<div class="sourceCode" id="cb271"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb271-1"><a href="data-and-programming.html#cb271-1" aria-hidden="true" tabindex="-1"></a><span class="fu">library</span>(tibble)</span>
-<span id="cb271-2"><a href="data-and-programming.html#cb271-2" aria-hidden="true" tabindex="-1"></a>example_data <span class="ot">=</span> <span class="fu">as_tibble</span>(example_data)</span>
-<span id="cb271-3"><a href="data-and-programming.html#cb271-3" aria-hidden="true" tabindex="-1"></a>example_data</span></code></pre></div>
+<div class="sourceCode" id="cb271"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb271-1"><a href="data-and-programming.html#cb271-1" tabindex="-1"></a><span class="fu">library</span>(tibble)</span>
+<span id="cb271-2"><a href="data-and-programming.html#cb271-2" tabindex="-1"></a>example_data <span class="ot">=</span> <span class="fu">as_tibble</span>(example_data)</span>
+<span id="cb271-3"><a href="data-and-programming.html#cb271-3" tabindex="-1"></a>example_data</span></code></pre></div>
 <pre><code>## # A tibble: 10 × 3
 ##        x y       z    
 ##    &lt;dbl&gt; &lt;chr&gt;   &lt;lgl&gt;
@@ -1212,7 +1212,7 @@ <h3><span class="header-section-number">3.2.7</span> Data Frames<a href="data-an
 ## 10     9 Goodbye FALSE</code></pre>
 <p>Alternatively, we could use the “Import Dataset” feature in RStudio which can be found in the environment window. (By default, the top-right pane of RStudio.) Once completed, this process will automatically generate the code to import a file. The resulting code will be shown in the console window. In recent versions of RStudio, <code>read_csv()</code> is used by default, thus reading in a tibble.</p>
 <p>Earlier we looked at installing packages, in particular the <code>ggplot2</code> package. (A package for visualization. While not necessary for this course, it is quickly growing in popularity.)</p>
-<div class="sourceCode" id="cb273"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb273-1"><a href="data-and-programming.html#cb273-1" aria-hidden="true" tabindex="-1"></a><span class="fu">library</span>(ggplot2)</span></code></pre></div>
+<div class="sourceCode" id="cb273"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb273-1"><a href="data-and-programming.html#cb273-1" tabindex="-1"></a><span class="fu">library</span>(ggplot2)</span></code></pre></div>
 <p>Inside the <code>ggplot2</code> package is a dataset called <code>mpg</code>. By loading the package using the <code>library()</code> function, we can now access <code>mpg</code>.</p>
 <p>When using data from inside a package, there are three things we would generally like to do:</p>
 <ul>
@@ -1221,7 +1221,7 @@ <h3><span class="header-section-number">3.2.7</span> Data Frames<a href="data-an
 <li>Visualize the data.</li>
 </ul>
 <p>To look at the data, we have two useful commands: <code>head()</code> and <code>str()</code>.</p>
-<div class="sourceCode" id="cb274"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb274-1"><a href="data-and-programming.html#cb274-1" aria-hidden="true" tabindex="-1"></a><span class="fu">head</span>(mpg, <span class="at">n =</span> <span class="dv">10</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb274"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb274-1"><a href="data-and-programming.html#cb274-1" tabindex="-1"></a><span class="fu">head</span>(mpg, <span class="at">n =</span> <span class="dv">10</span>)</span></code></pre></div>
 <pre><code>## # A tibble: 10 × 11
 ##    manufacturer model      displ  year   cyl trans drv     cty   hwy fl    class
 ##    &lt;chr&gt;        &lt;chr&gt;      &lt;dbl&gt; &lt;int&gt; &lt;int&gt; &lt;chr&gt; &lt;chr&gt; &lt;int&gt; &lt;int&gt; &lt;chr&gt; &lt;chr&gt;
@@ -1236,7 +1236,7 @@ <h3><span class="header-section-number">3.2.7</span> Data Frames<a href="data-an
 ##  9 audi         a4 quattro   1.8  1999     4 auto… 4        16    25 p     comp…
 ## 10 audi         a4 quattro   2    2008     4 manu… 4        20    28 p     comp…</code></pre>
 <p>The function <code>head()</code> will display the first <code>n</code> observations of the data frame. The <code>head()</code> function was more useful before tibbles. Notice that <code>mpg</code> is a tibble already, so the output from <code>head()</code> indicates there are only <code>10</code> observations. Note that this applies to <code>head(mpg, n = 10)</code> and not <code>mpg</code> itself. Also note that tibbles print a limited number of rows and columns by default. The last line of the printed output indicates which rows and columns were omitted.</p>
-<div class="sourceCode" id="cb276"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb276-1"><a href="data-and-programming.html#cb276-1" aria-hidden="true" tabindex="-1"></a>mpg</span></code></pre></div>
+<div class="sourceCode" id="cb276"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb276-1"><a href="data-and-programming.html#cb276-1" tabindex="-1"></a>mpg</span></code></pre></div>
 <pre><code>## # A tibble: 234 × 11
 ##    manufacturer model      displ  year   cyl trans drv     cty   hwy fl    class
 ##    &lt;chr&gt;        &lt;chr&gt;      &lt;dbl&gt; &lt;int&gt; &lt;int&gt; &lt;chr&gt; &lt;chr&gt; &lt;int&gt; &lt;int&gt; &lt;chr&gt; &lt;chr&gt;
@@ -1252,7 +1252,7 @@ <h3><span class="header-section-number">3.2.7</span> Data Frames<a href="data-an
 ## 10 audi         a4 quattro   2    2008     4 manu… 4        20    28 p     comp…
 ## # ℹ 224 more rows</code></pre>
 <p>The function <code>str()</code> will display the “structure” of the data frame. It will display the number of <strong>observations</strong> and <strong>variables</strong>, list the variables, give the type of each variable, and show some elements of each variable. This information can also be found in the “Environment” window in RStudio.</p>
-<div class="sourceCode" id="cb278"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb278-1"><a href="data-and-programming.html#cb278-1" aria-hidden="true" tabindex="-1"></a><span class="fu">str</span>(mpg)</span></code></pre></div>
+<div class="sourceCode" id="cb278"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb278-1"><a href="data-and-programming.html#cb278-1" tabindex="-1"></a><span class="fu">str</span>(mpg)</span></code></pre></div>
 <pre><code>## tibble [234 × 11] (S3: tbl_df/tbl/data.frame)
 ##  $ manufacturer: chr [1:234] &quot;audi&quot; &quot;audi&quot; &quot;audi&quot; &quot;audi&quot; ...
 ##  $ model       : chr [1:234] &quot;a4&quot; &quot;a4&quot; &quot;a4&quot; &quot;a4&quot; ...
@@ -1268,14 +1268,14 @@ <h3><span class="header-section-number">3.2.7</span> Data Frames<a href="data-an
 <p>It is important to note that while matrices have rows and columns, data frames (tibbles) instead have observations and variables. When displayed in the console or viewer, each row is an observation and each column is a variable. However generally speaking, their order does not matter, it is simply a side-effect of how the data was entered or stored.</p>
 <p>In this dataset an observation is for a particular model-year of a car, and the variables describe attributes of the car, for example its highway fuel efficiency.</p>
 <p>To understand more about the data set, we use the <code>?</code> operator to pull up the documentation for the data.</p>
-<div class="sourceCode" id="cb280"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb280-1"><a href="data-and-programming.html#cb280-1" aria-hidden="true" tabindex="-1"></a>?mpg</span></code></pre></div>
+<div class="sourceCode" id="cb280"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb280-1"><a href="data-and-programming.html#cb280-1" tabindex="-1"></a>?mpg</span></code></pre></div>
 <p><code>R</code> has a number of functions for quickly working with and extracting basic information from data frames. To quickly obtain a vector of the variable names, we use the <code>names()</code> function.</p>
-<div class="sourceCode" id="cb281"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb281-1"><a href="data-and-programming.html#cb281-1" aria-hidden="true" tabindex="-1"></a><span class="fu">names</span>(mpg)</span></code></pre></div>
+<div class="sourceCode" id="cb281"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb281-1"><a href="data-and-programming.html#cb281-1" tabindex="-1"></a><span class="fu">names</span>(mpg)</span></code></pre></div>
 <pre><code>##  [1] &quot;manufacturer&quot; &quot;model&quot;        &quot;displ&quot;        &quot;year&quot;         &quot;cyl&quot;         
 ##  [6] &quot;trans&quot;        &quot;drv&quot;          &quot;cty&quot;          &quot;hwy&quot;          &quot;fl&quot;          
 ## [11] &quot;class&quot;</code></pre>
 <p>To access one of the variables <strong>as a vector</strong>, we use the <code>$</code> operator.</p>
-<div class="sourceCode" id="cb283"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb283-1"><a href="data-and-programming.html#cb283-1" aria-hidden="true" tabindex="-1"></a>mpg<span class="sc">$</span>year</span></code></pre></div>
+<div class="sourceCode" id="cb283"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb283-1"><a href="data-and-programming.html#cb283-1" tabindex="-1"></a>mpg<span class="sc">$</span>year</span></code></pre></div>
 <pre><code>##   [1] 1999 1999 2008 2008 1999 1999 2008 1999 1999 2008 2008 1999 1999 2008 2008
 ##  [16] 1999 2008 2008 2008 2008 2008 1999 2008 1999 1999 2008 2008 2008 2008 2008
 ##  [31] 1999 1999 1999 2008 1999 2008 2008 1999 1999 1999 1999 2008 2008 2008 1999
@@ -1292,7 +1292,7 @@ <h3><span class="header-section-number">3.2.7</span> Data Frames<a href="data-an
 ## [196] 1999 2008 2008 1999 2008 1999 1999 2008 1999 1999 2008 2008 1999 1999 2008
 ## [211] 2008 1999 1999 1999 1999 2008 2008 2008 2008 1999 1999 1999 1999 1999 1999
 ## [226] 2008 2008 1999 1999 2008 2008 1999 1999 2008</code></pre>
-<div class="sourceCode" id="cb285"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb285-1"><a href="data-and-programming.html#cb285-1" aria-hidden="true" tabindex="-1"></a>mpg<span class="sc">$</span>hwy</span></code></pre></div>
+<div class="sourceCode" id="cb285"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb285-1"><a href="data-and-programming.html#cb285-1" tabindex="-1"></a>mpg<span class="sc">$</span>hwy</span></code></pre></div>
 <pre><code>##   [1] 29 29 31 30 26 26 27 26 25 28 27 25 25 25 25 24 25 23 20 15 20 17 17 26 23
 ##  [26] 26 25 24 19 14 15 17 27 30 26 29 26 24 24 22 22 24 24 17 22 21 23 23 19 18
 ##  [51] 17 17 19 19 12 17 15 17 17 12 17 16 18 15 16 12 17 17 16 12 15 16 17 15 17
@@ -1304,15 +1304,15 @@ <h3><span class="header-section-number">3.2.7</span> Data Frames<a href="data-an
 ## [201] 20 20 22 17 19 18 20 29 26 29 29 24 44 29 26 29 29 29 29 23 24 44 41 29 26
 ## [226] 28 29 29 29 28 29 26 26 26</code></pre>
 <p>We can use the <code>dim()</code>, <code>nrow()</code> and <code>ncol()</code> functions to obtain information about the dimension of the data frame.</p>
-<div class="sourceCode" id="cb287"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb287-1"><a href="data-and-programming.html#cb287-1" aria-hidden="true" tabindex="-1"></a><span class="fu">dim</span>(mpg)</span></code></pre></div>
+<div class="sourceCode" id="cb287"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb287-1"><a href="data-and-programming.html#cb287-1" tabindex="-1"></a><span class="fu">dim</span>(mpg)</span></code></pre></div>
 <pre><code>## [1] 234  11</code></pre>
-<div class="sourceCode" id="cb289"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb289-1"><a href="data-and-programming.html#cb289-1" aria-hidden="true" tabindex="-1"></a><span class="fu">nrow</span>(mpg)</span></code></pre></div>
+<div class="sourceCode" id="cb289"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb289-1"><a href="data-and-programming.html#cb289-1" tabindex="-1"></a><span class="fu">nrow</span>(mpg)</span></code></pre></div>
 <pre><code>## [1] 234</code></pre>
-<div class="sourceCode" id="cb291"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb291-1"><a href="data-and-programming.html#cb291-1" aria-hidden="true" tabindex="-1"></a><span class="fu">ncol</span>(mpg)</span></code></pre></div>
+<div class="sourceCode" id="cb291"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb291-1"><a href="data-and-programming.html#cb291-1" tabindex="-1"></a><span class="fu">ncol</span>(mpg)</span></code></pre></div>
 <pre><code>## [1] 11</code></pre>
 <p>Here <code>nrow()</code> is also the number of observations, which in most cases is the <em>sample size</em>.</p>
 <p>Subsetting data frames can work much like subsetting matrices using square brackets, <code>[,]</code>. Here, we find fuel efficient vehicles earning over 35 miles per gallon and only display <code>manufacturer</code>, <code>model</code> and <code>year</code>.</p>
-<div class="sourceCode" id="cb293"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb293-1"><a href="data-and-programming.html#cb293-1" aria-hidden="true" tabindex="-1"></a>mpg[mpg<span class="sc">$</span>hwy <span class="sc">&gt;</span> <span class="dv">35</span>, <span class="fu">c</span>(<span class="st">&quot;manufacturer&quot;</span>, <span class="st">&quot;model&quot;</span>, <span class="st">&quot;year&quot;</span>)]</span></code></pre></div>
+<div class="sourceCode" id="cb293"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb293-1"><a href="data-and-programming.html#cb293-1" tabindex="-1"></a>mpg[mpg<span class="sc">$</span>hwy <span class="sc">&gt;</span> <span class="dv">35</span>, <span class="fu">c</span>(<span class="st">&quot;manufacturer&quot;</span>, <span class="st">&quot;model&quot;</span>, <span class="st">&quot;year&quot;</span>)]</span></code></pre></div>
 <pre><code>## # A tibble: 6 × 3
 ##   manufacturer model       year
 ##   &lt;chr&gt;        &lt;chr&gt;      &lt;int&gt;
@@ -1323,10 +1323,10 @@ <h3><span class="header-section-number">3.2.7</span> Data Frames<a href="data-an
 ## 5 volkswagen   new beetle  1999
 ## 6 volkswagen   new beetle  1999</code></pre>
 <p>An alternative would be to use the <code>subset()</code> function, which has a much more readable syntax.</p>
-<div class="sourceCode" id="cb295"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb295-1"><a href="data-and-programming.html#cb295-1" aria-hidden="true" tabindex="-1"></a><span class="fu">subset</span>(mpg, <span class="at">subset =</span> hwy <span class="sc">&gt;</span> <span class="dv">35</span>, <span class="at">select =</span> <span class="fu">c</span>(<span class="st">&quot;manufacturer&quot;</span>, <span class="st">&quot;model&quot;</span>, <span class="st">&quot;year&quot;</span>))</span></code></pre></div>
+<div class="sourceCode" id="cb295"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb295-1"><a href="data-and-programming.html#cb295-1" tabindex="-1"></a><span class="fu">subset</span>(mpg, <span class="at">subset =</span> hwy <span class="sc">&gt;</span> <span class="dv">35</span>, <span class="at">select =</span> <span class="fu">c</span>(<span class="st">&quot;manufacturer&quot;</span>, <span class="st">&quot;model&quot;</span>, <span class="st">&quot;year&quot;</span>))</span></code></pre></div>
 <p>Lastly, we could use the <code>filter</code> and <code>select</code> functions from the <code>dplyr</code> package which introduces the <code>%&gt;%</code> operator from the <code>magrittr</code> package. This is not necessary for this course, however the <code>dplyr</code> package is something you should be aware of as it is becoming a popular tool in the <code>R</code> world.</p>
-<div class="sourceCode" id="cb296"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb296-1"><a href="data-and-programming.html#cb296-1" aria-hidden="true" tabindex="-1"></a><span class="fu">library</span>(dplyr)</span>
-<span id="cb296-2"><a href="data-and-programming.html#cb296-2" aria-hidden="true" tabindex="-1"></a>mpg <span class="sc">%&gt;%</span> <span class="fu">filter</span>(hwy <span class="sc">&gt;</span> <span class="dv">35</span>) <span class="sc">%&gt;%</span> <span class="fu">select</span>(manufacturer, model, year)</span></code></pre></div>
+<div class="sourceCode" id="cb296"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb296-1"><a href="data-and-programming.html#cb296-1" tabindex="-1"></a><span class="fu">library</span>(dplyr)</span>
+<span id="cb296-2"><a href="data-and-programming.html#cb296-2" tabindex="-1"></a>mpg <span class="sc">%&gt;%</span> <span class="fu">filter</span>(hwy <span class="sc">&gt;</span> <span class="dv">35</span>) <span class="sc">%&gt;%</span> <span class="fu">select</span>(manufacturer, model, year)</span></code></pre></div>
 <p>All three approaches produce the same results. Which you use will be largely based on a given situation as well as user preference.</p>
 <p>When subsetting a data frame, be aware of what is being returned, as sometimes it may be a vector instead of a data frame. Also note that there are differences between subsetting a data frame and a tibble. A data frame operates more like a matrix where it is possible to reduce the subset to a vector. A tibble operates more like a list where it always subsets to another tibble.</p>
 </div>
@@ -1336,49 +1336,49 @@ <h2><span class="header-section-number">3.3</span> Programming Basics<a href="da
 <div id="control-flow" class="section level3 hasAnchor" number="3.3.1">
 <h3><span class="header-section-number">3.3.1</span> Control Flow<a href="data-and-programming.html#control-flow" class="anchor-section" aria-label="Anchor link to header"></a></h3>
 <p>In <code>R</code>, the if/else syntax is:</p>
-<div class="sourceCode" id="cb297"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb297-1"><a href="data-and-programming.html#cb297-1" aria-hidden="true" tabindex="-1"></a><span class="cf">if</span> (...) {</span>
-<span id="cb297-2"><a href="data-and-programming.html#cb297-2" aria-hidden="true" tabindex="-1"></a>  some R code</span>
-<span id="cb297-3"><a href="data-and-programming.html#cb297-3" aria-hidden="true" tabindex="-1"></a>} <span class="cf">else</span> {</span>
-<span id="cb297-4"><a href="data-and-programming.html#cb297-4" aria-hidden="true" tabindex="-1"></a>  more R code</span>
-<span id="cb297-5"><a href="data-and-programming.html#cb297-5" aria-hidden="true" tabindex="-1"></a>}</span></code></pre></div>
+<div class="sourceCode" id="cb297"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb297-1"><a href="data-and-programming.html#cb297-1" tabindex="-1"></a><span class="cf">if</span> (...) {</span>
+<span id="cb297-2"><a href="data-and-programming.html#cb297-2" tabindex="-1"></a>  some R code</span>
+<span id="cb297-3"><a href="data-and-programming.html#cb297-3" tabindex="-1"></a>} <span class="cf">else</span> {</span>
+<span id="cb297-4"><a href="data-and-programming.html#cb297-4" tabindex="-1"></a>  more R code</span>
+<span id="cb297-5"><a href="data-and-programming.html#cb297-5" tabindex="-1"></a>}</span></code></pre></div>
 <p>For example,</p>
-<div class="sourceCode" id="cb298"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb298-1"><a href="data-and-programming.html#cb298-1" aria-hidden="true" tabindex="-1"></a>x <span class="ot">=</span> <span class="dv">1</span></span>
-<span id="cb298-2"><a href="data-and-programming.html#cb298-2" aria-hidden="true" tabindex="-1"></a>y <span class="ot">=</span> <span class="dv">3</span></span>
-<span id="cb298-3"><a href="data-and-programming.html#cb298-3" aria-hidden="true" tabindex="-1"></a><span class="cf">if</span> (x <span class="sc">&gt;</span> y) {</span>
-<span id="cb298-4"><a href="data-and-programming.html#cb298-4" aria-hidden="true" tabindex="-1"></a>  z <span class="ot">=</span> x <span class="sc">*</span> y</span>
-<span id="cb298-5"><a href="data-and-programming.html#cb298-5" aria-hidden="true" tabindex="-1"></a>  <span class="fu">print</span>(<span class="st">&quot;x is larger than y&quot;</span>)</span>
-<span id="cb298-6"><a href="data-and-programming.html#cb298-6" aria-hidden="true" tabindex="-1"></a>} <span class="cf">else</span> {</span>
-<span id="cb298-7"><a href="data-and-programming.html#cb298-7" aria-hidden="true" tabindex="-1"></a>  z <span class="ot">=</span> x <span class="sc">+</span> <span class="dv">5</span> <span class="sc">*</span> y</span>
-<span id="cb298-8"><a href="data-and-programming.html#cb298-8" aria-hidden="true" tabindex="-1"></a>  <span class="fu">print</span>(<span class="st">&quot;x is less than or equal to y&quot;</span>)</span>
-<span id="cb298-9"><a href="data-and-programming.html#cb298-9" aria-hidden="true" tabindex="-1"></a>}</span></code></pre></div>
+<div class="sourceCode" id="cb298"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb298-1"><a href="data-and-programming.html#cb298-1" tabindex="-1"></a>x <span class="ot">=</span> <span class="dv">1</span></span>
+<span id="cb298-2"><a href="data-and-programming.html#cb298-2" tabindex="-1"></a>y <span class="ot">=</span> <span class="dv">3</span></span>
+<span id="cb298-3"><a href="data-and-programming.html#cb298-3" tabindex="-1"></a><span class="cf">if</span> (x <span class="sc">&gt;</span> y) {</span>
+<span id="cb298-4"><a href="data-and-programming.html#cb298-4" tabindex="-1"></a>  z <span class="ot">=</span> x <span class="sc">*</span> y</span>
+<span id="cb298-5"><a href="data-and-programming.html#cb298-5" tabindex="-1"></a>  <span class="fu">print</span>(<span class="st">&quot;x is larger than y&quot;</span>)</span>
+<span id="cb298-6"><a href="data-and-programming.html#cb298-6" tabindex="-1"></a>} <span class="cf">else</span> {</span>
+<span id="cb298-7"><a href="data-and-programming.html#cb298-7" tabindex="-1"></a>  z <span class="ot">=</span> x <span class="sc">+</span> <span class="dv">5</span> <span class="sc">*</span> y</span>
+<span id="cb298-8"><a href="data-and-programming.html#cb298-8" tabindex="-1"></a>  <span class="fu">print</span>(<span class="st">&quot;x is less than or equal to y&quot;</span>)</span>
+<span id="cb298-9"><a href="data-and-programming.html#cb298-9" tabindex="-1"></a>}</span></code></pre></div>
 <pre><code>## [1] &quot;x is less than or equal to y&quot;</code></pre>
-<div class="sourceCode" id="cb300"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb300-1"><a href="data-and-programming.html#cb300-1" aria-hidden="true" tabindex="-1"></a>z</span></code></pre></div>
+<div class="sourceCode" id="cb300"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb300-1"><a href="data-and-programming.html#cb300-1" tabindex="-1"></a>z</span></code></pre></div>
 <pre><code>## [1] 16</code></pre>
 <p><code>R</code> also has a special function <code>ifelse()</code> which is very useful. It returns one of two specified values based on a conditional statement.</p>
-<div class="sourceCode" id="cb302"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb302-1"><a href="data-and-programming.html#cb302-1" aria-hidden="true" tabindex="-1"></a><span class="fu">ifelse</span>(<span class="dv">4</span> <span class="sc">&gt;</span> <span class="dv">3</span>, <span class="dv">1</span>, <span class="dv">0</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb302"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb302-1"><a href="data-and-programming.html#cb302-1" tabindex="-1"></a><span class="fu">ifelse</span>(<span class="dv">4</span> <span class="sc">&gt;</span> <span class="dv">3</span>, <span class="dv">1</span>, <span class="dv">0</span>)</span></code></pre></div>
 <pre><code>## [1] 1</code></pre>
 <p>The real power of <code>ifelse()</code> comes from its ability to be applied to vectors.</p>
-<div class="sourceCode" id="cb304"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb304-1"><a href="data-and-programming.html#cb304-1" aria-hidden="true" tabindex="-1"></a>fib <span class="ot">=</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">1</span>, <span class="dv">2</span>, <span class="dv">3</span>, <span class="dv">5</span>, <span class="dv">8</span>, <span class="dv">13</span>, <span class="dv">21</span>)</span>
-<span id="cb304-2"><a href="data-and-programming.html#cb304-2" aria-hidden="true" tabindex="-1"></a><span class="fu">ifelse</span>(fib <span class="sc">&gt;</span> <span class="dv">6</span>, <span class="st">&quot;Foo&quot;</span>, <span class="st">&quot;Bar&quot;</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb304"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb304-1"><a href="data-and-programming.html#cb304-1" tabindex="-1"></a>fib <span class="ot">=</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">1</span>, <span class="dv">2</span>, <span class="dv">3</span>, <span class="dv">5</span>, <span class="dv">8</span>, <span class="dv">13</span>, <span class="dv">21</span>)</span>
+<span id="cb304-2"><a href="data-and-programming.html#cb304-2" tabindex="-1"></a><span class="fu">ifelse</span>(fib <span class="sc">&gt;</span> <span class="dv">6</span>, <span class="st">&quot;Foo&quot;</span>, <span class="st">&quot;Bar&quot;</span>)</span></code></pre></div>
 <pre><code>## [1] &quot;Bar&quot; &quot;Bar&quot; &quot;Bar&quot; &quot;Bar&quot; &quot;Bar&quot; &quot;Foo&quot; &quot;Foo&quot; &quot;Foo&quot;</code></pre>
 <p>Now a <code>for</code> loop example,</p>
-<div class="sourceCode" id="cb306"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb306-1"><a href="data-and-programming.html#cb306-1" aria-hidden="true" tabindex="-1"></a>x <span class="ot">=</span> <span class="dv">11</span><span class="sc">:</span><span class="dv">15</span></span>
-<span id="cb306-2"><a href="data-and-programming.html#cb306-2" aria-hidden="true" tabindex="-1"></a><span class="cf">for</span> (i <span class="cf">in</span> <span class="dv">1</span><span class="sc">:</span><span class="dv">5</span>) {</span>
-<span id="cb306-3"><a href="data-and-programming.html#cb306-3" aria-hidden="true" tabindex="-1"></a>  x[i] <span class="ot">=</span> x[i] <span class="sc">*</span> <span class="dv">2</span></span>
-<span id="cb306-4"><a href="data-and-programming.html#cb306-4" aria-hidden="true" tabindex="-1"></a>}</span>
-<span id="cb306-5"><a href="data-and-programming.html#cb306-5" aria-hidden="true" tabindex="-1"></a></span>
-<span id="cb306-6"><a href="data-and-programming.html#cb306-6" aria-hidden="true" tabindex="-1"></a>x</span></code></pre></div>
+<div class="sourceCode" id="cb306"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb306-1"><a href="data-and-programming.html#cb306-1" tabindex="-1"></a>x <span class="ot">=</span> <span class="dv">11</span><span class="sc">:</span><span class="dv">15</span></span>
+<span id="cb306-2"><a href="data-and-programming.html#cb306-2" tabindex="-1"></a><span class="cf">for</span> (i <span class="cf">in</span> <span class="dv">1</span><span class="sc">:</span><span class="dv">5</span>) {</span>
+<span id="cb306-3"><a href="data-and-programming.html#cb306-3" tabindex="-1"></a>  x[i] <span class="ot">=</span> x[i] <span class="sc">*</span> <span class="dv">2</span></span>
+<span id="cb306-4"><a href="data-and-programming.html#cb306-4" tabindex="-1"></a>}</span>
+<span id="cb306-5"><a href="data-and-programming.html#cb306-5" tabindex="-1"></a></span>
+<span id="cb306-6"><a href="data-and-programming.html#cb306-6" tabindex="-1"></a>x</span></code></pre></div>
 <pre><code>## [1] 22 24 26 28 30</code></pre>
 <p>Note that this <code>for</code> loop is very normal in many programming languages, but not in <code>R</code>. In <code>R</code> we would not use a loop, instead we would simply use a vectorized operation.</p>
-<div class="sourceCode" id="cb308"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb308-1"><a href="data-and-programming.html#cb308-1" aria-hidden="true" tabindex="-1"></a>x <span class="ot">=</span> <span class="dv">11</span><span class="sc">:</span><span class="dv">15</span></span>
-<span id="cb308-2"><a href="data-and-programming.html#cb308-2" aria-hidden="true" tabindex="-1"></a>x <span class="ot">=</span> x <span class="sc">*</span> <span class="dv">2</span></span>
-<span id="cb308-3"><a href="data-and-programming.html#cb308-3" aria-hidden="true" tabindex="-1"></a>x</span></code></pre></div>
+<div class="sourceCode" id="cb308"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb308-1"><a href="data-and-programming.html#cb308-1" tabindex="-1"></a>x <span class="ot">=</span> <span class="dv">11</span><span class="sc">:</span><span class="dv">15</span></span>
+<span id="cb308-2"><a href="data-and-programming.html#cb308-2" tabindex="-1"></a>x <span class="ot">=</span> x <span class="sc">*</span> <span class="dv">2</span></span>
+<span id="cb308-3"><a href="data-and-programming.html#cb308-3" tabindex="-1"></a>x</span></code></pre></div>
 <pre><code>## [1] 22 24 26 28 30</code></pre>
 </div>
 <div id="functions" class="section level3 hasAnchor" number="3.3.2">
 <h3><span class="header-section-number">3.3.2</span> Functions<a href="data-and-programming.html#functions" class="anchor-section" aria-label="Anchor link to header"></a></h3>
 <p>So far we have been using functions, but haven’t actually discussed some of their details.</p>
-<div class="sourceCode" id="cb310"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb310-1"><a href="data-and-programming.html#cb310-1" aria-hidden="true" tabindex="-1"></a><span class="fu">function_name</span>(<span class="at">arg1 =</span> <span class="dv">10</span>, <span class="at">arg2 =</span> <span class="dv">20</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb310"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb310-1"><a href="data-and-programming.html#cb310-1" tabindex="-1"></a><span class="fu">function_name</span>(<span class="at">arg1 =</span> <span class="dv">10</span>, <span class="at">arg2 =</span> <span class="dv">20</span>)</span></code></pre></div>
 <p>To use a function, you simply type its name, followed by an open parenthesis, then specify values of its arguments, then finish with a closing parenthesis.</p>
 <p>An <strong>argument</strong> is a variable which is used in the body of the function. Specifying the values of the arguments is essentially providing the inputs to the function.</p>
 <p>We can also write our own functions in <code>R</code>. For example, we often like to “standardize” variables, that is, subtracting the sample mean, and dividing by the sample standard deviation.</p>
@@ -1391,42 +1391,42 @@ <h3><span class="header-section-number">3.3.2</span> Functions<a href="data-and-
 <li>Specify the arguments using <code>function()</code>.</li>
 <li>Write the body of the function within curly braces, <code>{}</code>.</li>
 </ul>
-<div class="sourceCode" id="cb311"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb311-1"><a href="data-and-programming.html#cb311-1" aria-hidden="true" tabindex="-1"></a>standardize <span class="ot">=</span> <span class="cf">function</span>(x) {</span>
-<span id="cb311-2"><a href="data-and-programming.html#cb311-2" aria-hidden="true" tabindex="-1"></a>  m <span class="ot">=</span> <span class="fu">mean</span>(x)</span>
-<span id="cb311-3"><a href="data-and-programming.html#cb311-3" aria-hidden="true" tabindex="-1"></a>  std <span class="ot">=</span> <span class="fu">sd</span>(x)</span>
-<span id="cb311-4"><a href="data-and-programming.html#cb311-4" aria-hidden="true" tabindex="-1"></a>  result <span class="ot">=</span> (x <span class="sc">-</span> m) <span class="sc">/</span> std</span>
-<span id="cb311-5"><a href="data-and-programming.html#cb311-5" aria-hidden="true" tabindex="-1"></a>  result</span>
-<span id="cb311-6"><a href="data-and-programming.html#cb311-6" aria-hidden="true" tabindex="-1"></a>}</span></code></pre></div>
+<div class="sourceCode" id="cb311"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb311-1"><a href="data-and-programming.html#cb311-1" tabindex="-1"></a>standardize <span class="ot">=</span> <span class="cf">function</span>(x) {</span>
+<span id="cb311-2"><a href="data-and-programming.html#cb311-2" tabindex="-1"></a>  m <span class="ot">=</span> <span class="fu">mean</span>(x)</span>
+<span id="cb311-3"><a href="data-and-programming.html#cb311-3" tabindex="-1"></a>  std <span class="ot">=</span> <span class="fu">sd</span>(x)</span>
+<span id="cb311-4"><a href="data-and-programming.html#cb311-4" tabindex="-1"></a>  result <span class="ot">=</span> (x <span class="sc">-</span> m) <span class="sc">/</span> std</span>
+<span id="cb311-5"><a href="data-and-programming.html#cb311-5" tabindex="-1"></a>  result</span>
+<span id="cb311-6"><a href="data-and-programming.html#cb311-6" tabindex="-1"></a>}</span></code></pre></div>
 <p>Here the name of the function is <code>standardize</code>, and the function has a single argument <code>x</code> which is used in the body of function. Note that the output of the final line of the body is what is returned by the function. In this case the function returns the vector stored in the variable <code>result</code>.</p>
 <p>To test our function, we will take a random sample of size <code>n = 10</code> from a normal distribution with a mean of <code>2</code> and a standard deviation of <code>5</code>.</p>
-<div class="sourceCode" id="cb312"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb312-1"><a href="data-and-programming.html#cb312-1" aria-hidden="true" tabindex="-1"></a>(<span class="at">test_sample =</span> <span class="fu">rnorm</span>(<span class="at">n =</span> <span class="dv">10</span>, <span class="at">mean =</span> <span class="dv">2</span>, <span class="at">sd =</span> <span class="dv">5</span>))</span></code></pre></div>
-<pre><code>##  [1] 14.1326555  3.6826188  0.4917934  2.7671252  0.2775572 -2.1484609
-##  [7] -2.6526105  7.5995334  3.8200937 -9.5182079</code></pre>
-<div class="sourceCode" id="cb314"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb314-1"><a href="data-and-programming.html#cb314-1" aria-hidden="true" tabindex="-1"></a><span class="fu">standardize</span>(<span class="at">x =</span> test_sample)</span></code></pre></div>
-<pre><code>##  [1]  1.9379339  0.2897899 -0.2134562  0.1454013 -0.2472448 -0.6298681
-##  [7] -0.7093808  0.9075522  0.3114719 -1.7921993</code></pre>
+<div class="sourceCode" id="cb312"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb312-1"><a href="data-and-programming.html#cb312-1" tabindex="-1"></a>(<span class="at">test_sample =</span> <span class="fu">rnorm</span>(<span class="at">n =</span> <span class="dv">10</span>, <span class="at">mean =</span> <span class="dv">2</span>, <span class="at">sd =</span> <span class="dv">5</span>))</span></code></pre></div>
+<pre><code>##  [1]  3.4663306 -1.8992181 -3.3143635 -3.4093843  5.3309762  0.7177388
+##  [7]  2.0023602  0.9305203  0.1863598  2.1499918</code></pre>
+<div class="sourceCode" id="cb314"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb314-1"><a href="data-and-programming.html#cb314-1" tabindex="-1"></a><span class="fu">standardize</span>(<span class="at">x =</span> test_sample)</span></code></pre></div>
+<pre><code>##  [1]  1.00144306 -0.88379048 -1.38101447 -1.41440089  1.65660299  0.03570074
+##  [7]  0.48706398  0.11046344 -0.15100402  0.53893564</code></pre>
 <p>This function could be written much more succinctly, simply performing all the operations on one line and immediately returning the result, without storing any of the intermediate results.</p>
-<div class="sourceCode" id="cb316"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb316-1"><a href="data-and-programming.html#cb316-1" aria-hidden="true" tabindex="-1"></a>standardize <span class="ot">=</span> <span class="cf">function</span>(x) {</span>
-<span id="cb316-2"><a href="data-and-programming.html#cb316-2" aria-hidden="true" tabindex="-1"></a>  (x <span class="sc">-</span> <span class="fu">mean</span>(x)) <span class="sc">/</span> <span class="fu">sd</span>(x)</span>
-<span id="cb316-3"><a href="data-and-programming.html#cb316-3" aria-hidden="true" tabindex="-1"></a>}</span></code></pre></div>
+<div class="sourceCode" id="cb316"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb316-1"><a href="data-and-programming.html#cb316-1" tabindex="-1"></a>standardize <span class="ot">=</span> <span class="cf">function</span>(x) {</span>
+<span id="cb316-2"><a href="data-and-programming.html#cb316-2" tabindex="-1"></a>  (x <span class="sc">-</span> <span class="fu">mean</span>(x)) <span class="sc">/</span> <span class="fu">sd</span>(x)</span>
+<span id="cb316-3"><a href="data-and-programming.html#cb316-3" tabindex="-1"></a>}</span></code></pre></div>
 <p>When specifying arguments, you can provide default arguments.</p>
-<div class="sourceCode" id="cb317"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb317-1"><a href="data-and-programming.html#cb317-1" aria-hidden="true" tabindex="-1"></a>power_of_num <span class="ot">=</span> <span class="cf">function</span>(num, <span class="at">power =</span> <span class="dv">2</span>) {</span>
-<span id="cb317-2"><a href="data-and-programming.html#cb317-2" aria-hidden="true" tabindex="-1"></a>  num <span class="sc">^</span> power</span>
-<span id="cb317-3"><a href="data-and-programming.html#cb317-3" aria-hidden="true" tabindex="-1"></a>}</span></code></pre></div>
+<div class="sourceCode" id="cb317"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb317-1"><a href="data-and-programming.html#cb317-1" tabindex="-1"></a>power_of_num <span class="ot">=</span> <span class="cf">function</span>(num, <span class="at">power =</span> <span class="dv">2</span>) {</span>
+<span id="cb317-2"><a href="data-and-programming.html#cb317-2" tabindex="-1"></a>  num <span class="sc">^</span> power</span>
+<span id="cb317-3"><a href="data-and-programming.html#cb317-3" tabindex="-1"></a>}</span></code></pre></div>
 <p>Let’s look at a number of ways that we could run this function to perform the operation <code>10^2</code> resulting in <code>100</code>.</p>
-<div class="sourceCode" id="cb318"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb318-1"><a href="data-and-programming.html#cb318-1" aria-hidden="true" tabindex="-1"></a><span class="fu">power_of_num</span>(<span class="dv">10</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb318"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb318-1"><a href="data-and-programming.html#cb318-1" tabindex="-1"></a><span class="fu">power_of_num</span>(<span class="dv">10</span>)</span></code></pre></div>
 <pre><code>## [1] 100</code></pre>
-<div class="sourceCode" id="cb320"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb320-1"><a href="data-and-programming.html#cb320-1" aria-hidden="true" tabindex="-1"></a><span class="fu">power_of_num</span>(<span class="dv">10</span>, <span class="dv">2</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb320"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb320-1"><a href="data-and-programming.html#cb320-1" tabindex="-1"></a><span class="fu">power_of_num</span>(<span class="dv">10</span>, <span class="dv">2</span>)</span></code></pre></div>
 <pre><code>## [1] 100</code></pre>
-<div class="sourceCode" id="cb322"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb322-1"><a href="data-and-programming.html#cb322-1" aria-hidden="true" tabindex="-1"></a><span class="fu">power_of_num</span>(<span class="at">num =</span> <span class="dv">10</span>, <span class="at">power =</span> <span class="dv">2</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb322"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb322-1"><a href="data-and-programming.html#cb322-1" tabindex="-1"></a><span class="fu">power_of_num</span>(<span class="at">num =</span> <span class="dv">10</span>, <span class="at">power =</span> <span class="dv">2</span>)</span></code></pre></div>
 <pre><code>## [1] 100</code></pre>
-<div class="sourceCode" id="cb324"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb324-1"><a href="data-and-programming.html#cb324-1" aria-hidden="true" tabindex="-1"></a><span class="fu">power_of_num</span>(<span class="at">power =</span> <span class="dv">2</span>, <span class="at">num =</span> <span class="dv">10</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb324"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb324-1"><a href="data-and-programming.html#cb324-1" tabindex="-1"></a><span class="fu">power_of_num</span>(<span class="at">power =</span> <span class="dv">2</span>, <span class="at">num =</span> <span class="dv">10</span>)</span></code></pre></div>
 <pre><code>## [1] 100</code></pre>
 <p>Note that without using the argument names, the order matters. The following code will not evaluate to the same output as the previous example.</p>
-<div class="sourceCode" id="cb326"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb326-1"><a href="data-and-programming.html#cb326-1" aria-hidden="true" tabindex="-1"></a><span class="fu">power_of_num</span>(<span class="dv">2</span>, <span class="dv">10</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb326"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb326-1"><a href="data-and-programming.html#cb326-1" tabindex="-1"></a><span class="fu">power_of_num</span>(<span class="dv">2</span>, <span class="dv">10</span>)</span></code></pre></div>
 <pre><code>## [1] 1024</code></pre>
 <p>Also, the following line of code would produce an error since arguments without a default value must be specified.</p>
-<div class="sourceCode" id="cb328"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb328-1"><a href="data-and-programming.html#cb328-1" aria-hidden="true" tabindex="-1"></a><span class="fu">power_of_num</span>(<span class="at">power =</span> <span class="dv">5</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb328"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb328-1"><a href="data-and-programming.html#cb328-1" tabindex="-1"></a><span class="fu">power_of_num</span>(<span class="at">power =</span> <span class="dv">5</span>)</span></code></pre></div>
 <p>To further illustrate a function with a default argument, we will write a function that calculates sample variance two ways.</p>
 <p>By default, it will calculate the unbiased estimate of <span class="math inline">\(\sigma^2\)</span>, which we will call <span class="math inline">\(s^2\)</span>.</p>
 <p><span class="math display">\[
@@ -1436,19 +1436,19 @@ <h3><span class="header-section-number">3.3.2</span> Functions<a href="data-and-
 <p><span class="math display">\[
 \hat{\sigma}^2 = \frac{1}{n}\sum_{i=1}^{n}(x - \bar{x})^2
 \]</span></p>
-<div class="sourceCode" id="cb329"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb329-1"><a href="data-and-programming.html#cb329-1" aria-hidden="true" tabindex="-1"></a>get_var <span class="ot">=</span> <span class="cf">function</span>(x, <span class="at">biased =</span> <span class="cn">FALSE</span>) {</span>
-<span id="cb329-2"><a href="data-and-programming.html#cb329-2" aria-hidden="true" tabindex="-1"></a>  n <span class="ot">=</span> <span class="fu">length</span>(x) <span class="sc">-</span> <span class="dv">1</span> <span class="sc">*</span> <span class="sc">!</span>biased</span>
-<span id="cb329-3"><a href="data-and-programming.html#cb329-3" aria-hidden="true" tabindex="-1"></a>  (<span class="dv">1</span> <span class="sc">/</span> n) <span class="sc">*</span> <span class="fu">sum</span>((x <span class="sc">-</span> <span class="fu">mean</span>(x)) <span class="sc">^</span> <span class="dv">2</span>)</span>
-<span id="cb329-4"><a href="data-and-programming.html#cb329-4" aria-hidden="true" tabindex="-1"></a>}</span></code></pre></div>
-<div class="sourceCode" id="cb330"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb330-1"><a href="data-and-programming.html#cb330-1" aria-hidden="true" tabindex="-1"></a><span class="fu">get_var</span>(test_sample)</span></code></pre></div>
-<pre><code>## [1] 40.20178</code></pre>
-<div class="sourceCode" id="cb332"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb332-1"><a href="data-and-programming.html#cb332-1" aria-hidden="true" tabindex="-1"></a><span class="fu">get_var</span>(test_sample, <span class="at">biased =</span> <span class="cn">FALSE</span>)</span></code></pre></div>
-<pre><code>## [1] 40.20178</code></pre>
-<div class="sourceCode" id="cb334"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb334-1"><a href="data-and-programming.html#cb334-1" aria-hidden="true" tabindex="-1"></a><span class="fu">var</span>(test_sample)</span></code></pre></div>
-<pre><code>## [1] 40.20178</code></pre>
+<div class="sourceCode" id="cb329"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb329-1"><a href="data-and-programming.html#cb329-1" tabindex="-1"></a>get_var <span class="ot">=</span> <span class="cf">function</span>(x, <span class="at">biased =</span> <span class="cn">FALSE</span>) {</span>
+<span id="cb329-2"><a href="data-and-programming.html#cb329-2" tabindex="-1"></a>  n <span class="ot">=</span> <span class="fu">length</span>(x) <span class="sc">-</span> <span class="dv">1</span> <span class="sc">*</span> <span class="sc">!</span>biased</span>
+<span id="cb329-3"><a href="data-and-programming.html#cb329-3" tabindex="-1"></a>  (<span class="dv">1</span> <span class="sc">/</span> n) <span class="sc">*</span> <span class="fu">sum</span>((x <span class="sc">-</span> <span class="fu">mean</span>(x)) <span class="sc">^</span> <span class="dv">2</span>)</span>
+<span id="cb329-4"><a href="data-and-programming.html#cb329-4" tabindex="-1"></a>}</span></code></pre></div>
+<div class="sourceCode" id="cb330"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb330-1"><a href="data-and-programming.html#cb330-1" tabindex="-1"></a><span class="fu">get_var</span>(test_sample)</span></code></pre></div>
+<pre><code>## [1] 8.100242</code></pre>
+<div class="sourceCode" id="cb332"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb332-1"><a href="data-and-programming.html#cb332-1" tabindex="-1"></a><span class="fu">get_var</span>(test_sample, <span class="at">biased =</span> <span class="cn">FALSE</span>)</span></code></pre></div>
+<pre><code>## [1] 8.100242</code></pre>
+<div class="sourceCode" id="cb334"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb334-1"><a href="data-and-programming.html#cb334-1" tabindex="-1"></a><span class="fu">var</span>(test_sample)</span></code></pre></div>
+<pre><code>## [1] 8.100242</code></pre>
 <p>We see the function is working as expected, and when returning the unbiased estimate it matches <code>R</code>’s built-in function <code>var()</code>. Finally, let’s examine the biased estimate of <span class="math inline">\(\sigma^2\)</span>.</p>
-<div class="sourceCode" id="cb336"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb336-1"><a href="data-and-programming.html#cb336-1" aria-hidden="true" tabindex="-1"></a><span class="fu">get_var</span>(test_sample, <span class="at">biased =</span> <span class="cn">TRUE</span>)</span></code></pre></div>
-<pre><code>## [1] 36.18161</code></pre>
+<div class="sourceCode" id="cb336"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb336-1"><a href="data-and-programming.html#cb336-1" tabindex="-1"></a><span class="fu">get_var</span>(test_sample, <span class="at">biased =</span> <span class="cn">TRUE</span>)</span></code></pre></div>
+<pre><code>## [1] 7.290217</code></pre>
 
 </div>
 </div>
diff --git a/data-and-programming.md b/data-and-programming.md
index 6510d4a..05267c8 100644
--- a/data-and-programming.md
+++ b/data-and-programming.md
@@ -35,7 +35,7 @@ Many operations in `R` make heavy use of **vectors**. Vectors in `R` are indexed
 Possibly the most common way to create a vector in `R` is using the `c()` function, which is short for "combine." As the name suggests, it combines a list of elements separated by commas. 
 
 
-```r
+``` r
 c(1, 3, 5, 7, 8, 9)
 ```
 
@@ -46,7 +46,7 @@ c(1, 3, 5, 7, 8, 9)
 Here `R` simply outputs this vector. If we would like to store this vector in a **variable**, we can do so with the **assignment** operator `=`. In this case the variable `x` now holds the vector we just created, and we can access the vector by typing `x`.
 
 
-```r
+``` r
 x = c(1, 3, 5, 7, 8, 9)
 x
 ```
@@ -64,7 +64,7 @@ If you wish to use `<-`, you will still need to use `=`, however only for argume
 Because vectors must contain elements that are all the same type, `R` will automatically coerce to a single type when attempting to create a vector that combines multiple types.
 
 
-```r
+``` r
 c(42, "Statistics", TRUE)
 ```
 
@@ -72,7 +72,7 @@ c(42, "Statistics", TRUE)
 ## [1] "42"         "Statistics" "TRUE"
 ```
 
-```r
+``` r
 c(42, TRUE)
 ```
 
@@ -83,7 +83,7 @@ c(42, TRUE)
 Frequently you may wish to create a vector based on a sequence of numbers. The quickest and easiest way to do this is with the `:` operator, which creates a sequence of integers between two specified integers.
 
 
-```r
+``` r
 (y = 1:100)
 ```
 
@@ -101,7 +101,7 @@ Here we see `R` labeling the rows after the first since this is a large vector.
 Note that scalars do not exist in `R`. They are simply vectors of length `1`.
 
 
-```r
+``` r
 2
 ```
 
@@ -112,7 +112,7 @@ Note that scalars do not exist in `R`. They are simply vectors of length `1`.
 To create a sequence that is not limited to consecutive integers, use the `seq()` function to define a sequence by its start, end, and increment.
 
 
-```r
+``` r
 seq(from = 1.5, to = 4.2, by = 0.1)
 ```
 
@@ -124,7 +124,7 @@ seq(from = 1.5, to = 4.2, by = 0.1)
 We will discuss functions in detail later, but note here that the input labels `from`, `to`, and `by` are optional.
 
 
-```r
+``` r
 seq(1.5, 4.2, 0.1)
 ```
 
@@ -136,7 +136,7 @@ seq(1.5, 4.2, 0.1)
 Another common operation to create a vector is `rep()`, which can repeat a single value a number of times.
 
 
-```r
+``` r
 rep("A", times = 10)
 ```
 
@@ -147,7 +147,7 @@ rep("A", times = 10)
 The `rep()` function can be used to repeat a vector some number of times.
 
 
-```r
+``` r
 rep(x, times = 3)
 ```
 
@@ -165,7 +165,7 @@ We have now seen four different ways to create vectors:
 So far we have mostly used them in isolation, but they are often used together.
 
 
-```r
+``` r
 c(x, rep(seq(1, 9, 2), 3), c(1, 2, 3), 42, 2:4)
 ```
 
@@ -177,7 +177,7 @@ c(x, rep(seq(1, 9, 2), 3), c(1, 2, 3), 42, 2:4)
 The length of a vector can be obtained with the `length()` function.
 
 
-```r
+``` r
 length(x)
 ```
 
@@ -185,7 +185,7 @@ length(x)
 ## [1] 6
 ```
 
-```r
+``` r
 length(y)
 ```
 
@@ -198,7 +198,7 @@ length(y)
 To subset a vector, we use square brackets, `[]`. 
 
 
-```r
+``` r
 x
 ```
 
@@ -206,7 +206,7 @@ x
 ## [1] 1 3 5 7 8 9
 ```
 
-```r
+``` r
 x[1]
 ```
 
@@ -214,7 +214,7 @@ x[1]
 ## [1] 1
 ```
 
-```r
+``` r
 x[3]
 ```
 
@@ -225,7 +225,7 @@ x[3]
 We see that `x[1]` returns the first element, and `x[3]` returns the third element.
 
 
-```r
+``` r
 x[-2]
 ```
 
@@ -236,7 +236,7 @@ x[-2]
 We can also exclude certain indexes, in this case the second element.
 
 
-```r
+``` r
 x[1:3]
 ```
 
@@ -244,7 +244,7 @@ x[1:3]
 ## [1] 1 3 5
 ```
 
-```r
+``` r
 x[c(1,3,4)]
 ```
 
@@ -257,7 +257,7 @@ Lastly we see that we can subset based on a vector of indices.
 All of the above are subsetting a vector using a vector of indexes. (Remember a single number is still a vector.) We could instead use a vector of logical values.
 
 
-```r
+``` r
 z = c(TRUE, TRUE, FALSE, TRUE, TRUE, FALSE)
 z
 ```
@@ -267,7 +267,7 @@ z
 ```
 
 
-```r
+``` r
 x[z]
 ```
 
@@ -280,7 +280,7 @@ x[z]
 One of the biggest strengths of `R` is its use of vectorized operations. (Frequently the lack of understanding of this concept leads to a belief that `R` is *slow*. `R` is not the fastest language, but it has a reputation for being slower than it really is.)
 
 
-```r
+``` r
 x = 1:10
 x + 1
 ```
@@ -289,7 +289,7 @@ x + 1
 ##  [1]  2  3  4  5  6  7  8  9 10 11
 ```
 
-```r
+``` r
 2 * x
 ```
 
@@ -297,7 +297,7 @@ x + 1
 ##  [1]  2  4  6  8 10 12 14 16 18 20
 ```
 
-```r
+``` r
 2 ^ x
 ```
 
@@ -305,7 +305,7 @@ x + 1
 ##  [1]    2    4    8   16   32   64  128  256  512 1024
 ```
 
-```r
+``` r
 sqrt(x)
 ```
 
@@ -314,7 +314,7 @@ sqrt(x)
 ##  [9] 3.000000 3.162278
 ```
 
-```r
+``` r
 log(x)
 ```
 
@@ -343,12 +343,12 @@ We see that when a function like `log()` is called on a vector `x`, a vector is
 In `R`, logical operators are vectorized. 
 
 
-```r
+``` r
 x = c(1, 3, 5, 7, 8, 9)
 ```
 
 
-```r
+``` r
 x > 3
 ```
 
@@ -356,7 +356,7 @@ x > 3
 ## [1] FALSE FALSE  TRUE  TRUE  TRUE  TRUE
 ```
 
-```r
+``` r
 x < 3
 ```
 
@@ -364,7 +364,7 @@ x < 3
 ## [1]  TRUE FALSE FALSE FALSE FALSE FALSE
 ```
 
-```r
+``` r
 x == 3
 ```
 
@@ -372,7 +372,7 @@ x == 3
 ## [1] FALSE  TRUE FALSE FALSE FALSE FALSE
 ```
 
-```r
+``` r
 x != 3
 ```
 
@@ -381,7 +381,7 @@ x != 3
 ```
 
 
-```r
+``` r
 x == 3 & x != 3
 ```
 
@@ -389,7 +389,7 @@ x == 3 & x != 3
 ## [1] FALSE FALSE FALSE FALSE FALSE FALSE
 ```
 
-```r
+``` r
 x == 3 | x != 3
 ```
 
@@ -400,7 +400,7 @@ x == 3 | x != 3
 This is extremely useful for subsetting.
 
 
-```r
+``` r
 x[x > 3]
 ```
 
@@ -408,7 +408,7 @@ x[x > 3]
 ## [1] 5 7 8 9
 ```
 
-```r
+``` r
 x[x != 3]
 ```
 
@@ -417,7 +417,7 @@ x[x != 3]
 ```
 
 
-```r
+``` r
 sum(x > 3)
 ```
 
@@ -425,7 +425,7 @@ sum(x > 3)
 ## [1] 4
 ```
 
-```r
+``` r
 as.numeric(x > 3)
 ```
 
@@ -436,7 +436,7 @@ as.numeric(x > 3)
 Here we see that using the `sum()` function on a vector of logical `TRUE` and `FALSE` values that is the result of `x > 3` results in a numeric result. `R` is first automatically coercing the logical to numeric where `TRUE` is `1` and `FALSE` is `0`. This coercion from logical to numeric happens for most mathematical operations. If you are interested in more detail, check out [Advanced R](https://adv-r.hadley.nz/vectors-chap.html?q=coercion#testing-and-coercion).
 
 
-```r
+``` r
 which(x > 3)
 ```
 
@@ -444,7 +444,7 @@ which(x > 3)
 ## [1] 3 4 5 6
 ```
 
-```r
+``` r
 x[which(x > 3)]
 ```
 
@@ -452,7 +452,7 @@ x[which(x > 3)]
 ## [1] 5 7 8 9
 ```
 
-```r
+``` r
 max(x)
 ```
 
@@ -460,7 +460,7 @@ max(x)
 ## [1] 9
 ```
 
-```r
+``` r
 which(x == max(x))
 ```
 
@@ -468,7 +468,7 @@ which(x == max(x))
 ## [1] 6
 ```
 
-```r
+``` r
 which.max(x)
 ```
 
@@ -479,13 +479,13 @@ which.max(x)
 ### More Vectorization
 
 
-```r
+``` r
 x = c(1, 3, 5, 7, 8, 9)
 y = 1:100
 ```
 
 
-```r
+``` r
 x + 2
 ```
 
@@ -493,7 +493,7 @@ x + 2
 ## [1]  3  5  7  9 10 11
 ```
 
-```r
+``` r
 x + rep(2, 6)
 ```
 
@@ -502,7 +502,7 @@ x + rep(2, 6)
 ```
 
 
-```r
+``` r
 x > 3
 ```
 
@@ -510,7 +510,7 @@ x > 3
 ## [1] FALSE FALSE  TRUE  TRUE  TRUE  TRUE
 ```
 
-```r
+``` r
 x > rep(3, 6)
 ```
 
@@ -519,7 +519,7 @@ x > rep(3, 6)
 ```
 
 
-```r
+``` r
 x + y
 ```
 
@@ -537,7 +537,7 @@ x + y
 ##  [91]  92  95  98 101 103 105  98 101 104 107
 ```
 
-```r
+``` r
 length(x)
 ```
 
@@ -545,7 +545,7 @@ length(x)
 ## [1] 6
 ```
 
-```r
+``` r
 length(y)
 ```
 
@@ -553,7 +553,7 @@ length(y)
 ## [1] 100
 ```
 
-```r
+``` r
 length(y) / length(x)
 ```
 
@@ -561,7 +561,7 @@ length(y) / length(x)
 ## [1] 16.66667
 ```
 
-```r
+``` r
 (x + y) - y
 ```
 
@@ -577,7 +577,7 @@ length(y) / length(x)
 ```
 
 
-```r
+``` r
 y = 1:60
 x + y
 ```
@@ -588,7 +588,7 @@ x + y
 ## [51] 56 59 61 63 56 59 62 65 67 69
 ```
 
-```r
+``` r
 length(y) / length(x)
 ```
 
@@ -597,7 +597,7 @@ length(y) / length(x)
 ```
 
 
-```r
+``` r
 rep(x, 10) + y
 ```
 
@@ -608,7 +608,7 @@ rep(x, 10) + y
 ```
 
 
-```r
+``` r
 all(x + y == rep(x, 10) + y)
 ```
 
@@ -616,7 +616,7 @@ all(x + y == rep(x, 10) + y)
 ## [1] TRUE
 ```
 
-```r
+``` r
 identical(x + y, rep(x, 10) + y)
 ```
 
@@ -625,13 +625,13 @@ identical(x + y, rep(x, 10) + y)
 ```
 
 
-```r
+``` r
 # ?any
 # ?all.equal
 ```
 
 
-```r
+``` r
 x = c(1, 3, 5)
 y = c(1, 2, 4)
 x == y
@@ -641,7 +641,7 @@ x == y
 ## [1]  TRUE FALSE FALSE
 ```
 
-```r
+``` r
 all(x == y)
 ```
 
@@ -649,7 +649,7 @@ all(x == y)
 ## [1] FALSE
 ```
 
-```r
+``` r
 any(x == y)
 ```
 
@@ -660,7 +660,7 @@ any(x == y)
 While `all` returns `TRUE` only when all of its arguments are `TRUE`, `any` returns `TRUE` when at least one of its arguments is `TRUE`.
 
 
-```r
+``` r
 x = c(10 ^ (-8))
 y = c(10 ^ (-9))
 all(x == y)
@@ -670,7 +670,7 @@ all(x == y)
 ## [1] FALSE
 ```
 
-```r
+``` r
 all.equal(x, y)
 ```
 
@@ -687,7 +687,7 @@ The `all.equal` function tests "near equality" with a default tolerance value ar
 Matrices can be created using the `matrix` function. 
 
 
-```r
+``` r
 x = 1:9
 x
 ```
@@ -696,7 +696,7 @@ x
 ## [1] 1 2 3 4 5 6 7 8 9
 ```
 
-```r
+``` r
 X = matrix(x, nrow = 3, ncol = 3)
 X
 ```
@@ -713,7 +713,7 @@ Note here that we are using two different variables: lower case `x`, which store
 By default the `matrix` function reorders a vector into columns, but we can also tell `R` to use rows instead.
 
 
-```r
+``` r
 Y = matrix(x, nrow = 3, ncol = 3, byrow = TRUE)
 Y
 ```
@@ -728,7 +728,7 @@ Y
 We can also create a matrix of a specified dimension where every element is the same, in this case `0`.
 
 
-```r
+``` r
 Z = matrix(0, 2, 4)
 Z
 ```
@@ -742,7 +742,7 @@ Z
 Like vectors, matrices can be subsetted using square brackets, `[]`. However, since matrices are two-dimensional, we need to specify both a row and a column when subsetting.
 
 
-```r
+``` r
 X
 ```
 
@@ -753,7 +753,7 @@ X
 ## [3,]    3    6    9
 ```
 
-```r
+``` r
 X[1, 2]
 ```
 
@@ -764,7 +764,7 @@ X[1, 2]
 Here we accessed the element in the first row and the second column. We could also subset an entire row or column.
 
 
-```r
+``` r
 X[1, ]
 ```
 
@@ -772,7 +772,7 @@ X[1, ]
 ## [1] 1 4 7
 ```
 
-```r
+``` r
 X[, 2]
 ```
 
@@ -783,7 +783,7 @@ X[, 2]
 We can also use vectors to subset more than one row or column at a time. Here we subset to the first and third column of the second row.
 
 
-```r
+``` r
 X[2, c(1, 3)]
 ```
 
@@ -794,7 +794,7 @@ X[2, c(1, 3)]
 Matrices can also be created by combining vectors as columns, using `cbind`, or combining vectors as rows, using `rbind`.
 
 
-```r
+``` r
 x = 1:9
 rev(x)
 ```
@@ -803,7 +803,7 @@ rev(x)
 ## [1] 9 8 7 6 5 4 3 2 1
 ```
 
-```r
+``` r
 rep(1, 9)
 ```
 
@@ -812,7 +812,7 @@ rep(1, 9)
 ```
 
 
-```r
+``` r
 rbind(x, rev(x), rep(1, 9))
 ```
 
@@ -824,7 +824,7 @@ rbind(x, rev(x), rep(1, 9))
 ```
 
 
-```r
+``` r
 cbind(col_1 = x, col_2 = rev(x), col_3 = rep(1, 9))
 ```
 
@@ -846,7 +846,7 @@ When using `rbind` and `cbind` you can specify "argument" names that will be use
 `R` can then be used to perform matrix calculations.
 
 
-```r
+``` r
 x = 1:9
 y = 9:1
 X = matrix(x, 3, 3)
@@ -861,7 +861,7 @@ X
 ## [3,]    3    6    9
 ```
 
-```r
+``` r
 Y
 ```
 
@@ -873,7 +873,7 @@ Y
 ```
 
 
-```r
+``` r
 X + Y
 ```
 
@@ -884,7 +884,7 @@ X + Y
 ## [3,]   10   10   10
 ```
 
-```r
+``` r
 X - Y
 ```
 
@@ -895,7 +895,7 @@ X - Y
 ## [3,]   -4    2    8
 ```
 
-```r
+``` r
 X * Y
 ```
 
@@ -906,7 +906,7 @@ X * Y
 ## [3,]   21   24    9
 ```
 
-```r
+``` r
 X / Y
 ```
 
@@ -920,7 +920,7 @@ X / Y
 Note that `X * Y` is not matrix multiplication. It is element by element multiplication. (Same for `X / Y`). Instead, matrix multiplication uses `%*%`. Other matrix functions include `t()` which gives the transpose of a matrix and `solve()` which returns the inverse of a square matrix if it is invertible.
 
 
-```r
+``` r
 X %*% Y
 ```
 
@@ -931,7 +931,7 @@ X %*% Y
 ## [3,]  138   84   30
 ```
 
-```r
+``` r
 t(X)
 ```
 
@@ -943,7 +943,7 @@ t(X)
 ```
 
 
-```r
+``` r
 Z = matrix(c(9, 2, -3, 2, 4, -2, -3, -2, 16), 3, byrow = TRUE)
 Z
 ```
@@ -955,7 +955,7 @@ Z
 ## [3,]   -3   -2   16
 ```
 
-```r
+``` r
 solve(Z)
 ```
 
@@ -969,18 +969,18 @@ solve(Z)
 To verify that `solve(Z)` returns the inverse, we multiply it by `Z`. We would expect this to return the identity matrix, however we see that this is not the case due to some computational issues. However, `R` also has the `all.equal()` function which checks for equality, with some small tolerance which accounts for some computational issues. The `identical()` function is used to check for exact equality.
 
 
-```r
+``` r
 solve(Z) %*% Z
 ```
 
 ```
 ##              [,1]          [,2]         [,3]
 ## [1,] 1.000000e+00 -6.245005e-17 0.000000e+00
-## [2,] 8.326673e-17  1.000000e+00 5.551115e-17
+## [2,] 7.979728e-17  1.000000e+00 5.551115e-17
 ## [3,] 2.775558e-17  0.000000e+00 1.000000e+00
 ```
 
-```r
+``` r
 diag(3)
 ```
 
@@ -991,7 +991,7 @@ diag(3)
 ## [3,]    0    0    1
 ```
 
-```r
+``` r
 all.equal(solve(Z) %*% Z, diag(3))
 ```
 
@@ -1002,7 +1002,7 @@ all.equal(solve(Z) %*% Z, diag(3))
 `R` has a number of matrix specific functions for obtaining dimension and summary information.
 
 
-```r
+``` r
 X = matrix(1:6, 2, 3)
 X
 ```
@@ -1013,7 +1013,7 @@ X
 ## [2,]    2    4    6
 ```
 
-```r
+``` r
 dim(X)
 ```
 
@@ -1021,7 +1021,7 @@ dim(X)
 ## [1] 2 3
 ```
 
-```r
+``` r
 rowSums(X)
 ```
 
@@ -1029,7 +1029,7 @@ rowSums(X)
 ## [1]  9 12
 ```
 
-```r
+``` r
 colSums(X)
 ```
 
@@ -1037,7 +1037,7 @@ colSums(X)
 ## [1]  3  7 11
 ```
 
-```r
+``` r
 rowMeans(X)
 ```
 
@@ -1045,7 +1045,7 @@ rowMeans(X)
 ## [1] 3 4
 ```
 
-```r
+``` r
 colMeans(X)
 ```
 
@@ -1056,7 +1056,7 @@ colMeans(X)
 The `diag()` function can be used in a number of ways. We can extract the diagonal of a matrix.
 
 
-```r
+``` r
 diag(Z)
 ```
 
@@ -1067,7 +1067,7 @@ diag(Z)
 Or create a matrix with specified elements on the diagonal. (And `0` on the off-diagonals.)
 
 
-```r
+``` r
 diag(1:5)
 ```
 
@@ -1083,7 +1083,7 @@ diag(1:5)
 Or, lastly, create a square matrix of a certain dimension with `1` for every element of the diagonal and `0` for the off-diagonals.
 
 
-```r
+``` r
 diag(5)
 ```
 
@@ -1101,7 +1101,7 @@ diag(5)
 Certain operations in `R`, for example `%*%` have different behavior on vectors and matrices. To illustrate this, we will first create two vectors.
 
 
-```r
+``` r
 a_vec = c(1, 2, 3)
 b_vec = c(2, 2, 2)
 ```
@@ -1109,7 +1109,7 @@ b_vec = c(2, 2, 2)
 Note that these are indeed vectors. They are not matrices.
 
 
-```r
+``` r
 c(is.vector(a_vec), is.vector(b_vec))
 ```
 
@@ -1117,7 +1117,7 @@ c(is.vector(a_vec), is.vector(b_vec))
 ## [1] TRUE TRUE
 ```
 
-```r
+``` r
 c(is.matrix(a_vec), is.matrix(b_vec))
 ```
 
@@ -1134,7 +1134,7 @@ The dot product of vectors $\boldsymbol{a} = \lbrack a_1, a_2, \cdots a_n \rbrac
 \]
 
 
-```r
+``` r
 a_vec %*% b_vec # inner product
 ```
 
@@ -1143,7 +1143,7 @@ a_vec %*% b_vec # inner product
 ## [1,]   12
 ```
 
-```r
+``` r
 a_vec %o% b_vec # outer product
 ```
 
@@ -1159,7 +1159,7 @@ The `%o%` operator is used to calculate the **outer product** of the two vectors
 When vectors are coerced to become matrices, they are column vectors. So a vector of length $n$ becomes an $n \times 1$ matrix after coercion.
 
 
-```r
+``` r
 as.matrix(a_vec)
 ```
 
@@ -1173,7 +1173,7 @@ as.matrix(a_vec)
 If we use the `%*%` operator on matrices, `%*%` again performs the expected matrix multiplication. So you might expect the following to produce an error, because the dimensions are incorrect.
 
 
-```r
+``` r
 as.matrix(a_vec) %*% b_vec
 ```
 
@@ -1189,14 +1189,14 @@ At face value this is a $3 \times 1$ matrix, multiplied by a $3 \times 1$ matrix
 If we had coerced both, then `R` would produce an error.
 
 
-```r
+``` r
 as.matrix(a_vec) %*% as.matrix(b_vec)
 ```
 
 Another way to calculate a *dot product* is with the `crossprod()` function. Given two vectors, the `crossprod()` function calculates their dot product. The function has a rather misleading name.
 
 
-```r
+``` r
 crossprod(a_vec, b_vec)  # inner product
 ```
 
@@ -1205,7 +1205,7 @@ crossprod(a_vec, b_vec)  # inner product
 ## [1,]   12
 ```
 
-```r
+``` r
 tcrossprod(a_vec, b_vec)  # outer product
 ```
 
@@ -1231,7 +1231,7 @@ X^\top X
 is used repeatedly.
 
 
-```r
+``` r
 C_mat = matrix(c(1, 2, 3, 4, 5, 6), 2, 3)
 D_mat = matrix(c(2, 2, 2, 2, 2, 2), 2, 3)
 ```
@@ -1239,7 +1239,7 @@ D_mat = matrix(c(2, 2, 2, 2, 2, 2), 2, 3)
 This is useful both as a shortcut for a frequent calculation and as a more efficient implementation than using `t()` and `%*%`.
 
 
-```r
+``` r
 crossprod(C_mat, D_mat)
 ```
 
@@ -1250,7 +1250,7 @@ crossprod(C_mat, D_mat)
 ## [3,]   22   22   22
 ```
 
-```r
+``` r
 t(C_mat) %*% D_mat
 ```
 
@@ -1261,7 +1261,7 @@ t(C_mat) %*% D_mat
 ## [3,]   22   22   22
 ```
 
-```r
+``` r
 all.equal(crossprod(C_mat, D_mat), t(C_mat) %*% D_mat)
 ```
 
@@ -1270,7 +1270,7 @@ all.equal(crossprod(C_mat, D_mat), t(C_mat) %*% D_mat)
 ```
 
 
-```r
+``` r
 crossprod(C_mat, C_mat)
 ```
 
@@ -1281,7 +1281,7 @@ crossprod(C_mat, C_mat)
 ## [3,]   17   39   61
 ```
 
-```r
+``` r
 t(C_mat) %*% C_mat
 ```
 
@@ -1292,7 +1292,7 @@ t(C_mat) %*% C_mat
 ## [3,]   17   39   61
 ```
 
-```r
+``` r
 all.equal(crossprod(C_mat, C_mat), t(C_mat) %*% C_mat)
 ```
 
@@ -1305,7 +1305,7 @@ all.equal(crossprod(C_mat, C_mat), t(C_mat) %*% C_mat)
 A list is a one-dimensional heterogeneous data structure. So it is indexed like a vector with a single integer value, but each element can contain an element of any type.
 
 
-```r
+``` r
 # creation
 list(42, "Hello", TRUE)
 ```
@@ -1321,7 +1321,7 @@ list(42, "Hello", TRUE)
 ## [1] TRUE
 ```
 
-```r
+``` r
 ex_list = list(
   a = c(1, 2, 3, 4),
   b = TRUE,
@@ -1337,7 +1337,7 @@ Lists can be subset using two syntaxes, the `$` operator, and square brackets `[
 - `ex_list[[1]]` returns the first element of the list, in this case, a vector.
 
 
-```r
+``` r
 # subsetting
 ex_list$e
 ```
@@ -1351,7 +1351,7 @@ ex_list$e
 ## [5,]    0    0    0    0    1
 ```
 
-```r
+``` r
 ex_list[1:2]
 ```
 
@@ -1363,7 +1363,7 @@ ex_list[1:2]
 ## [1] TRUE
 ```
 
-```r
+``` r
 ex_list[1]
 ```
 
@@ -1372,7 +1372,7 @@ ex_list[1]
 ## [1] 1 2 3 4
 ```
 
-```r
+``` r
 ex_list[[1]]
 ```
 
@@ -1380,7 +1380,7 @@ ex_list[[1]]
 ## [1] 1 2 3 4
 ```
 
-```r
+``` r
 ex_list[c("e", "a")]
 ```
 
@@ -1397,7 +1397,7 @@ ex_list[c("e", "a")]
 ## [1] 1 2 3 4
 ```
 
-```r
+``` r
 ex_list["e"]
 ```
 
@@ -1411,7 +1411,7 @@ ex_list["e"]
 ## [5,]    0    0    0    0    1
 ```
 
-```r
+``` r
 ex_list[["e"]]
 ```
 
@@ -1424,7 +1424,7 @@ ex_list[["e"]]
 ## [5,]    0    0    0    0    1
 ```
 
-```r
+``` r
 ex_list$d
 ```
 
@@ -1432,7 +1432,7 @@ ex_list$d
 ## function(arg = 42) {print("Hello World!")}
 ```
 
-```r
+``` r
 ex_list$d(arg = 1)
 ```
 
@@ -1445,7 +1445,7 @@ ex_list$d(arg = 1)
 We have previously seen vectors and matrices for storing data as we introduced `R`. We will now introduce a **data frame** which will be the most common way that we store and interact with data in this course.
 
 
-```r
+``` r
 example_data = data.frame(x = c(1, 3, 5, 7, 9, 1, 3, 5, 7, 9),
                           y = c(rep("Hello", 9), "Goodbye"),
                           z = rep(c(TRUE, FALSE), 5))
@@ -1454,7 +1454,7 @@ example_data = data.frame(x = c(1, 3, 5, 7, 9, 1, 3, 5, 7, 9),
 Unlike a matrix, which can be thought of as a vector rearranged into rows and columns, a data frame is not required to have the same data type for each element. A data frame is a **list** of vectors. So, each vector must contain the same data type, but the different vectors can store different data types. 
 
 
-```r
+``` r
 example_data
 ```
 
@@ -1475,7 +1475,7 @@ example_data
 Unlike a list which has more flexibility, the elements of a data frame must all be vectors, and have the same length.
 
 
-```r
+``` r
 example_data$x
 ```
 
@@ -1483,7 +1483,7 @@ example_data$x
 ##  [1] 1 3 5 7 9 1 3 5 7 9
 ```
 
-```r
+``` r
 all.equal(length(example_data$x),
           length(example_data$y),
           length(example_data$z))
@@ -1493,7 +1493,7 @@ all.equal(length(example_data$x),
 ## [1] TRUE
 ```
 
-```r
+``` r
 str(example_data)
 ```
 
@@ -1504,7 +1504,7 @@ str(example_data)
 ##  $ z: logi  TRUE FALSE TRUE FALSE TRUE FALSE ...
 ```
 
-```r
+``` r
 nrow(example_data)
 ```
 
@@ -1512,7 +1512,7 @@ nrow(example_data)
 ## [1] 10
 ```
 
-```r
+``` r
 ncol(example_data)
 ```
 
@@ -1520,7 +1520,7 @@ ncol(example_data)
 ## [1] 3
 ```
 
-```r
+``` r
 dim(example_data)
 ```
 
@@ -1535,7 +1535,7 @@ The `data.frame()` function above is one way to create a data frame. We can also
 [The example data above can also be found here as a .csv file.](data/example-data.csv) To read this data into `R`, we would use the `read_csv()` function from the `readr` package. Note that `R` has a built-in function `read.csv()` that operates very similarly. The `readr` function `read_csv()` has a number of advantages. For example, it is much faster reading larger data. [It also uses the `tibble` package to read the data as a tibble.](https://cran.r-project.org/web/packages/tibble/vignettes/tibble.html){target="_blank"}
 
 
-```r
+``` r
 library(readr)
 example_data_from_csv = read_csv("data/example-data.csv")
 ```
@@ -1543,7 +1543,7 @@ example_data_from_csv = read_csv("data/example-data.csv")
 This particular line of code assumes that the file `example_data.csv` exists in a folder called `data` in your current working directory.
 
 
-```r
+``` r
 example_data_from_csv
 ```
 
@@ -1568,7 +1568,7 @@ A tibble is simply a data frame that prints with sanity. Notice in the output ab
 The `as_tibble()` function can be used to coerce a regular data frame to a tibble.
 
 
-```r
+``` r
 library(tibble)
 example_data = as_tibble(example_data)
 example_data
@@ -1595,7 +1595,7 @@ Alternatively, we could use the "Import Dataset" feature in RStudio which can be
 Earlier we looked at installing packages, in particular the `ggplot2` package. (A package for visualization. While not necessary for this course, it is quickly growing in popularity.) 
 
 
-```r
+``` r
 library(ggplot2)
 ```
 
@@ -1610,7 +1610,7 @@ When using data from inside a package, there are three things we would generally
 To look at the data, we have two useful commands: `head()` and `str()`.
 
 
-```r
+``` r
 head(mpg, n = 10)
 ```
 
@@ -1633,7 +1633,7 @@ head(mpg, n = 10)
 The function `head()` will display the first `n` observations of the data frame. The `head()` function was more useful before tibbles. Notice that `mpg` is a tibble already, so the output from `head()` indicates there are only `10` observations. Note that this applies to `head(mpg, n = 10)` and not `mpg` itself. Also note that tibbles print a limited number of rows and columns by default. The last line of the printed output indicates which rows and columns were omitted.
 
 
-```r
+``` r
 mpg
 ```
 
@@ -1657,7 +1657,7 @@ mpg
 The function `str()` will display the "structure" of the data frame. It will display the number of **observations** and **variables**, list the variables, give the type of each variable, and show some elements of each variable. This information can also be found in the "Environment" window in RStudio.
 
 
-```r
+``` r
 str(mpg)
 ```
 
@@ -1683,14 +1683,14 @@ In this dataset an observation is for a particular model-year of a car, and the
 To understand more about the data set, we use the `?` operator to pull up the documentation for the data.
 
 
-```r
+``` r
 ?mpg
 ```
 
 `R` has a number of functions for quickly working with and extracting basic information from data frames. To quickly obtain a vector of the variable names, we use the `names()` function.
 
 
-```r
+``` r
 names(mpg)
 ```
 
@@ -1703,7 +1703,7 @@ names(mpg)
 To access one of the variables **as a vector**, we use the `$` operator.
 
 
-```r
+``` r
 mpg$year
 ```
 
@@ -1726,7 +1726,7 @@ mpg$year
 ## [226] 2008 2008 1999 1999 2008 2008 1999 1999 2008
 ```
 
-```r
+``` r
 mpg$hwy
 ```
 
@@ -1746,7 +1746,7 @@ mpg$hwy
 We can use the `dim()`, `nrow()` and `ncol()` functions to obtain information about the dimension of the data frame.
 
 
-```r
+``` r
 dim(mpg)
 ```
 
@@ -1754,7 +1754,7 @@ dim(mpg)
 ## [1] 234  11
 ```
 
-```r
+``` r
 nrow(mpg)
 ```
 
@@ -1762,7 +1762,7 @@ nrow(mpg)
 ## [1] 234
 ```
 
-```r
+``` r
 ncol(mpg)
 ```
 
@@ -1775,7 +1775,7 @@ Here `nrow()` is also the number of observations, which in most cases is the *sa
 Subsetting data frames can work much like subsetting matrices using square brackets, `[,]`. Here, we find fuel efficient vehicles earning over 35 miles per gallon and only display `manufacturer`, `model` and `year`.
 
 
-```r
+``` r
 mpg[mpg$hwy > 35, c("manufacturer", "model", "year")]
 ```
 
@@ -1794,14 +1794,14 @@ mpg[mpg$hwy > 35, c("manufacturer", "model", "year")]
 An alternative would be to use the `subset()` function, which has a much more readable syntax.
 
 
-```r
+``` r
 subset(mpg, subset = hwy > 35, select = c("manufacturer", "model", "year"))
 ```
 
 Lastly, we could use the `filter` and `select` functions from the `dplyr` package which introduces the `%>%` operator from the `magrittr` package. This is not necessary for this course, however the `dplyr` package is something you should be aware of as it is becoming a popular tool in the `R` world.
 
 
-```r
+``` r
 library(dplyr)
 mpg %>% filter(hwy > 35) %>% select(manufacturer, model, year)
 ```
@@ -1817,7 +1817,7 @@ When subsetting a data frame, be aware of what is being returned, as sometimes i
 In `R`, the if/else syntax is:
 
 
-```r
+``` r
 if (...) {
   some R code
 } else {
@@ -1828,7 +1828,7 @@ if (...) {
 For example,
 
 
-```r
+``` r
 x = 1
 y = 3
 if (x > y) {
@@ -1844,7 +1844,7 @@ if (x > y) {
 ## [1] "x is less than or equal to y"
 ```
 
-```r
+``` r
 z
 ```
 
@@ -1855,7 +1855,7 @@ z
 `R` also has a special function `ifelse()` which is very useful. It returns one of two specified values based on a conditional statement.
 
 
-```r
+``` r
 ifelse(4 > 3, 1, 0)
 ```
 
@@ -1866,7 +1866,7 @@ ifelse(4 > 3, 1, 0)
 The real power of `ifelse()` comes from its ability to be applied to vectors.
 
 
-```r
+``` r
 fib = c(1, 1, 2, 3, 5, 8, 13, 21)
 ifelse(fib > 6, "Foo", "Bar")
 ```
@@ -1878,7 +1878,7 @@ ifelse(fib > 6, "Foo", "Bar")
 Now a `for` loop example,
 
 
-```r
+``` r
 x = 11:15
 for (i in 1:5) {
   x[i] = x[i] * 2
@@ -1894,7 +1894,7 @@ x
 Note that this `for` loop is very normal in many programming languages, but not in `R`. In `R` we would not use a loop, instead we would simply use a vectorized operation.
 
 
-```r
+``` r
 x = 11:15
 x = x * 2
 x
@@ -1909,7 +1909,7 @@ x
 So far we have been using functions, but haven't actually discussed some of their details.
 
 
-```r
+``` r
 function_name(arg1 = 10, arg2 = 20)
 ```
 
@@ -1930,7 +1930,7 @@ In `R` we would write a function to do this. When writing a function, there are
 - Write the body of the function within curly braces, `{}`.
 
 
-```r
+``` r
 standardize = function(x) {
   m = mean(x)
   std = sd(x)
@@ -1944,28 +1944,28 @@ Here the name of the function is `standardize`, and the function has a single ar
 To test our function, we will take a random sample of size `n = 10` from a normal distribution with a mean of `2` and a standard deviation of `5`.
 
 
-```r
+``` r
 (test_sample = rnorm(n = 10, mean = 2, sd = 5))
 ```
 
 ```
-##  [1] -1.655666 -3.073760  4.389262  7.668348  1.623974  7.688881 13.272477
-##  [8] -3.536843  4.483411  2.058641
+##  [1] 12.835022  9.646464  3.305105  5.454080 -0.171593 -9.308482 -5.405810
+##  [8]  4.136374  8.970424  4.704120
 ```
 
-```r
+``` r
 standardize(x = test_sample)
 ```
 
 ```
-##  [1] -0.9264749 -1.1920270  0.2054970  0.8195378 -0.3123303  0.8233828
-##  [7]  1.8689658 -1.2787438  0.2231272 -0.2309346
+##  [1]  1.38310669  0.91486460 -0.01636882  0.29920979 -0.52692450 -1.86868350
+##  [7] -1.29557322  0.10570369  0.81558762  0.18907765
 ```
 
 This function could be written much more succinctly, simply performing all the operations on one line and immediately returning the result, without storing any of the intermediate results.
 
 
-```r
+``` r
 standardize = function(x) {
   (x - mean(x)) / sd(x)
 }
@@ -1974,7 +1974,7 @@ standardize = function(x) {
 When specifying arguments, you can provide default arguments.
 
 
-```r
+``` r
 power_of_num = function(num, power = 2) {
   num ^ power
 }
@@ -1983,7 +1983,7 @@ power_of_num = function(num, power = 2) {
 Let's look at a number of ways that we could run this function to perform the operation `10^2` resulting in `100`.
 
 
-```r
+``` r
 power_of_num(10)
 ```
 
@@ -1991,7 +1991,7 @@ power_of_num(10)
 ## [1] 100
 ```
 
-```r
+``` r
 power_of_num(10, 2)
 ```
 
@@ -1999,7 +1999,7 @@ power_of_num(10, 2)
 ## [1] 100
 ```
 
-```r
+``` r
 power_of_num(num = 10, power = 2)
 ```
 
@@ -2007,7 +2007,7 @@ power_of_num(num = 10, power = 2)
 ## [1] 100
 ```
 
-```r
+``` r
 power_of_num(power = 2, num = 10)
 ```
 
@@ -2018,7 +2018,7 @@ power_of_num(power = 2, num = 10)
 Note that without using the argument names, the order matters. The following code will not evaluate to the same output as the previous example.
 
 
-```r
+``` r
 power_of_num(2, 10)
 ```
 
@@ -2029,7 +2029,7 @@ power_of_num(2, 10)
 Also, the following line of code would produce an error since arguments without a default value must be specified.
 
 
-```r
+``` r
 power_of_num(power = 5)
 ```
 
@@ -2048,7 +2048,7 @@ It will also have the ability to return the biased estimate (based on maximum li
 \]
 
 
-```r
+``` r
 get_var = function(x, biased = FALSE) {
   n = length(x) - 1 * !biased
   (1 / n) * sum((x - mean(x)) ^ 2)
@@ -2056,39 +2056,39 @@ get_var = function(x, biased = FALSE) {
 ```
 
 
-```r
+``` r
 get_var(test_sample)
 ```
 
 ```
-## [1] 28.51747
+## [1] 46.37113
 ```
 
-```r
+``` r
 get_var(test_sample, biased = FALSE)
 ```
 
 ```
-## [1] 28.51747
+## [1] 46.37113
 ```
 
-```r
+``` r
 var(test_sample)
 ```
 
 ```
-## [1] 28.51747
+## [1] 46.37113
 ```
 
 We see the function is working as expected, and when returning the unbiased estimate it matches `R`'s built-in function `var()`. Finally, let's examine the biased estimate of $\sigma^2$.
 
 
-```r
+``` r
 get_var(test_sample, biased = TRUE)
 ```
 
 ```
-## [1] 25.66572
+## [1] 41.73401
 ```
 
 
diff --git a/data-summary.md b/data-summary.md
index 8f192cb..507746e 100644
--- a/data-summary.md
+++ b/data-summary.md
@@ -5,7 +5,7 @@
 `R` has built-in functions for a large number of summary statistics. For numeric variables, we can summarize data with the center and spread. We'll again look at the `mpg` dataset from the `ggplot2` package.
 
 
-```r
+``` r
 library(ggplot2)
 ```
 
@@ -32,7 +32,7 @@ library(ggplot2)
 For categorical variables, counts and percentages can be used for summary.
 
 
-```r
+``` r
 table(mpg$drv)
 ```
 
@@ -42,7 +42,7 @@ table(mpg$drv)
 ## 103 106  25
 ```
 
-```r
+``` r
 table(mpg$drv) / nrow(mpg)
 ```
 
@@ -68,7 +68,7 @@ We will look at four methods of visualizing data that we will use throughout the
 When visualizing a single numerical variable, a **histogram** will be our go-to tool, which can be created in `R` using the `hist()` function.
 
 
-```r
+``` r
 hist(mpg$cty)
 ```
 
@@ -77,7 +77,7 @@ hist(mpg$cty)
 The histogram function has a number of parameters which can be changed to make our plot look much nicer. Use the `?` operator to read the documentation for the `hist()` to see a full list of these parameters.
 
 
-```r
+``` r
 hist(mpg$cty,
      xlab   = "Miles Per Gallon (City)",
      main   = "Histogram of MPG (City)",
@@ -95,14 +95,14 @@ Importantly, you should always be sure to label your axes and give the plot a ti
 Somewhat similar to a histogram, a barplot can provide a visual summary of a categorical variable, or a numeric variable with a finite number of values, like a ranking from 1 to 10.
 
 
-```r
+``` r
 barplot(table(mpg$drv))
 ```
 
 ![](data-summary_files/figure-latex/unnamed-chunk-5-1.pdf)<!-- --> 
 
 
-```r
+``` r
 barplot(table(mpg$drv),
      xlab   = "Drivetrain (f = FWD, r = RWD, 4 = 4WD)",
      ylab   = "Frequency",
@@ -118,7 +118,7 @@ barplot(table(mpg$drv),
 To visualize the relationship between a numerical and categorical variable, we will use a **boxplot**. In the `mpg` dataset, the `drv` variable takes a small, finite number of values. A car can only be front wheel drive, 4 wheel drive, or rear wheel drive.
 
 
-```r
+``` r
 unique(mpg$drv)
 ```
 
@@ -129,7 +129,7 @@ unique(mpg$drv)
 First note that we can use a single boxplot as an alternative to a histogram for visualizing a single numerical variable. To do so in `R`, we use the `boxplot()` function.
 
 
-```r
+``` r
 boxplot(mpg$hwy)
 ```
 
@@ -138,7 +138,7 @@ boxplot(mpg$hwy)
 However, more often we will use boxplots to compare a numerical variable for different values of a categorical variable.
 
 
-```r
+``` r
 boxplot(hwy ~ drv, data = mpg)
 ```
 
@@ -147,7 +147,7 @@ boxplot(hwy ~ drv, data = mpg)
 Here we used the `boxplot()` command to create side-by-side boxplots. However, since we are now dealing with two variables, the syntax has changed. The `R` syntax `hwy ~ drv, data = mpg` reads "Plot the `hwy` variable against the `drv` variable using the dataset `mpg`." We see the use of a `~` (which specifies a formula) and also a `data = ` argument. This will be a syntax that is common to many functions we will use in this course. 
 
 
-```r
+``` r
 boxplot(hwy ~ drv, data = mpg,
      xlab   = "Drivetrain (f = FWD, r = RWD, 4 = 4WD)",
      ylab   = "Miles Per Gallon (Highway)",
@@ -167,14 +167,14 @@ Again, `boxplot()` has a number of additional arguments which have the ability t
 Lastly, to visualize the relationship between two numeric variables we will use a **scatterplot**. This can be done with the `plot()` function and the `~` syntax we just used with a boxplot. (The function `plot()` can also be used more generally; see the documentation for details.)
 
 
-```r
+``` r
 plot(hwy ~ displ, data = mpg)
 ```
 
 ![](data-summary_files/figure-latex/unnamed-chunk-11-1.pdf)<!-- --> 
 
 
-```r
+``` r
 plot(hwy ~ displ, data = mpg,
      xlab = "Engine Displacement (in Liters)",
      ylab = "Miles Per Gallon (Highway)",
diff --git a/diagnostics.md b/diagnostics.md
index fa65b38..53bb52d 100644
--- a/diagnostics.md
+++ b/diagnostics.md
@@ -127,7 +127,7 @@ We'll now look at a number of tools for checking the assumptions of a linear mod
 \]
 
 
-```r
+``` r
 sim_1 = function(sample_size = 500) {
   x = runif(n = sample_size) * 5
   y = 3 + 5 * x + rnorm(n = sample_size, mean = 0, sd = 1)
@@ -154,7 +154,7 @@ Probably our most useful tool will be a **Fitted versus Residuals Plot**. It wil
 Data generated from Model 1 above should not show any signs of violating assumptions, so we'll use this to see what a good fitted versus residuals plot should look like. First, we'll simulate observations from this model.
 
 
-```r
+``` r
 set.seed(42)
 sim_data_1 = sim_1()
 head(sim_data_1)
@@ -173,7 +173,7 @@ head(sim_data_1)
 We then fit the model and add the fitted line to a scatterplot.
 
 
-```r
+``` r
 plot(y ~ x, data = sim_data_1, col = "grey", pch = 20,
      main = "Data from Model 1")
 fit_1 = lm(y ~ x, data = sim_data_1)
@@ -187,7 +187,7 @@ abline(fit_1, col = "darkorange", lwd = 3)
 We now plot a fitted versus residuals plot. Note, this is residuals on the $y$-axis despite the ordering in the name. Sometimes you will see this called a residuals versus fitted, or residuals versus predicted plot.
 
 
-```r
+``` r
 plot(fitted(fit_1), resid(fit_1), col = "grey", pch = 20,
      xlab = "Fitted", ylab = "Residuals", main = "Data from Model 1")
 abline(h = 0, col = "darkorange", lwd = 2)
@@ -209,7 +209,7 @@ To get a better idea of how a fitted versus residuals plot can be useful, we wil
 Model 2 is an example of non-constant variance. In this case, the variance is larger for larger values of the predictor variable $x$.
 
 
-```r
+``` r
 set.seed(42)
 sim_data_2 = sim_2()
 fit_2 = lm(y ~ x, data = sim_data_2)
@@ -225,7 +225,7 @@ abline(fit_2, col = "darkorange", lwd = 3)
 This actually is rather easy to see here by adding the fitted line to a scatterplot. This is because we are only performing simple linear regression. With multiple regression, a fitted versus residuals plot is a necessity, since adding a fitted regression to a scatterplot isn't exactly possible.
 
 
-```r
+``` r
 plot(fitted(fit_2), resid(fit_2), col = "grey", pch = 20,
      xlab = "Fitted", ylab = "Residuals", main = "Data from Model 2")
 abline(h = 0, col = "darkorange", lwd = 2)
@@ -240,7 +240,7 @@ On the fitted versus residuals plot, we see two things very clearly. For any fit
 Now we will demonstrate a model which does not meet the linearity assumption. Model 3 is an example of a model where $Y$ is not a linear combination of the predictors. In this case the predictor is $x$, but the model uses $x^2$. (We'll see later that this is something that a "linear" model can deal with. The fix is simple, just make $x^2$ a predictor!)
 
 
-```r
+``` r
 set.seed(42)
 sim_data_3 = sim_3()
 fit_3 = lm(y ~ x, data = sim_data_3)
@@ -256,7 +256,7 @@ abline(fit_3, col = "darkorange", lwd = 3)
 Again, this is rather clear on the scatterplot, but again, we wouldn't be able to check this plot for multiple regression.
 
 
-```r
+``` r
 plot(fitted(fit_3), resid(fit_3), col = "grey", pch = 20,
      xlab = "Fitted", ylab = "Residuals", main = "Data from Model 3")
 abline(h = 0, col = "darkorange", lwd = 2)
@@ -282,7 +282,7 @@ Isn't that convenient? A test that will specifically test the **constant varianc
 The Breusch-Pagan Test can not be performed by default in `R`, however the function `bptest` in the `lmtest` package implements the test.
 
 
-```r
+``` r
 #install.packages("lmtest")
 library(lmtest)
 ```
@@ -294,7 +294,7 @@ Let's try it on the three models we fit above. Recall,
 - `fit_3` violated linearity, but not constant variance.
 
 
-```r
+``` r
 bptest(fit_1)
 ```
 
@@ -309,7 +309,7 @@ bptest(fit_1)
 For `fit_1` we see a large p-value, so we do not reject the null of homoscedasticity, which is what we would expect.
 
 
-```r
+``` r
 bptest(fit_2)
 ```
 
@@ -324,7 +324,7 @@ bptest(fit_2)
 For `fit_2` we see a small p-value, so we reject the null of homoscedasticity. The constant variance assumption is violated. This matches our findings with a fitted versus residuals plot.
 
 
-```r
+``` r
 bptest(fit_3)
 ```
 
@@ -343,7 +343,7 @@ Lastly, for `fit_3` we again see a large p-value, so we do not reject the null o
 We have a number of tools for assessing the normality assumption. The most obvious would be to make a histogram of the residuals. If it appears roughly normal, then we'll believe the errors could truly be normal.
 
 
-```r
+``` r
 par(mfrow = c(1, 3))
 hist(resid(fit_1),
      xlab   = "Residuals",
@@ -378,7 +378,7 @@ Another visual method for assessing the normality of errors, which is more power
 In `R` these are very easy to make. The `qqnorm()` function plots the points, and the `qqline()` function adds the necessary line. We create a Q-Q plot for the residuals of `fit_1` to check if the errors could truly be normally distributed.
 
 
-```r
+``` r
 qqnorm(resid(fit_1), main = "Normal Q-Q Plot, fit_1", col = "darkgrey")
 qqline(resid(fit_1), col = "dodgerblue", lwd = 2)
 ```
@@ -396,7 +396,7 @@ The [Wikipedia page for Normal probability plots](http://en.wikipedia.org/wiki/N
 Also, to get a better idea of how Q-Q plots work, here is a quick function which creates a Q-Q plot:
 
 
-```r
+``` r
 qq_plot = function(e) {
 
   n = length(e)
@@ -422,7 +422,7 @@ qq_plot = function(e) {
 We can then verify that it is essentially equivalent to using `qqnorm()` and `qqline()` in `R`.
 
 
-```r
+``` r
 set.seed(420)
 x = rnorm(100, mean = 0 , sd = 1)
 par(mfrow = c(1, 2))
@@ -440,7 +440,7 @@ To get a better idea of what "close to the line" means, we perform a number of s
 First we simulate data from a normal distribution with different sample sizes, and each time create a Q-Q plot.
 
 
-```r
+``` r
 par(mfrow = c(1, 3))
 set.seed(420)
 qq_plot(rnorm(10))
@@ -457,7 +457,7 @@ Since this data **is** sampled from a normal distribution, these are all, by def
 Next, we simulate data from a $t$ distribution with a small degrees of freedom, for different sample sizes.
 
 
-```r
+``` r
 par(mfrow = c(1, 3))
 set.seed(420)
 qq_plot(rt(10, df = 4))
@@ -474,7 +474,7 @@ Recall that as the degrees of freedom for a $t$ distribution become larger, the
 Next, we simulate data from an exponential distribution.
 
 
-```r
+``` r
 par(mfrow = c(1, 3))
 set.seed(420)
 qq_plot(rexp(10))
@@ -499,7 +499,7 @@ Returning to our three regressions, recall,
 We'll now create a Q-Q plot for each to assess normality of errors.
 
 
-```r
+``` r
 qqnorm(resid(fit_1), main = "Normal Q-Q Plot, fit_1", col = "darkgrey")
 qqline(resid(fit_1), col = "dodgerblue", lwd = 2)
 ```
@@ -511,7 +511,7 @@ qqline(resid(fit_1), col = "dodgerblue", lwd = 2)
 For `fit_1`, we have a near perfect Q-Q plot. We would believe the errors follow a normal distribution.
 
 
-```r
+``` r
 qqnorm(resid(fit_2), main = "Normal Q-Q Plot, fit_2", col = "darkgrey")
 qqline(resid(fit_2), col = "dodgerblue", lwd = 2)
 ```
@@ -523,7 +523,7 @@ qqline(resid(fit_2), col = "dodgerblue", lwd = 2)
 For `fit_2`, we have a suspect Q-Q plot. We would probably **not** believe the errors follow a normal distribution.
 
 
-```r
+``` r
 qqnorm(resid(fit_3), main = "Normal Q-Q Plot, fit_3", col = "darkgrey")
 qqline(resid(fit_3), col = "dodgerblue", lwd = 2)
 ```
@@ -539,7 +539,7 @@ Lastly, for `fit_3`, we again have a suspect Q-Q plot. We would probably **not**
 Histograms and Q-Q Plots give a nice visual representation of the residuals distribution, however if we are interested in formal testing, there are a number of options available. A commonly used test is the **Shapiro–Wilk test**, which is implemented in `R`.
 
 
-```r
+``` r
 set.seed(42)
 shapiro.test(rnorm(25))
 ```
@@ -552,7 +552,7 @@ shapiro.test(rnorm(25))
 ## W = 0.9499, p-value = 0.2495
 ```
 
-```r
+``` r
 shapiro.test(rexp(25))
 ```
 
@@ -573,7 +573,7 @@ In the above examples, we see we fail to reject for the data sampled from normal
 Returning again to `fit_1`, `fit_2`, and `fit_3`, we see the result of running `shapiro.test()` on the residuals of each, returns a result for each that matches decisions based on the Q-Q plots.
 
 
-```r
+``` r
 shapiro.test(resid(fit_1))
 ```
 
@@ -586,7 +586,7 @@ shapiro.test(resid(fit_1))
 ```
 
 
-```r
+``` r
 shapiro.test(resid(fit_2))
 ```
 
@@ -599,7 +599,7 @@ shapiro.test(resid(fit_2))
 ```
 
 
-```r
+``` r
 shapiro.test(resid(fit_3))
 ```
 
@@ -618,7 +618,7 @@ In addition to checking the assumptions of regression, we also look for any "unu
 The following three plots are inspired by an example from [Linear Models with R](http://www.maths.bath.ac.uk/~jjf23/LMR/){target="_blank"}.
 
 
-```r
+``` r
 par(mfrow = c(1, 3))
 set.seed(42)
 ex_data  = data.frame(x = 1:10,
@@ -671,7 +671,7 @@ The blue solid line in each plot is a regression fit to the 10 original data poi
 The slope of the regression for the original ten points, the solid blue line, is given by:
 
 
-```r
+``` r
 coef(ex_model)[2]
 ```
 
@@ -683,7 +683,7 @@ coef(ex_model)[2]
 The added point in the first plot has a *small* effect on the slope, which becomes:
 
 
-```r
+``` r
 coef(model_1)[2]
 ```
 
@@ -697,7 +697,7 @@ We will say that this point has low leverage, is an outlier due to its large res
 The added point in the second plot also has a *small* effect on the slope, which is:
 
 
-```r
+``` r
 coef(model_2)[2]
 ```
 
@@ -711,7 +711,7 @@ We will say that this point has high leverage, is not an outlier due to its smal
 Lastly, the added point in the third plot has a *large* effect on the slope, which is now:
 
 
-```r
+``` r
 coef(model_3)[2]
 ```
 
@@ -793,7 +793,7 @@ This expression should be familiar. (Think back to inference for SLR.) It sugges
 There are multiple ways to find leverages in `R`.
 
 
-```r
+``` r
 lev_ex = data.frame(
   x1 = c(0, 11, 11, 7, 4, 10, 5, 8),
   x2 = c(1, 5, 4, 3, 1, 4, 4, 2),
@@ -812,7 +812,7 @@ Here we've created some multivariate data. Notice that we have plotted the $x$ v
 We could calculate the leverages using the expressions defined above. We first create the $X$ matrix, then calculate $H$ as defined, and extract the diagonal elements.
 
 
-```r
+``` r
 X = cbind(rep(1, 8), lev_ex$x1, lev_ex$x2)
 H = X %*% solve(t(X) %*% X) %*% t(X)
 diag(H)
@@ -825,7 +825,7 @@ diag(H)
 Notice here, we have two predictors, so the regression would have 3 $\beta$ parameters, so the sum of the diagonal elements is 3.
 
 
-```r
+``` r
 sum(diag(H))
 ```
 
@@ -836,7 +836,7 @@ sum(diag(H))
 Alternatively, the method we will use more often, is to simply fit a regression, then use the `hatvalues()` function, which returns the leverages.
 
 
-```r
+``` r
 lev_fit = lm(y ~ ., data = lev_ex)
 hatvalues(lev_fit)
 ```
@@ -849,7 +849,7 @@ hatvalues(lev_fit)
 Again, note that here we have "used" the $y$ values to fit the regression, but `R` still ignores them when calculating the leverages, as leverages only depend on the $x$ values.
 
 
-```r
+``` r
 coef(lev_fit)
 ```
 
@@ -862,7 +862,7 @@ coef(lev_fit)
 Let's see what happens to these coefficients when we modify the `y` value of the point with the highest leverage.
 
 
-```r
+``` r
 which.max(hatvalues(lev_fit))
 ```
 
@@ -871,7 +871,7 @@ which.max(hatvalues(lev_fit))
 ## 1
 ```
 
-```r
+``` r
 lev_ex[which.max(hatvalues(lev_fit)),]
 ```
 
@@ -883,7 +883,7 @@ lev_ex[which.max(hatvalues(lev_fit)),]
 We see that the original `y` value is 11. We'll create a copy of the data, and modify this point to have a `y` value of `20`.
 
 
-```r
+``` r
 lev_ex_1 = lev_ex
 lev_ex_1$y[1] = 20
 lm(y ~ ., data = lev_ex_1)
@@ -904,7 +904,7 @@ Notice the **large** changes in the coefficients. Also notice that each of the c
 Now let's see what happens to these coefficients when we modify the `y` value of the point with the lowest leverage.
 
 
-```r
+``` r
 which.min(hatvalues(lev_fit))
 ```
 
@@ -913,7 +913,7 @@ which.min(hatvalues(lev_fit))
 ## 4
 ```
 
-```r
+``` r
 lev_ex[which.min(hatvalues(lev_fit)),]
 ```
 
@@ -925,7 +925,7 @@ lev_ex[which.min(hatvalues(lev_fit)),]
 We see that the original `y` value is 14. We'll again create a copy of the data, and modify this point to have a `y` value of `30`.
 
 
-```r
+``` r
 lev_ex_2 = lev_ex
 lev_ex_2$y[4] = 30
 lm(y ~ ., data = lev_ex_2)
@@ -944,7 +944,7 @@ lm(y ~ ., data = lev_ex_2)
 This time despite a large change in the `y` value, there is only small change in the coefficients. Also, only the intercept has changed!
 
 
-```r
+``` r
 mean(lev_ex$x1)
 ```
 
@@ -952,7 +952,7 @@ mean(lev_ex$x1)
 ## [1] 7
 ```
 
-```r
+``` r
 mean(lev_ex$x2)
 ```
 
@@ -960,7 +960,7 @@ mean(lev_ex$x2)
 ## [1] 3
 ```
 
-```r
+``` r
 lev_ex[4,]
 ```
 
@@ -974,7 +974,7 @@ Notice that this point was the mean of both of the predictors.
 Returning to our three plots, each with an added point, we can calculate the leverages for each. Note that the 11th data point each time is the added data point.
 
 
-```r
+``` r
 hatvalues(model_1)
 ```
 
@@ -985,7 +985,7 @@ hatvalues(model_1)
 ## 0.16721022 0.24014985 0.33732922 0.09100926
 ```
 
-```r
+``` r
 hatvalues(model_2)
 ```
 
@@ -996,7 +996,7 @@ hatvalues(model_2)
 ## 0.09919028 0.11578947 0.14129555 0.66599190
 ```
 
-```r
+``` r
 hatvalues(model_3)
 ```
 
@@ -1010,7 +1010,7 @@ hatvalues(model_3)
 Are any of these large?
 
 
-```r
+``` r
 hatvalues(model_1) > 2 * mean(hatvalues(model_1))
 ```
 
@@ -1019,7 +1019,7 @@ hatvalues(model_1) > 2 * mean(hatvalues(model_1))
 ## FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
 ```
 
-```r
+``` r
 hatvalues(model_2) > 2 * mean(hatvalues(model_2))
 ```
 
@@ -1028,7 +1028,7 @@ hatvalues(model_2) > 2 * mean(hatvalues(model_2))
 ## FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE  TRUE
 ```
 
-```r
+``` r
 hatvalues(model_3) > 2 * mean(hatvalues(model_3))
 ```
 
@@ -1074,7 +1074,7 @@ We can use this fact to identify "large" residuals. For example, standardized re
 Returning again to our three plots, each with an added point, we can calculate the residuals and standardized residuals for each. Standardized residuals can be obtained in `R` by using `rstandard()` where we would normally use `resid()`.
 
 
-```r
+``` r
 resid(model_1)
 ```
 
@@ -1085,7 +1085,7 @@ resid(model_1)
 ## -1.1459548  0.9420814 -1.1641029  4.4138254
 ```
 
-```r
+``` r
 rstandard(model_1)
 ```
 
@@ -1096,7 +1096,7 @@ rstandard(model_1)
 ## -0.7165857  0.6167268 -0.8160389  2.6418234
 ```
 
-```r
+``` r
 rstandard(model_1)[abs(rstandard(model_1)) > 2]
 ```
 
@@ -1108,7 +1108,7 @@ rstandard(model_1)[abs(rstandard(model_1)) > 2]
 In the first plot, we see that the 11th point, the added point, is a large standardized residual.
 
 
-```r
+``` r
 resid(model_2)
 ```
 
@@ -1119,7 +1119,7 @@ resid(model_2)
 ##  0.87788484 -0.77755647  1.28626601 -0.84413207  0.12449986
 ```
 
-```r
+``` r
 rstandard(model_2)
 ```
 
@@ -1130,7 +1130,7 @@ rstandard(model_2)
 ##  1.10506546 -0.98294409  1.64121833 -1.09295417  0.25846620
 ```
 
-```r
+``` r
 rstandard(model_2)[abs(rstandard(model_2)) > 2]
 ```
 
@@ -1141,7 +1141,7 @@ rstandard(model_2)[abs(rstandard(model_2)) > 2]
 In the second plot, we see that there are no points with large standardized residuals.
 
 
-```r
+``` r
 resid(model_3)
 ```
 
@@ -1152,7 +1152,7 @@ resid(model_3)
 ## -0.02113027 -2.03808722 -0.33578039 -2.82769411  3.69191633
 ```
 
-```r
+``` r
 rstandard(model_3)
 ```
 
@@ -1163,7 +1163,7 @@ rstandard(model_3)
 ## -0.01157256 -1.12656475 -0.18882474 -1.63206526  2.70453408
 ```
 
-```r
+``` r
 rstandard(model_3)[abs(rstandard(model_3)) > 2]
 ```
 
@@ -1210,7 +1210,7 @@ Recall that the circled points in each plot have different characteristics:
 We'll now directly check if each of these is influential.
 
 
-```r
+``` r
 cooks.distance(model_1)[11] > 4 / length(cooks.distance(model_1))
 ```
 
@@ -1219,7 +1219,7 @@ cooks.distance(model_1)[11] > 4 / length(cooks.distance(model_1))
 ## FALSE
 ```
 
-```r
+``` r
 cooks.distance(model_2)[11] > 4 / length(cooks.distance(model_2))
 ```
 
@@ -1228,7 +1228,7 @@ cooks.distance(model_2)[11] > 4 / length(cooks.distance(model_2))
 ## FALSE
 ```
 
-```r
+``` r
 cooks.distance(model_3)[11] > 4 / length(cooks.distance(model_3))
 ```
 
@@ -1248,12 +1248,12 @@ Last chapter we fit an additive regression to the `mtcars` data with `mpg` as th
 First, fit the model as we did last chapter.
 
 
-```r
+``` r
 mpg_hp_add = lm(mpg ~ hp + am, data = mtcars)
 ```
 
 
-```r
+``` r
 plot(fitted(mpg_hp_add), resid(mpg_hp_add), col = "grey", pch = 20,
      xlab = "Fitted", ylab = "Residual",
      main = "mtcars: Fitted versus Residuals")
@@ -1267,7 +1267,7 @@ abline(h = 0, col = "darkorange", lwd = 2)
 The fitted versus residuals plot looks good. We don't see any obvious pattern, and the variance looks roughly constant. (Maybe a little larger for large fitted values, but not enough to worry about.)
 
 
-```r
+``` r
 bptest(mpg_hp_add)
 ```
 
@@ -1282,7 +1282,7 @@ bptest(mpg_hp_add)
 The Breusch-Pagan test verifies this, at least for a small $\alpha$ value.
 
 
-```r
+``` r
 qqnorm(resid(mpg_hp_add), col = "darkgrey")
 qqline(resid(mpg_hp_add), col = "dodgerblue", lwd = 2)
 ```
@@ -1294,7 +1294,7 @@ qqline(resid(mpg_hp_add), col = "dodgerblue", lwd = 2)
 The Q-Q plot looks extremely good and the Shapiro-Wilk test agrees.
 
 
-```r
+``` r
 shapiro.test(resid(mpg_hp_add))
 ```
 
@@ -1307,7 +1307,7 @@ shapiro.test(resid(mpg_hp_add))
 ```
 
 
-```r
+``` r
 sum(hatvalues(mpg_hp_add) > 2 * mean(hatvalues(mpg_hp_add)))
 ```
 
@@ -1318,7 +1318,7 @@ sum(hatvalues(mpg_hp_add) > 2 * mean(hatvalues(mpg_hp_add)))
 We see that there are two points of large leverage.
 
 
-```r
+``` r
 sum(abs(rstandard(mpg_hp_add)) > 2)
 ```
 
@@ -1329,7 +1329,7 @@ sum(abs(rstandard(mpg_hp_add)) > 2)
 There is also one point with a large residual. Do these result in any points that are considered influential?
 
 
-```r
+``` r
 cd_mpg_hp_add = cooks.distance(mpg_hp_add)
 sum(cd_mpg_hp_add > 4 / length(cd_mpg_hp_add))
 ```
@@ -1338,7 +1338,7 @@ sum(cd_mpg_hp_add > 4 / length(cd_mpg_hp_add))
 ## [1] 2
 ```
 
-```r
+``` r
 large_cd_mpg = cd_mpg_hp_add > 4 / length(cd_mpg_hp_add)
 cd_mpg_hp_add[large_cd_mpg]
 ```
@@ -1351,7 +1351,7 @@ cd_mpg_hp_add[large_cd_mpg]
 We find two influential points. Interestingly, they are **very** different cars.
 
 
-```r
+``` r
 coef(mpg_hp_add)
 ```
 
@@ -1363,7 +1363,7 @@ coef(mpg_hp_add)
 Since the diagnostics looked good, there isn't much need to worry about these two points, but let's see how much the coefficients change if we remove them.
 
 
-```r
+``` r
 mpg_hp_add_fix = lm(mpg ~ hp + am,
                     data = mtcars,
                     subset = cd_mpg_hp_add <= 4 / length(cd_mpg_hp_add))
@@ -1378,7 +1378,7 @@ coef(mpg_hp_add_fix)
 It seems there isn't much of a change in the coefficients as a result of removing the supposed influential points. Notice we did not create a new dataset to accomplish this. We instead used the `subset` argument to `lm()`. Think about what the code `cd_mpg_hp_add <= 4 / length(cd_mpg_hp_add)` does here.
 
 
-```r
+``` r
 par(mfrow = c(2, 2))
 plot(mpg_hp_add)
 ```
@@ -1396,7 +1396,7 @@ Let's consider the model `big_model` from last chapter which was fit to the `aut
 
 
 
-```r
+``` r
 str(autompg)
 ```
 
@@ -1414,12 +1414,12 @@ str(autompg)
 ```
 
 
-```r
+``` r
 big_model = lm(mpg ~ disp * hp * domestic, data = autompg)
 ```
 
 
-```r
+``` r
 qqnorm(resid(big_model), col = "darkgrey")
 qqline(resid(big_model), col = "dodgerblue", lwd = 2)
 ```
@@ -1428,7 +1428,7 @@ qqline(resid(big_model), col = "dodgerblue", lwd = 2)
 
 \begin{center}\includegraphics{diagnostics_files/figure-latex/unnamed-chunk-63-1} \end{center}
 
-```r
+``` r
 shapiro.test(resid(big_model))
 ```
 
@@ -1443,7 +1443,7 @@ shapiro.test(resid(big_model))
 Here both the Q-Q plot, and the Shapiro-Wilk test suggest that the normality assumption is violated.
 
 
-```r
+``` r
 big_mod_cd = cooks.distance(big_model)
 sum(big_mod_cd > 4 / length(big_mod_cd))
 ```
@@ -1455,7 +1455,7 @@ sum(big_mod_cd > 4 / length(big_mod_cd))
 Here, we find 31, so perhaps removing them will help!
 
 
-```r
+``` r
 big_model_fix = lm(mpg ~ disp * hp * domestic,
                    data = autompg,
                    subset = big_mod_cd < 4 / length(big_mod_cd))
@@ -1467,7 +1467,7 @@ qqline(resid(big_model_fix), col = "dodgerblue", lwd = 2)
 
 \begin{center}\includegraphics{diagnostics_files/figure-latex/unnamed-chunk-65-1} \end{center}
 
-```r
+``` r
 shapiro.test(resid(big_model_fix))
 ```
 
@@ -1489,4 +1489,4 @@ The `R` Markdown file for this chapter can be found here:
 
 - [`diagnostics.Rmd`](diagnostics.Rmd){target="_blank"}
 
-The file was created using `R` version `4.3.2`.
+The file was created using `R` version `4.4.1`.
diff --git a/index.html b/index.html
index 9698e48..95d0fd1 100644
--- a/index.html
+++ b/index.html
@@ -6,7 +6,7 @@
   <meta http-equiv="X-UA-Compatible" content="IE=edge" />
   <title>Applied Statistics with R</title>
   <meta name="description" content="Applied Statistics with R" />
-  <meta name="generator" content="bookdown 0.37 and GitBook 2.6.7" />
+  <meta name="generator" content="bookdown 0.40 and GitBook 2.6.7" />
 
   <meta property="og:title" content="Applied Statistics with R" />
   <meta property="og:type" content="book" />
@@ -55,7 +55,7 @@
 
 <style type="text/css">
 pre > code.sourceCode { white-space: pre; position: relative; }
-pre > code.sourceCode > span { display: inline-block; line-height: 1.25; }
+pre > code.sourceCode > span { line-height: 1.25; }
 pre > code.sourceCode > span:empty { height: 1.2em; }
 .sourceCode { overflow: visible; }
 code.sourceCode > span { color: inherit; text-decoration: inherit; }
@@ -484,9 +484,9 @@ <h2><span class="header-section-number">1.1</span> About This Book<a href="index
 <div id="conventions" class="section level2 hasAnchor" number="1.2">
 <h2><span class="header-section-number">1.2</span> Conventions<a href="index.html#conventions" class="anchor-section" aria-label="Anchor link to header"></a></h2>
 <p><code>R</code> code will be typeset using a <code>monospace</code> font which is syntax highlighted.</p>
-<div class="sourceCode" id="cb1"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1-1"><a href="index.html#cb1-1" aria-hidden="true" tabindex="-1"></a>a <span class="ot">=</span> <span class="dv">3</span></span>
-<span id="cb1-2"><a href="index.html#cb1-2" aria-hidden="true" tabindex="-1"></a>b <span class="ot">=</span> <span class="dv">4</span></span>
-<span id="cb1-3"><a href="index.html#cb1-3" aria-hidden="true" tabindex="-1"></a><span class="fu">sqrt</span>(a<span class="sc">^</span><span class="dv">2</span> <span class="sc">+</span> b<span class="sc">^</span><span class="dv">2</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1-1"><a href="index.html#cb1-1" tabindex="-1"></a>a <span class="ot">=</span> <span class="dv">3</span></span>
+<span id="cb1-2"><a href="index.html#cb1-2" tabindex="-1"></a>b <span class="ot">=</span> <span class="dv">4</span></span>
+<span id="cb1-3"><a href="index.html#cb1-3" tabindex="-1"></a><span class="fu">sqrt</span>(a<span class="sc">^</span><span class="dv">2</span> <span class="sc">+</span> b<span class="sc">^</span><span class="dv">2</span>)</span></code></pre></div>
 <p><code>R</code> output lines, which would appear in the console will begin with <code>##</code>. They will generally not be syntax highlighted.</p>
 <pre><code>## [1] 5</code></pre>
 <p>We use the quantity <span class="math inline">\(p\)</span> to refer to the number of <span class="math inline">\(\beta\)</span> parameters in a linear model, <strong>not</strong> the number of predictors. Don’t worry if you don’t know what this means yet!</p>
@@ -556,9 +556,9 @@ <h2><span class="header-section-number">1.3</span> Acknowledgements<a href="inde
 </div>
 <div id="license" class="section level2 hasAnchor" number="1.4">
 <h2><span class="header-section-number">1.4</span> License<a href="index.html#license" class="anchor-section" aria-label="Anchor link to header"></a></h2>
-<div class="figure">
-<img src="images/cc.png" alt="" />
-<p class="caption">This work is licensed under a <a href="http://creativecommons.org/licenses/by-nc-sa/4.0/">Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License</a>.</p>
+<div class="float">
+<img src="images/cc.png" alt="This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License." />
+<div class="figcaption">This work is licensed under a <a href="http://creativecommons.org/licenses/by-nc-sa/4.0/">Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License</a>.</div>
 </div>
 
 </div>
diff --git a/index.md b/index.md
index b3136fb..830d03b 100644
--- a/index.md
+++ b/index.md
@@ -37,7 +37,7 @@ $$
 `R` code will be typeset using a `monospace` font which is syntax highlighted.
 
 
-```r
+``` r
 a = 3
 b = 4
 sqrt(a^2 + b^2)
diff --git a/inference-for-simple-linear-regression.html b/inference-for-simple-linear-regression.html
index dfb995b..2575e39 100644
--- a/inference-for-simple-linear-regression.html
+++ b/inference-for-simple-linear-regression.html
@@ -6,7 +6,7 @@
   <meta http-equiv="X-UA-Compatible" content="IE=edge" />
   <title>Chapter 8 Inference for Simple Linear Regression | Applied Statistics with R</title>
   <meta name="description" content="Chapter 8 Inference for Simple Linear Regression | Applied Statistics with R" />
-  <meta name="generator" content="bookdown 0.37 and GitBook 2.6.7" />
+  <meta name="generator" content="bookdown 0.40 and GitBook 2.6.7" />
 
   <meta property="og:title" content="Chapter 8 Inference for Simple Linear Regression | Applied Statistics with R" />
   <meta property="og:type" content="book" />
@@ -55,7 +55,7 @@
 
 <style type="text/css">
 pre > code.sourceCode { white-space: pre; position: relative; }
-pre > code.sourceCode > span { display: inline-block; line-height: 1.25; }
+pre > code.sourceCode > span { line-height: 1.25; }
 pre > code.sourceCode > span:empty { height: 1.2em; }
 .sourceCode { overflow: visible; }
 code.sourceCode > span { color: inherit; text-decoration: inherit; }
@@ -497,8 +497,8 @@ <h1><span class="header-section-number">Chapter 8</span> Inference for Simple Li
 \]</span></p>
 <p>which we also called <span class="math inline">\(\text{RSE}\)</span>, for “Residual Standard Error.”</p>
 <p>When applied to the <code>cars</code> data, we obtained the following results:</p>
-<div class="sourceCode" id="cb525"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb525-1"><a href="inference-for-simple-linear-regression.html#cb525-1" aria-hidden="true" tabindex="-1"></a>stop_dist_model <span class="ot">=</span> <span class="fu">lm</span>(dist <span class="sc">~</span> speed, <span class="at">data =</span> cars)</span>
-<span id="cb525-2"><a href="inference-for-simple-linear-regression.html#cb525-2" aria-hidden="true" tabindex="-1"></a><span class="fu">summary</span>(stop_dist_model)</span></code></pre></div>
+<div class="sourceCode" id="cb525"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb525-1"><a href="inference-for-simple-linear-regression.html#cb525-1" tabindex="-1"></a>stop_dist_model <span class="ot">=</span> <span class="fu">lm</span>(dist <span class="sc">~</span> speed, <span class="at">data =</span> cars)</span>
+<span id="cb525-2"><a href="inference-for-simple-linear-regression.html#cb525-2" tabindex="-1"></a><span class="fu">summary</span>(stop_dist_model)</span></code></pre></div>
 <pre><code>## 
 ## Call:
 ## lm(formula = dist ~ speed, data = cars)
@@ -518,14 +518,14 @@ <h1><span class="header-section-number">Chapter 8</span> Inference for Simple Li
 ## Multiple R-squared:  0.6511, Adjusted R-squared:  0.6438 
 ## F-statistic: 89.57 on 1 and 48 DF,  p-value: 1.49e-12</code></pre>
 <p>Last chapter, we only discussed the <code>Estimate</code>, <code>Residual standard error</code>, and <code>Multiple R-squared</code> values. In this chapter, we will discuss all of the information under <code>Coefficients</code> as well as <code>F-statistic</code>.</p>
-<div class="sourceCode" id="cb527"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb527-1"><a href="inference-for-simple-linear-regression.html#cb527-1" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(dist <span class="sc">~</span> speed, <span class="at">data =</span> cars,</span>
-<span id="cb527-2"><a href="inference-for-simple-linear-regression.html#cb527-2" aria-hidden="true" tabindex="-1"></a>     <span class="at">xlab =</span> <span class="st">&quot;Speed (in Miles Per Hour)&quot;</span>,</span>
-<span id="cb527-3"><a href="inference-for-simple-linear-regression.html#cb527-3" aria-hidden="true" tabindex="-1"></a>     <span class="at">ylab =</span> <span class="st">&quot;Stopping Distance (in Feet)&quot;</span>,</span>
-<span id="cb527-4"><a href="inference-for-simple-linear-regression.html#cb527-4" aria-hidden="true" tabindex="-1"></a>     <span class="at">main =</span> <span class="st">&quot;Stopping Distance vs Speed&quot;</span>,</span>
-<span id="cb527-5"><a href="inference-for-simple-linear-regression.html#cb527-5" aria-hidden="true" tabindex="-1"></a>     <span class="at">pch  =</span> <span class="dv">20</span>,</span>
-<span id="cb527-6"><a href="inference-for-simple-linear-regression.html#cb527-6" aria-hidden="true" tabindex="-1"></a>     <span class="at">cex  =</span> <span class="dv">2</span>,</span>
-<span id="cb527-7"><a href="inference-for-simple-linear-regression.html#cb527-7" aria-hidden="true" tabindex="-1"></a>     <span class="at">col  =</span> <span class="st">&quot;grey&quot;</span>)</span>
-<span id="cb527-8"><a href="inference-for-simple-linear-regression.html#cb527-8" aria-hidden="true" tabindex="-1"></a><span class="fu">abline</span>(stop_dist_model, <span class="at">lwd =</span> <span class="dv">5</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb527"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb527-1"><a href="inference-for-simple-linear-regression.html#cb527-1" tabindex="-1"></a><span class="fu">plot</span>(dist <span class="sc">~</span> speed, <span class="at">data =</span> cars,</span>
+<span id="cb527-2"><a href="inference-for-simple-linear-regression.html#cb527-2" tabindex="-1"></a>     <span class="at">xlab =</span> <span class="st">&quot;Speed (in Miles Per Hour)&quot;</span>,</span>
+<span id="cb527-3"><a href="inference-for-simple-linear-regression.html#cb527-3" tabindex="-1"></a>     <span class="at">ylab =</span> <span class="st">&quot;Stopping Distance (in Feet)&quot;</span>,</span>
+<span id="cb527-4"><a href="inference-for-simple-linear-regression.html#cb527-4" tabindex="-1"></a>     <span class="at">main =</span> <span class="st">&quot;Stopping Distance vs Speed&quot;</span>,</span>
+<span id="cb527-5"><a href="inference-for-simple-linear-regression.html#cb527-5" tabindex="-1"></a>     <span class="at">pch  =</span> <span class="dv">20</span>,</span>
+<span id="cb527-6"><a href="inference-for-simple-linear-regression.html#cb527-6" tabindex="-1"></a>     <span class="at">cex  =</span> <span class="dv">2</span>,</span>
+<span id="cb527-7"><a href="inference-for-simple-linear-regression.html#cb527-7" tabindex="-1"></a>     <span class="at">col  =</span> <span class="st">&quot;grey&quot;</span>)</span>
+<span id="cb527-8"><a href="inference-for-simple-linear-regression.html#cb527-8" tabindex="-1"></a><span class="fu">abline</span>(stop_dist_model, <span class="at">lwd =</span> <span class="dv">5</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>)</span></code></pre></div>
 <p><img src="slr-inf_files/figure-html/unnamed-chunk-3-1.png" width="672" style="display: block; margin: auto;" /></p>
 <p>To get started, we’ll note that there is another equivalent expression for <span class="math inline">\(S_{xy}\)</span> which we did not see last chapter,</p>
 <p><span class="math display">\[
@@ -639,18 +639,18 @@ <h3><span class="header-section-number">8.2.1</span> Simulating Sampling Distrib
 \hat{\beta}_0 \sim N\left(  \beta_0, \sigma^2 \left(\frac{1}{n} + \frac{\bar{x}^2}{S_{xx}}\right) \right).
 \]</span></p>
 <p>First we need to decide ahead of time what our <span class="math inline">\(x\)</span> values will be for this simulation, since the <span class="math inline">\(x\)</span> values in SLR are also considered known quantities. The choice of <span class="math inline">\(x\)</span> values is arbitrary. Here we also set a seed for randomization, and calculate <span class="math inline">\(S_{xx}\)</span> which we will need going forward.</p>
-<div class="sourceCode" id="cb528"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb528-1"><a href="inference-for-simple-linear-regression.html#cb528-1" aria-hidden="true" tabindex="-1"></a><span class="fu">set.seed</span>(<span class="dv">42</span>)</span>
-<span id="cb528-2"><a href="inference-for-simple-linear-regression.html#cb528-2" aria-hidden="true" tabindex="-1"></a>sample_size <span class="ot">=</span> <span class="dv">100</span> <span class="co"># this is n</span></span>
-<span id="cb528-3"><a href="inference-for-simple-linear-regression.html#cb528-3" aria-hidden="true" tabindex="-1"></a>x <span class="ot">=</span> <span class="fu">seq</span>(<span class="sc">-</span><span class="dv">1</span>, <span class="dv">1</span>, <span class="at">length =</span> sample_size)</span>
-<span id="cb528-4"><a href="inference-for-simple-linear-regression.html#cb528-4" aria-hidden="true" tabindex="-1"></a>Sxx <span class="ot">=</span> <span class="fu">sum</span>((x <span class="sc">-</span> <span class="fu">mean</span>(x)) <span class="sc">^</span> <span class="dv">2</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb528"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb528-1"><a href="inference-for-simple-linear-regression.html#cb528-1" tabindex="-1"></a><span class="fu">set.seed</span>(<span class="dv">42</span>)</span>
+<span id="cb528-2"><a href="inference-for-simple-linear-regression.html#cb528-2" tabindex="-1"></a>sample_size <span class="ot">=</span> <span class="dv">100</span> <span class="co"># this is n</span></span>
+<span id="cb528-3"><a href="inference-for-simple-linear-regression.html#cb528-3" tabindex="-1"></a>x <span class="ot">=</span> <span class="fu">seq</span>(<span class="sc">-</span><span class="dv">1</span>, <span class="dv">1</span>, <span class="at">length =</span> sample_size)</span>
+<span id="cb528-4"><a href="inference-for-simple-linear-regression.html#cb528-4" tabindex="-1"></a>Sxx <span class="ot">=</span> <span class="fu">sum</span>((x <span class="sc">-</span> <span class="fu">mean</span>(x)) <span class="sc">^</span> <span class="dv">2</span>)</span></code></pre></div>
 <p>We also fix our parameter values.</p>
-<div class="sourceCode" id="cb529"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb529-1"><a href="inference-for-simple-linear-regression.html#cb529-1" aria-hidden="true" tabindex="-1"></a>beta_0 <span class="ot">=</span> <span class="dv">3</span></span>
-<span id="cb529-2"><a href="inference-for-simple-linear-regression.html#cb529-2" aria-hidden="true" tabindex="-1"></a>beta_1 <span class="ot">=</span> <span class="dv">6</span></span>
-<span id="cb529-3"><a href="inference-for-simple-linear-regression.html#cb529-3" aria-hidden="true" tabindex="-1"></a>sigma  <span class="ot">=</span> <span class="dv">2</span></span></code></pre></div>
+<div class="sourceCode" id="cb529"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb529-1"><a href="inference-for-simple-linear-regression.html#cb529-1" tabindex="-1"></a>beta_0 <span class="ot">=</span> <span class="dv">3</span></span>
+<span id="cb529-2"><a href="inference-for-simple-linear-regression.html#cb529-2" tabindex="-1"></a>beta_1 <span class="ot">=</span> <span class="dv">6</span></span>
+<span id="cb529-3"><a href="inference-for-simple-linear-regression.html#cb529-3" tabindex="-1"></a>sigma  <span class="ot">=</span> <span class="dv">2</span></span></code></pre></div>
 <p>With this information, we know the sampling distributions should be:</p>
-<div class="sourceCode" id="cb530"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb530-1"><a href="inference-for-simple-linear-regression.html#cb530-1" aria-hidden="true" tabindex="-1"></a>(<span class="at">var_beta_1_hat =</span> sigma <span class="sc">^</span> <span class="dv">2</span> <span class="sc">/</span> Sxx)</span></code></pre></div>
+<div class="sourceCode" id="cb530"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb530-1"><a href="inference-for-simple-linear-regression.html#cb530-1" tabindex="-1"></a>(<span class="at">var_beta_1_hat =</span> sigma <span class="sc">^</span> <span class="dv">2</span> <span class="sc">/</span> Sxx)</span></code></pre></div>
 <pre><code>## [1] 0.1176238</code></pre>
-<div class="sourceCode" id="cb532"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb532-1"><a href="inference-for-simple-linear-regression.html#cb532-1" aria-hidden="true" tabindex="-1"></a>(<span class="at">var_beta_0_hat =</span> sigma <span class="sc">^</span> <span class="dv">2</span> <span class="sc">*</span> (<span class="dv">1</span> <span class="sc">/</span> sample_size <span class="sc">+</span> <span class="fu">mean</span>(x) <span class="sc">^</span> <span class="dv">2</span> <span class="sc">/</span> Sxx))</span></code></pre></div>
+<div class="sourceCode" id="cb532"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb532-1"><a href="inference-for-simple-linear-regression.html#cb532-1" tabindex="-1"></a>(<span class="at">var_beta_0_hat =</span> sigma <span class="sc">^</span> <span class="dv">2</span> <span class="sc">*</span> (<span class="dv">1</span> <span class="sc">/</span> sample_size <span class="sc">+</span> <span class="fu">mean</span>(x) <span class="sc">^</span> <span class="dv">2</span> <span class="sc">/</span> Sxx))</span></code></pre></div>
 <pre><code>## [1] 0.04</code></pre>
 <p><span class="math display">\[
 \hat{\beta}_1 \sim N(  6, 0.1176238)
@@ -674,64 +674,64 @@ <h3><span class="header-section-number">8.2.1</span> Simulating Sampling Distrib
 \end{aligned}
 \]</span></p>
 <p>We now simulate data from this model 10,000 times. Note this may not be the most <code>R</code> way of doing the simulation. We perform the simulation in this manner in an attempt at clarity. For example, we could have used the <code>sim_slr()</code> function from the previous chapter. We also simply store variables in the global environment instead of creating a data frame for each new simulated dataset.</p>
-<div class="sourceCode" id="cb534"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb534-1"><a href="inference-for-simple-linear-regression.html#cb534-1" aria-hidden="true" tabindex="-1"></a>num_samples <span class="ot">=</span> <span class="dv">10000</span></span>
-<span id="cb534-2"><a href="inference-for-simple-linear-regression.html#cb534-2" aria-hidden="true" tabindex="-1"></a>beta_0_hats <span class="ot">=</span> <span class="fu">rep</span>(<span class="dv">0</span>, num_samples)</span>
-<span id="cb534-3"><a href="inference-for-simple-linear-regression.html#cb534-3" aria-hidden="true" tabindex="-1"></a>beta_1_hats <span class="ot">=</span> <span class="fu">rep</span>(<span class="dv">0</span>, num_samples)</span>
-<span id="cb534-4"><a href="inference-for-simple-linear-regression.html#cb534-4" aria-hidden="true" tabindex="-1"></a></span>
-<span id="cb534-5"><a href="inference-for-simple-linear-regression.html#cb534-5" aria-hidden="true" tabindex="-1"></a><span class="cf">for</span> (i <span class="cf">in</span> <span class="dv">1</span><span class="sc">:</span>num_samples) {</span>
-<span id="cb534-6"><a href="inference-for-simple-linear-regression.html#cb534-6" aria-hidden="true" tabindex="-1"></a>  eps <span class="ot">=</span> <span class="fu">rnorm</span>(sample_size, <span class="at">mean =</span> <span class="dv">0</span>, <span class="at">sd =</span> sigma)</span>
-<span id="cb534-7"><a href="inference-for-simple-linear-regression.html#cb534-7" aria-hidden="true" tabindex="-1"></a>  y   <span class="ot">=</span> beta_0 <span class="sc">+</span> beta_1 <span class="sc">*</span> x <span class="sc">+</span> eps</span>
-<span id="cb534-8"><a href="inference-for-simple-linear-regression.html#cb534-8" aria-hidden="true" tabindex="-1"></a>  </span>
-<span id="cb534-9"><a href="inference-for-simple-linear-regression.html#cb534-9" aria-hidden="true" tabindex="-1"></a>  sim_model <span class="ot">=</span> <span class="fu">lm</span>(y <span class="sc">~</span> x)</span>
-<span id="cb534-10"><a href="inference-for-simple-linear-regression.html#cb534-10" aria-hidden="true" tabindex="-1"></a>  </span>
-<span id="cb534-11"><a href="inference-for-simple-linear-regression.html#cb534-11" aria-hidden="true" tabindex="-1"></a>  beta_0_hats[i] <span class="ot">=</span> <span class="fu">coef</span>(sim_model)[<span class="dv">1</span>]</span>
-<span id="cb534-12"><a href="inference-for-simple-linear-regression.html#cb534-12" aria-hidden="true" tabindex="-1"></a>  beta_1_hats[i] <span class="ot">=</span> <span class="fu">coef</span>(sim_model)[<span class="dv">2</span>]</span>
-<span id="cb534-13"><a href="inference-for-simple-linear-regression.html#cb534-13" aria-hidden="true" tabindex="-1"></a>}</span></code></pre></div>
+<div class="sourceCode" id="cb534"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb534-1"><a href="inference-for-simple-linear-regression.html#cb534-1" tabindex="-1"></a>num_samples <span class="ot">=</span> <span class="dv">10000</span></span>
+<span id="cb534-2"><a href="inference-for-simple-linear-regression.html#cb534-2" tabindex="-1"></a>beta_0_hats <span class="ot">=</span> <span class="fu">rep</span>(<span class="dv">0</span>, num_samples)</span>
+<span id="cb534-3"><a href="inference-for-simple-linear-regression.html#cb534-3" tabindex="-1"></a>beta_1_hats <span class="ot">=</span> <span class="fu">rep</span>(<span class="dv">0</span>, num_samples)</span>
+<span id="cb534-4"><a href="inference-for-simple-linear-regression.html#cb534-4" tabindex="-1"></a></span>
+<span id="cb534-5"><a href="inference-for-simple-linear-regression.html#cb534-5" tabindex="-1"></a><span class="cf">for</span> (i <span class="cf">in</span> <span class="dv">1</span><span class="sc">:</span>num_samples) {</span>
+<span id="cb534-6"><a href="inference-for-simple-linear-regression.html#cb534-6" tabindex="-1"></a>  eps <span class="ot">=</span> <span class="fu">rnorm</span>(sample_size, <span class="at">mean =</span> <span class="dv">0</span>, <span class="at">sd =</span> sigma)</span>
+<span id="cb534-7"><a href="inference-for-simple-linear-regression.html#cb534-7" tabindex="-1"></a>  y   <span class="ot">=</span> beta_0 <span class="sc">+</span> beta_1 <span class="sc">*</span> x <span class="sc">+</span> eps</span>
+<span id="cb534-8"><a href="inference-for-simple-linear-regression.html#cb534-8" tabindex="-1"></a>  </span>
+<span id="cb534-9"><a href="inference-for-simple-linear-regression.html#cb534-9" tabindex="-1"></a>  sim_model <span class="ot">=</span> <span class="fu">lm</span>(y <span class="sc">~</span> x)</span>
+<span id="cb534-10"><a href="inference-for-simple-linear-regression.html#cb534-10" tabindex="-1"></a>  </span>
+<span id="cb534-11"><a href="inference-for-simple-linear-regression.html#cb534-11" tabindex="-1"></a>  beta_0_hats[i] <span class="ot">=</span> <span class="fu">coef</span>(sim_model)[<span class="dv">1</span>]</span>
+<span id="cb534-12"><a href="inference-for-simple-linear-regression.html#cb534-12" tabindex="-1"></a>  beta_1_hats[i] <span class="ot">=</span> <span class="fu">coef</span>(sim_model)[<span class="dv">2</span>]</span>
+<span id="cb534-13"><a href="inference-for-simple-linear-regression.html#cb534-13" tabindex="-1"></a>}</span></code></pre></div>
 <p>Each time we simulated the data, we obtained values of the estimated coefficients. The variables <code>beta_0_hats</code> and <code>beta_1_hats</code> now store 10,000 simulated values of <span class="math inline">\(\hat{\beta}_0\)</span> and <span class="math inline">\(\hat{\beta}_1\)</span> respectively.</p>
 <p>We first verify the distribution of <span class="math inline">\(\hat{\beta}_1\)</span>.</p>
-<div class="sourceCode" id="cb535"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb535-1"><a href="inference-for-simple-linear-regression.html#cb535-1" aria-hidden="true" tabindex="-1"></a><span class="fu">mean</span>(beta_1_hats) <span class="co"># empirical mean</span></span></code></pre></div>
+<div class="sourceCode" id="cb535"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb535-1"><a href="inference-for-simple-linear-regression.html#cb535-1" tabindex="-1"></a><span class="fu">mean</span>(beta_1_hats) <span class="co"># empirical mean</span></span></code></pre></div>
 <pre><code>## [1] 6.001998</code></pre>
-<div class="sourceCode" id="cb537"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb537-1"><a href="inference-for-simple-linear-regression.html#cb537-1" aria-hidden="true" tabindex="-1"></a>beta_1            <span class="co"># true mean</span></span></code></pre></div>
+<div class="sourceCode" id="cb537"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb537-1"><a href="inference-for-simple-linear-regression.html#cb537-1" tabindex="-1"></a>beta_1            <span class="co"># true mean</span></span></code></pre></div>
 <pre><code>## [1] 6</code></pre>
-<div class="sourceCode" id="cb539"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb539-1"><a href="inference-for-simple-linear-regression.html#cb539-1" aria-hidden="true" tabindex="-1"></a><span class="fu">var</span>(beta_1_hats)  <span class="co"># empirical variance</span></span></code></pre></div>
+<div class="sourceCode" id="cb539"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb539-1"><a href="inference-for-simple-linear-regression.html#cb539-1" tabindex="-1"></a><span class="fu">var</span>(beta_1_hats)  <span class="co"># empirical variance</span></span></code></pre></div>
 <pre><code>## [1] 0.11899</code></pre>
-<div class="sourceCode" id="cb541"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb541-1"><a href="inference-for-simple-linear-regression.html#cb541-1" aria-hidden="true" tabindex="-1"></a>var_beta_1_hat    <span class="co"># true variance</span></span></code></pre></div>
+<div class="sourceCode" id="cb541"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb541-1"><a href="inference-for-simple-linear-regression.html#cb541-1" tabindex="-1"></a>var_beta_1_hat    <span class="co"># true variance</span></span></code></pre></div>
 <pre><code>## [1] 0.1176238</code></pre>
 <p>We see that the empirical and true means and variances are <em>very</em> similar. We also verify that the empirical distribution is normal. To do so, we plot a histogram of the <code>beta_1_hats</code>, and add the curve for the true distribution of <span class="math inline">\(\hat{\beta}_1\)</span>. We use <code>prob = TRUE</code> to put the histogram on the same scale as the normal curve.</p>
-<div class="sourceCode" id="cb543"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb543-1"><a href="inference-for-simple-linear-regression.html#cb543-1" aria-hidden="true" tabindex="-1"></a><span class="co"># note need to use prob = TRUE</span></span>
-<span id="cb543-2"><a href="inference-for-simple-linear-regression.html#cb543-2" aria-hidden="true" tabindex="-1"></a><span class="fu">hist</span>(beta_1_hats, <span class="at">prob =</span> <span class="cn">TRUE</span>, <span class="at">breaks =</span> <span class="dv">20</span>, </span>
-<span id="cb543-3"><a href="inference-for-simple-linear-regression.html#cb543-3" aria-hidden="true" tabindex="-1"></a>     <span class="at">xlab =</span> <span class="fu">expression</span>(<span class="fu">hat</span>(beta)[<span class="dv">1</span>]), <span class="at">main =</span> <span class="st">&quot;&quot;</span>, <span class="at">border =</span> <span class="st">&quot;dodgerblue&quot;</span>)</span>
-<span id="cb543-4"><a href="inference-for-simple-linear-regression.html#cb543-4" aria-hidden="true" tabindex="-1"></a><span class="fu">curve</span>(<span class="fu">dnorm</span>(x, <span class="at">mean =</span> beta_1, <span class="at">sd =</span> <span class="fu">sqrt</span>(var_beta_1_hat)), </span>
-<span id="cb543-5"><a href="inference-for-simple-linear-regression.html#cb543-5" aria-hidden="true" tabindex="-1"></a>      <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">add =</span> <span class="cn">TRUE</span>, <span class="at">lwd =</span> <span class="dv">3</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb543"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb543-1"><a href="inference-for-simple-linear-regression.html#cb543-1" tabindex="-1"></a><span class="co"># note need to use prob = TRUE</span></span>
+<span id="cb543-2"><a href="inference-for-simple-linear-regression.html#cb543-2" tabindex="-1"></a><span class="fu">hist</span>(beta_1_hats, <span class="at">prob =</span> <span class="cn">TRUE</span>, <span class="at">breaks =</span> <span class="dv">20</span>, </span>
+<span id="cb543-3"><a href="inference-for-simple-linear-regression.html#cb543-3" tabindex="-1"></a>     <span class="at">xlab =</span> <span class="fu">expression</span>(<span class="fu">hat</span>(beta)[<span class="dv">1</span>]), <span class="at">main =</span> <span class="st">&quot;&quot;</span>, <span class="at">border =</span> <span class="st">&quot;dodgerblue&quot;</span>)</span>
+<span id="cb543-4"><a href="inference-for-simple-linear-regression.html#cb543-4" tabindex="-1"></a><span class="fu">curve</span>(<span class="fu">dnorm</span>(x, <span class="at">mean =</span> beta_1, <span class="at">sd =</span> <span class="fu">sqrt</span>(var_beta_1_hat)), </span>
+<span id="cb543-5"><a href="inference-for-simple-linear-regression.html#cb543-5" tabindex="-1"></a>      <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">add =</span> <span class="cn">TRUE</span>, <span class="at">lwd =</span> <span class="dv">3</span>)</span></code></pre></div>
 <p><img src="slr-inf_files/figure-html/unnamed-chunk-9-1.png" width="672" style="display: block; margin: auto;" /></p>
 <p>We then repeat the process for <span class="math inline">\(\hat{\beta}_0\)</span>.</p>
-<div class="sourceCode" id="cb544"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb544-1"><a href="inference-for-simple-linear-regression.html#cb544-1" aria-hidden="true" tabindex="-1"></a><span class="fu">mean</span>(beta_0_hats) <span class="co"># empirical mean</span></span></code></pre></div>
+<div class="sourceCode" id="cb544"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb544-1"><a href="inference-for-simple-linear-regression.html#cb544-1" tabindex="-1"></a><span class="fu">mean</span>(beta_0_hats) <span class="co"># empirical mean</span></span></code></pre></div>
 <pre><code>## [1] 3.001147</code></pre>
-<div class="sourceCode" id="cb546"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb546-1"><a href="inference-for-simple-linear-regression.html#cb546-1" aria-hidden="true" tabindex="-1"></a>beta_0            <span class="co"># true mean</span></span></code></pre></div>
+<div class="sourceCode" id="cb546"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb546-1"><a href="inference-for-simple-linear-regression.html#cb546-1" tabindex="-1"></a>beta_0            <span class="co"># true mean</span></span></code></pre></div>
 <pre><code>## [1] 3</code></pre>
-<div class="sourceCode" id="cb548"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb548-1"><a href="inference-for-simple-linear-regression.html#cb548-1" aria-hidden="true" tabindex="-1"></a><span class="fu">var</span>(beta_0_hats)  <span class="co"># empirical variance</span></span></code></pre></div>
+<div class="sourceCode" id="cb548"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb548-1"><a href="inference-for-simple-linear-regression.html#cb548-1" tabindex="-1"></a><span class="fu">var</span>(beta_0_hats)  <span class="co"># empirical variance</span></span></code></pre></div>
 <pre><code>## [1] 0.04017924</code></pre>
-<div class="sourceCode" id="cb550"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb550-1"><a href="inference-for-simple-linear-regression.html#cb550-1" aria-hidden="true" tabindex="-1"></a>var_beta_0_hat    <span class="co"># true variance</span></span></code></pre></div>
+<div class="sourceCode" id="cb550"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb550-1"><a href="inference-for-simple-linear-regression.html#cb550-1" tabindex="-1"></a>var_beta_0_hat    <span class="co"># true variance</span></span></code></pre></div>
 <pre><code>## [1] 0.04</code></pre>
-<div class="sourceCode" id="cb552"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb552-1"><a href="inference-for-simple-linear-regression.html#cb552-1" aria-hidden="true" tabindex="-1"></a><span class="fu">hist</span>(beta_0_hats, <span class="at">prob =</span> <span class="cn">TRUE</span>, <span class="at">breaks =</span> <span class="dv">25</span>, </span>
-<span id="cb552-2"><a href="inference-for-simple-linear-regression.html#cb552-2" aria-hidden="true" tabindex="-1"></a>     <span class="at">xlab =</span> <span class="fu">expression</span>(<span class="fu">hat</span>(beta)[<span class="dv">0</span>]), <span class="at">main =</span> <span class="st">&quot;&quot;</span>, <span class="at">border =</span> <span class="st">&quot;dodgerblue&quot;</span>)</span>
-<span id="cb552-3"><a href="inference-for-simple-linear-regression.html#cb552-3" aria-hidden="true" tabindex="-1"></a><span class="fu">curve</span>(<span class="fu">dnorm</span>(x, <span class="at">mean =</span> beta_0, <span class="at">sd =</span> <span class="fu">sqrt</span>(var_beta_0_hat)),</span>
-<span id="cb552-4"><a href="inference-for-simple-linear-regression.html#cb552-4" aria-hidden="true" tabindex="-1"></a>      <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">add =</span> <span class="cn">TRUE</span>, <span class="at">lwd =</span> <span class="dv">3</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb552"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb552-1"><a href="inference-for-simple-linear-regression.html#cb552-1" tabindex="-1"></a><span class="fu">hist</span>(beta_0_hats, <span class="at">prob =</span> <span class="cn">TRUE</span>, <span class="at">breaks =</span> <span class="dv">25</span>, </span>
+<span id="cb552-2"><a href="inference-for-simple-linear-regression.html#cb552-2" tabindex="-1"></a>     <span class="at">xlab =</span> <span class="fu">expression</span>(<span class="fu">hat</span>(beta)[<span class="dv">0</span>]), <span class="at">main =</span> <span class="st">&quot;&quot;</span>, <span class="at">border =</span> <span class="st">&quot;dodgerblue&quot;</span>)</span>
+<span id="cb552-3"><a href="inference-for-simple-linear-regression.html#cb552-3" tabindex="-1"></a><span class="fu">curve</span>(<span class="fu">dnorm</span>(x, <span class="at">mean =</span> beta_0, <span class="at">sd =</span> <span class="fu">sqrt</span>(var_beta_0_hat)),</span>
+<span id="cb552-4"><a href="inference-for-simple-linear-regression.html#cb552-4" tabindex="-1"></a>      <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">add =</span> <span class="cn">TRUE</span>, <span class="at">lwd =</span> <span class="dv">3</span>)</span></code></pre></div>
 <p><img src="slr-inf_files/figure-html/unnamed-chunk-11-1.png" width="672" style="display: block; margin: auto;" /></p>
 <p>In this simulation study, we have only simulated a finite number of samples. To truly verify the distributional results, we would need to observe an infinite number of samples. However, the following plot should make it clear that if we continued simulating, the empirical results would get closer and closer to what we should expect.</p>
-<div class="sourceCode" id="cb553"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb553-1"><a href="inference-for-simple-linear-regression.html#cb553-1" aria-hidden="true" tabindex="-1"></a><span class="fu">par</span>(<span class="at">mar =</span> <span class="fu">c</span>(<span class="dv">5</span>, <span class="dv">5</span>, <span class="dv">1</span>, <span class="dv">1</span>)) <span class="co"># adjusted plot margins, otherwise the &quot;hat&quot; does not display</span></span>
-<span id="cb553-2"><a href="inference-for-simple-linear-regression.html#cb553-2" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(<span class="fu">cumsum</span>(beta_1_hats) <span class="sc">/</span> (<span class="dv">1</span><span class="sc">:</span><span class="fu">length</span>(beta_1_hats)), <span class="at">type =</span> <span class="st">&quot;l&quot;</span>, <span class="at">ylim =</span> <span class="fu">c</span>(<span class="fl">5.95</span>, <span class="fl">6.05</span>),</span>
-<span id="cb553-3"><a href="inference-for-simple-linear-regression.html#cb553-3" aria-hidden="true" tabindex="-1"></a>     <span class="at">xlab =</span> <span class="st">&quot;Number of Simulations&quot;</span>,</span>
-<span id="cb553-4"><a href="inference-for-simple-linear-regression.html#cb553-4" aria-hidden="true" tabindex="-1"></a>     <span class="at">ylab =</span> <span class="fu">expression</span>(<span class="st">&quot;Empirical Mean of &quot;</span> <span class="sc">~</span> <span class="fu">hat</span>(beta)[<span class="dv">1</span>]),</span>
-<span id="cb553-5"><a href="inference-for-simple-linear-regression.html#cb553-5" aria-hidden="true" tabindex="-1"></a>     <span class="at">col  =</span> <span class="st">&quot;dodgerblue&quot;</span>)</span>
-<span id="cb553-6"><a href="inference-for-simple-linear-regression.html#cb553-6" aria-hidden="true" tabindex="-1"></a><span class="fu">abline</span>(<span class="at">h =</span> <span class="dv">6</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb553"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb553-1"><a href="inference-for-simple-linear-regression.html#cb553-1" tabindex="-1"></a><span class="fu">par</span>(<span class="at">mar =</span> <span class="fu">c</span>(<span class="dv">5</span>, <span class="dv">5</span>, <span class="dv">1</span>, <span class="dv">1</span>)) <span class="co"># adjusted plot margins, otherwise the &quot;hat&quot; does not display</span></span>
+<span id="cb553-2"><a href="inference-for-simple-linear-regression.html#cb553-2" tabindex="-1"></a><span class="fu">plot</span>(<span class="fu">cumsum</span>(beta_1_hats) <span class="sc">/</span> (<span class="dv">1</span><span class="sc">:</span><span class="fu">length</span>(beta_1_hats)), <span class="at">type =</span> <span class="st">&quot;l&quot;</span>, <span class="at">ylim =</span> <span class="fu">c</span>(<span class="fl">5.95</span>, <span class="fl">6.05</span>),</span>
+<span id="cb553-3"><a href="inference-for-simple-linear-regression.html#cb553-3" tabindex="-1"></a>     <span class="at">xlab =</span> <span class="st">&quot;Number of Simulations&quot;</span>,</span>
+<span id="cb553-4"><a href="inference-for-simple-linear-regression.html#cb553-4" tabindex="-1"></a>     <span class="at">ylab =</span> <span class="fu">expression</span>(<span class="st">&quot;Empirical Mean of &quot;</span> <span class="sc">~</span> <span class="fu">hat</span>(beta)[<span class="dv">1</span>]),</span>
+<span id="cb553-5"><a href="inference-for-simple-linear-regression.html#cb553-5" tabindex="-1"></a>     <span class="at">col  =</span> <span class="st">&quot;dodgerblue&quot;</span>)</span>
+<span id="cb553-6"><a href="inference-for-simple-linear-regression.html#cb553-6" tabindex="-1"></a><span class="fu">abline</span>(<span class="at">h =</span> <span class="dv">6</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
 <p><img src="slr-inf_files/figure-html/unnamed-chunk-12-1.png" width="672" style="display: block; margin: auto;" /></p>
-<div class="sourceCode" id="cb554"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb554-1"><a href="inference-for-simple-linear-regression.html#cb554-1" aria-hidden="true" tabindex="-1"></a><span class="fu">par</span>(<span class="at">mar =</span> <span class="fu">c</span>(<span class="dv">5</span>, <span class="dv">5</span>, <span class="dv">1</span>, <span class="dv">1</span>)) <span class="co"># adjusted plot margins, otherwise the &quot;hat&quot; does not display</span></span>
-<span id="cb554-2"><a href="inference-for-simple-linear-regression.html#cb554-2" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(<span class="fu">cumsum</span>(beta_0_hats) <span class="sc">/</span> (<span class="dv">1</span><span class="sc">:</span><span class="fu">length</span>(beta_0_hats)), <span class="at">type =</span> <span class="st">&quot;l&quot;</span>, <span class="at">ylim =</span> <span class="fu">c</span>(<span class="fl">2.95</span>, <span class="fl">3.05</span>),</span>
-<span id="cb554-3"><a href="inference-for-simple-linear-regression.html#cb554-3" aria-hidden="true" tabindex="-1"></a>     <span class="at">xlab =</span> <span class="st">&quot;Number of Simulations&quot;</span>,</span>
-<span id="cb554-4"><a href="inference-for-simple-linear-regression.html#cb554-4" aria-hidden="true" tabindex="-1"></a>     <span class="at">ylab =</span> <span class="fu">expression</span>(<span class="st">&quot;Empirical Mean of &quot;</span> <span class="sc">~</span> <span class="fu">hat</span>(beta)[<span class="dv">0</span>]),</span>
-<span id="cb554-5"><a href="inference-for-simple-linear-regression.html#cb554-5" aria-hidden="true" tabindex="-1"></a>     <span class="at">col  =</span> <span class="st">&quot;dodgerblue&quot;</span>)</span>
-<span id="cb554-6"><a href="inference-for-simple-linear-regression.html#cb554-6" aria-hidden="true" tabindex="-1"></a><span class="fu">abline</span>(<span class="at">h =</span> <span class="dv">3</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb554"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb554-1"><a href="inference-for-simple-linear-regression.html#cb554-1" tabindex="-1"></a><span class="fu">par</span>(<span class="at">mar =</span> <span class="fu">c</span>(<span class="dv">5</span>, <span class="dv">5</span>, <span class="dv">1</span>, <span class="dv">1</span>)) <span class="co"># adjusted plot margins, otherwise the &quot;hat&quot; does not display</span></span>
+<span id="cb554-2"><a href="inference-for-simple-linear-regression.html#cb554-2" tabindex="-1"></a><span class="fu">plot</span>(<span class="fu">cumsum</span>(beta_0_hats) <span class="sc">/</span> (<span class="dv">1</span><span class="sc">:</span><span class="fu">length</span>(beta_0_hats)), <span class="at">type =</span> <span class="st">&quot;l&quot;</span>, <span class="at">ylim =</span> <span class="fu">c</span>(<span class="fl">2.95</span>, <span class="fl">3.05</span>),</span>
+<span id="cb554-3"><a href="inference-for-simple-linear-regression.html#cb554-3" tabindex="-1"></a>     <span class="at">xlab =</span> <span class="st">&quot;Number of Simulations&quot;</span>,</span>
+<span id="cb554-4"><a href="inference-for-simple-linear-regression.html#cb554-4" tabindex="-1"></a>     <span class="at">ylab =</span> <span class="fu">expression</span>(<span class="st">&quot;Empirical Mean of &quot;</span> <span class="sc">~</span> <span class="fu">hat</span>(beta)[<span class="dv">0</span>]),</span>
+<span id="cb554-5"><a href="inference-for-simple-linear-regression.html#cb554-5" tabindex="-1"></a>     <span class="at">col  =</span> <span class="st">&quot;dodgerblue&quot;</span>)</span>
+<span id="cb554-6"><a href="inference-for-simple-linear-regression.html#cb554-6" tabindex="-1"></a><span class="fu">abline</span>(<span class="at">h =</span> <span class="dv">3</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
 <p><img src="slr-inf_files/figure-html/unnamed-chunk-12-2.png" width="672" style="display: block; margin: auto;" /></p>
 </div>
 </div>
@@ -804,20 +804,20 @@ <h2><span class="header-section-number">8.3</span> Standard Errors<a href="infer
 \]</span></p>
 <p>where <span class="math inline">\(Z \sim N(0,1)\)</span>.</p>
 <p>Recall that a <span class="math inline">\(t\)</span> distribution is similar to a standard normal, but with heavier tails. As the degrees of freedom increases, the <span class="math inline">\(t\)</span> distribution becomes more and more like a standard normal. Below we plot a standard normal distribution as well as two examples of a <span class="math inline">\(t\)</span> distribution with different degrees of freedom. Notice how the <span class="math inline">\(t\)</span> distribution with the larger degrees of freedom is more similar to the standard normal curve.</p>
-<div class="sourceCode" id="cb555"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb555-1"><a href="inference-for-simple-linear-regression.html#cb555-1" aria-hidden="true" tabindex="-1"></a><span class="co"># define grid of x values</span></span>
-<span id="cb555-2"><a href="inference-for-simple-linear-regression.html#cb555-2" aria-hidden="true" tabindex="-1"></a>x <span class="ot">=</span> <span class="fu">seq</span>(<span class="sc">-</span><span class="dv">4</span>, <span class="dv">4</span>, <span class="at">length =</span> <span class="dv">100</span>)</span>
-<span id="cb555-3"><a href="inference-for-simple-linear-regression.html#cb555-3" aria-hidden="true" tabindex="-1"></a></span>
-<span id="cb555-4"><a href="inference-for-simple-linear-regression.html#cb555-4" aria-hidden="true" tabindex="-1"></a><span class="co"># plot curve for standard normal</span></span>
-<span id="cb555-5"><a href="inference-for-simple-linear-regression.html#cb555-5" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(x, <span class="fu">dnorm</span>(x), <span class="at">type =</span> <span class="st">&quot;l&quot;</span>, <span class="at">lty =</span> <span class="dv">1</span>, <span class="at">lwd =</span> <span class="dv">2</span>,</span>
-<span id="cb555-6"><a href="inference-for-simple-linear-regression.html#cb555-6" aria-hidden="true" tabindex="-1"></a>     <span class="at">xlab =</span> <span class="st">&quot;x&quot;</span>, <span class="at">ylab =</span> <span class="st">&quot;Density&quot;</span>, <span class="at">main =</span> <span class="st">&quot;Normal vs t Distributions&quot;</span>)</span>
-<span id="cb555-7"><a href="inference-for-simple-linear-regression.html#cb555-7" aria-hidden="true" tabindex="-1"></a><span class="co"># add curves for t distributions</span></span>
-<span id="cb555-8"><a href="inference-for-simple-linear-regression.html#cb555-8" aria-hidden="true" tabindex="-1"></a><span class="fu">lines</span>(x, <span class="fu">dt</span>(x, <span class="at">df =</span> <span class="dv">1</span>), <span class="at">lty =</span> <span class="dv">3</span>, <span class="at">lwd =</span> <span class="dv">2</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>)</span>
-<span id="cb555-9"><a href="inference-for-simple-linear-regression.html#cb555-9" aria-hidden="true" tabindex="-1"></a><span class="fu">lines</span>(x, <span class="fu">dt</span>(x, <span class="at">df =</span> <span class="dv">10</span>), <span class="at">lty =</span> <span class="dv">2</span>, <span class="at">lwd =</span> <span class="dv">2</span>, <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>)</span>
-<span id="cb555-10"><a href="inference-for-simple-linear-regression.html#cb555-10" aria-hidden="true" tabindex="-1"></a></span>
-<span id="cb555-11"><a href="inference-for-simple-linear-regression.html#cb555-11" aria-hidden="true" tabindex="-1"></a><span class="co"># add legend</span></span>
-<span id="cb555-12"><a href="inference-for-simple-linear-regression.html#cb555-12" aria-hidden="true" tabindex="-1"></a><span class="fu">legend</span>(<span class="st">&quot;topright&quot;</span>, <span class="at">title =</span> <span class="st">&quot;Distributions&quot;</span>,</span>
-<span id="cb555-13"><a href="inference-for-simple-linear-regression.html#cb555-13" aria-hidden="true" tabindex="-1"></a>       <span class="at">legend =</span> <span class="fu">c</span>(<span class="st">&quot;t, df = 1&quot;</span>, <span class="st">&quot;t, df = 10&quot;</span>, <span class="st">&quot;Standard Normal&quot;</span>), </span>
-<span id="cb555-14"><a href="inference-for-simple-linear-regression.html#cb555-14" aria-hidden="true" tabindex="-1"></a>       <span class="at">lwd =</span> <span class="dv">2</span>, <span class="at">lty =</span> <span class="fu">c</span>(<span class="dv">3</span>, <span class="dv">2</span>, <span class="dv">1</span>), <span class="at">col =</span> <span class="fu">c</span>(<span class="st">&quot;darkorange&quot;</span>, <span class="st">&quot;dodgerblue&quot;</span>, <span class="st">&quot;black&quot;</span>))</span></code></pre></div>
+<div class="sourceCode" id="cb555"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb555-1"><a href="inference-for-simple-linear-regression.html#cb555-1" tabindex="-1"></a><span class="co"># define grid of x values</span></span>
+<span id="cb555-2"><a href="inference-for-simple-linear-regression.html#cb555-2" tabindex="-1"></a>x <span class="ot">=</span> <span class="fu">seq</span>(<span class="sc">-</span><span class="dv">4</span>, <span class="dv">4</span>, <span class="at">length =</span> <span class="dv">100</span>)</span>
+<span id="cb555-3"><a href="inference-for-simple-linear-regression.html#cb555-3" tabindex="-1"></a></span>
+<span id="cb555-4"><a href="inference-for-simple-linear-regression.html#cb555-4" tabindex="-1"></a><span class="co"># plot curve for standard normal</span></span>
+<span id="cb555-5"><a href="inference-for-simple-linear-regression.html#cb555-5" tabindex="-1"></a><span class="fu">plot</span>(x, <span class="fu">dnorm</span>(x), <span class="at">type =</span> <span class="st">&quot;l&quot;</span>, <span class="at">lty =</span> <span class="dv">1</span>, <span class="at">lwd =</span> <span class="dv">2</span>,</span>
+<span id="cb555-6"><a href="inference-for-simple-linear-regression.html#cb555-6" tabindex="-1"></a>     <span class="at">xlab =</span> <span class="st">&quot;x&quot;</span>, <span class="at">ylab =</span> <span class="st">&quot;Density&quot;</span>, <span class="at">main =</span> <span class="st">&quot;Normal vs t Distributions&quot;</span>)</span>
+<span id="cb555-7"><a href="inference-for-simple-linear-regression.html#cb555-7" tabindex="-1"></a><span class="co"># add curves for t distributions</span></span>
+<span id="cb555-8"><a href="inference-for-simple-linear-regression.html#cb555-8" tabindex="-1"></a><span class="fu">lines</span>(x, <span class="fu">dt</span>(x, <span class="at">df =</span> <span class="dv">1</span>), <span class="at">lty =</span> <span class="dv">3</span>, <span class="at">lwd =</span> <span class="dv">2</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>)</span>
+<span id="cb555-9"><a href="inference-for-simple-linear-regression.html#cb555-9" tabindex="-1"></a><span class="fu">lines</span>(x, <span class="fu">dt</span>(x, <span class="at">df =</span> <span class="dv">10</span>), <span class="at">lty =</span> <span class="dv">2</span>, <span class="at">lwd =</span> <span class="dv">2</span>, <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>)</span>
+<span id="cb555-10"><a href="inference-for-simple-linear-regression.html#cb555-10" tabindex="-1"></a></span>
+<span id="cb555-11"><a href="inference-for-simple-linear-regression.html#cb555-11" tabindex="-1"></a><span class="co"># add legend</span></span>
+<span id="cb555-12"><a href="inference-for-simple-linear-regression.html#cb555-12" tabindex="-1"></a><span class="fu">legend</span>(<span class="st">&quot;topright&quot;</span>, <span class="at">title =</span> <span class="st">&quot;Distributions&quot;</span>,</span>
+<span id="cb555-13"><a href="inference-for-simple-linear-regression.html#cb555-13" tabindex="-1"></a>       <span class="at">legend =</span> <span class="fu">c</span>(<span class="st">&quot;t, df = 1&quot;</span>, <span class="st">&quot;t, df = 10&quot;</span>, <span class="st">&quot;Standard Normal&quot;</span>), </span>
+<span id="cb555-14"><a href="inference-for-simple-linear-regression.html#cb555-14" tabindex="-1"></a>       <span class="at">lwd =</span> <span class="dv">2</span>, <span class="at">lty =</span> <span class="fu">c</span>(<span class="dv">3</span>, <span class="dv">2</span>, <span class="dv">1</span>), <span class="at">col =</span> <span class="fu">c</span>(<span class="st">&quot;darkorange&quot;</span>, <span class="st">&quot;dodgerblue&quot;</span>, <span class="st">&quot;black&quot;</span>))</span></code></pre></div>
 <p><img src="slr-inf_files/figure-html/unnamed-chunk-13-1.png" width="672" style="display: block; margin: auto;" /></p>
 </div>
 <div id="confidence-intervals-for-slope-and-intercept" class="section level2 hasAnchor" number="8.4">
@@ -874,8 +874,8 @@ <h2><span class="header-section-number">8.5</span> Hypothesis Tests<a href="infe
 <div id="cars-example" class="section level2 hasAnchor" number="8.6">
 <h2><span class="header-section-number">8.6</span> <code>cars</code> Example<a href="inference-for-simple-linear-regression.html#cars-example" class="anchor-section" aria-label="Anchor link to header"></a></h2>
 <p>We now return to the <code>cars</code> example from last chapter to illustrate these concepts. We first fit the model using <code>lm()</code> then use <code>summary()</code> to view the results in greater detail.</p>
-<div class="sourceCode" id="cb556"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb556-1"><a href="inference-for-simple-linear-regression.html#cb556-1" aria-hidden="true" tabindex="-1"></a>stop_dist_model <span class="ot">=</span> <span class="fu">lm</span>(dist <span class="sc">~</span> speed, <span class="at">data =</span> cars)</span>
-<span id="cb556-2"><a href="inference-for-simple-linear-regression.html#cb556-2" aria-hidden="true" tabindex="-1"></a><span class="fu">summary</span>(stop_dist_model)</span></code></pre></div>
+<div class="sourceCode" id="cb556"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb556-1"><a href="inference-for-simple-linear-regression.html#cb556-1" tabindex="-1"></a>stop_dist_model <span class="ot">=</span> <span class="fu">lm</span>(dist <span class="sc">~</span> speed, <span class="at">data =</span> cars)</span>
+<span id="cb556-2"><a href="inference-for-simple-linear-regression.html#cb556-2" tabindex="-1"></a><span class="fu">summary</span>(stop_dist_model)</span></code></pre></div>
 <pre><code>## 
 ## Call:
 ## lm(formula = dist ~ speed, data = cars)
@@ -897,11 +897,11 @@ <h2><span class="header-section-number">8.6</span> <code>cars</code> Example<a h
 <div id="tests-in-r" class="section level3 hasAnchor" number="8.6.1">
 <h3><span class="header-section-number">8.6.1</span> Tests in <code>R</code><a href="inference-for-simple-linear-regression.html#tests-in-r" class="anchor-section" aria-label="Anchor link to header"></a></h3>
 <p>We will now discuss the results displayed called <code>Coefficients</code>. First recall that we can extract this information directly.</p>
-<div class="sourceCode" id="cb558"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb558-1"><a href="inference-for-simple-linear-regression.html#cb558-1" aria-hidden="true" tabindex="-1"></a><span class="fu">names</span>(<span class="fu">summary</span>(stop_dist_model))</span></code></pre></div>
+<div class="sourceCode" id="cb558"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb558-1"><a href="inference-for-simple-linear-regression.html#cb558-1" tabindex="-1"></a><span class="fu">names</span>(<span class="fu">summary</span>(stop_dist_model))</span></code></pre></div>
 <pre><code>##  [1] &quot;call&quot;          &quot;terms&quot;         &quot;residuals&quot;     &quot;coefficients&quot; 
 ##  [5] &quot;aliased&quot;       &quot;sigma&quot;         &quot;df&quot;            &quot;r.squared&quot;    
 ##  [9] &quot;adj.r.squared&quot; &quot;fstatistic&quot;    &quot;cov.unscaled&quot;</code></pre>
-<div class="sourceCode" id="cb560"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb560-1"><a href="inference-for-simple-linear-regression.html#cb560-1" aria-hidden="true" tabindex="-1"></a><span class="fu">summary</span>(stop_dist_model)<span class="sc">$</span>coefficients</span></code></pre></div>
+<div class="sourceCode" id="cb560"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb560-1"><a href="inference-for-simple-linear-regression.html#cb560-1" tabindex="-1"></a><span class="fu">summary</span>(stop_dist_model)<span class="sc">$</span>coefficients</span></code></pre></div>
 <pre><code>##               Estimate Std. Error   t value     Pr(&gt;|t|)
 ## (Intercept) -17.579095  6.7584402 -2.601058 1.231882e-02
 ## speed         3.932409  0.4155128  9.463990 1.489836e-12</code></pre>
@@ -915,7 +915,7 @@ <h3><span class="header-section-number">8.6.1</span> Tests in <code>R</code><a h
 \]</span></p>
 <p>which are our estimates for the model parameters <span class="math inline">\(\beta_0\)</span> and <span class="math inline">\(\beta_1\)</span>.</p>
 <p>Let’s now focus on the second row of output, which is relevant to <span class="math inline">\(\beta_1\)</span>.</p>
-<div class="sourceCode" id="cb562"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb562-1"><a href="inference-for-simple-linear-regression.html#cb562-1" aria-hidden="true" tabindex="-1"></a><span class="fu">summary</span>(stop_dist_model)<span class="sc">$</span>coefficients[<span class="dv">2</span>,]</span></code></pre></div>
+<div class="sourceCode" id="cb562"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb562-1"><a href="inference-for-simple-linear-regression.html#cb562-1" tabindex="-1"></a><span class="fu">summary</span>(stop_dist_model)<span class="sc">$</span>coefficients[<span class="dv">2</span>,]</span></code></pre></div>
 <pre><code>##     Estimate   Std. Error      t value     Pr(&gt;|t|) 
 ## 3.932409e+00 4.155128e-01 9.463990e+00 1.489836e-12</code></pre>
 <p>Again, the first value, <code>Estimate</code> is</p>
@@ -936,29 +936,29 @@ <h3><span class="header-section-number">8.6.1</span> Tests in <code>R</code><a h
 \]</span></p>
 <p>Note here, we are specifically testing whether or not <span class="math inline">\(\beta_1 = 0\)</span>.</p>
 <p>The first row of output reports the same values, but for <span class="math inline">\(\beta_0\)</span>.</p>
-<div class="sourceCode" id="cb564"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb564-1"><a href="inference-for-simple-linear-regression.html#cb564-1" aria-hidden="true" tabindex="-1"></a><span class="fu">summary</span>(stop_dist_model)<span class="sc">$</span>coefficients[<span class="dv">1</span>,]</span></code></pre></div>
+<div class="sourceCode" id="cb564"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb564-1"><a href="inference-for-simple-linear-regression.html#cb564-1" tabindex="-1"></a><span class="fu">summary</span>(stop_dist_model)<span class="sc">$</span>coefficients[<span class="dv">1</span>,]</span></code></pre></div>
 <pre><code>##     Estimate   Std. Error      t value     Pr(&gt;|t|) 
 ## -17.57909489   6.75844017  -2.60105800   0.01231882</code></pre>
 <p>In summary, the following code stores the information of <code>summary(stop_dist_model)$coefficients</code> in a new variable <code>stop_dist_model_test_info</code>, then extracts each element into a new variable which describes the information it contains.</p>
-<div class="sourceCode" id="cb566"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb566-1"><a href="inference-for-simple-linear-regression.html#cb566-1" aria-hidden="true" tabindex="-1"></a>stop_dist_model_test_info <span class="ot">=</span> <span class="fu">summary</span>(stop_dist_model)<span class="sc">$</span>coefficients</span>
-<span id="cb566-2"><a href="inference-for-simple-linear-regression.html#cb566-2" aria-hidden="true" tabindex="-1"></a></span>
-<span id="cb566-3"><a href="inference-for-simple-linear-regression.html#cb566-3" aria-hidden="true" tabindex="-1"></a>beta_0_hat      <span class="ot">=</span> stop_dist_model_test_info[<span class="dv">1</span>, <span class="dv">1</span>] <span class="co"># Estimate</span></span>
-<span id="cb566-4"><a href="inference-for-simple-linear-regression.html#cb566-4" aria-hidden="true" tabindex="-1"></a>beta_0_hat_se   <span class="ot">=</span> stop_dist_model_test_info[<span class="dv">1</span>, <span class="dv">2</span>] <span class="co"># Std. Error</span></span>
-<span id="cb566-5"><a href="inference-for-simple-linear-regression.html#cb566-5" aria-hidden="true" tabindex="-1"></a>beta_0_hat_t    <span class="ot">=</span> stop_dist_model_test_info[<span class="dv">1</span>, <span class="dv">3</span>] <span class="co"># t value</span></span>
-<span id="cb566-6"><a href="inference-for-simple-linear-regression.html#cb566-6" aria-hidden="true" tabindex="-1"></a>beta_0_hat_pval <span class="ot">=</span> stop_dist_model_test_info[<span class="dv">1</span>, <span class="dv">4</span>] <span class="co"># Pr(&gt;|t|)</span></span>
-<span id="cb566-7"><a href="inference-for-simple-linear-regression.html#cb566-7" aria-hidden="true" tabindex="-1"></a></span>
-<span id="cb566-8"><a href="inference-for-simple-linear-regression.html#cb566-8" aria-hidden="true" tabindex="-1"></a>beta_1_hat      <span class="ot">=</span> stop_dist_model_test_info[<span class="dv">2</span>, <span class="dv">1</span>] <span class="co"># Estimate</span></span>
-<span id="cb566-9"><a href="inference-for-simple-linear-regression.html#cb566-9" aria-hidden="true" tabindex="-1"></a>beta_1_hat_se   <span class="ot">=</span> stop_dist_model_test_info[<span class="dv">2</span>, <span class="dv">2</span>] <span class="co"># Std. Error</span></span>
-<span id="cb566-10"><a href="inference-for-simple-linear-regression.html#cb566-10" aria-hidden="true" tabindex="-1"></a>beta_1_hat_t    <span class="ot">=</span> stop_dist_model_test_info[<span class="dv">2</span>, <span class="dv">3</span>] <span class="co"># t value</span></span>
-<span id="cb566-11"><a href="inference-for-simple-linear-regression.html#cb566-11" aria-hidden="true" tabindex="-1"></a>beta_1_hat_pval <span class="ot">=</span> stop_dist_model_test_info[<span class="dv">2</span>, <span class="dv">4</span>] <span class="co"># Pr(&gt;|t|)</span></span></code></pre></div>
+<div class="sourceCode" id="cb566"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb566-1"><a href="inference-for-simple-linear-regression.html#cb566-1" tabindex="-1"></a>stop_dist_model_test_info <span class="ot">=</span> <span class="fu">summary</span>(stop_dist_model)<span class="sc">$</span>coefficients</span>
+<span id="cb566-2"><a href="inference-for-simple-linear-regression.html#cb566-2" tabindex="-1"></a></span>
+<span id="cb566-3"><a href="inference-for-simple-linear-regression.html#cb566-3" tabindex="-1"></a>beta_0_hat      <span class="ot">=</span> stop_dist_model_test_info[<span class="dv">1</span>, <span class="dv">1</span>] <span class="co"># Estimate</span></span>
+<span id="cb566-4"><a href="inference-for-simple-linear-regression.html#cb566-4" tabindex="-1"></a>beta_0_hat_se   <span class="ot">=</span> stop_dist_model_test_info[<span class="dv">1</span>, <span class="dv">2</span>] <span class="co"># Std. Error</span></span>
+<span id="cb566-5"><a href="inference-for-simple-linear-regression.html#cb566-5" tabindex="-1"></a>beta_0_hat_t    <span class="ot">=</span> stop_dist_model_test_info[<span class="dv">1</span>, <span class="dv">3</span>] <span class="co"># t value</span></span>
+<span id="cb566-6"><a href="inference-for-simple-linear-regression.html#cb566-6" tabindex="-1"></a>beta_0_hat_pval <span class="ot">=</span> stop_dist_model_test_info[<span class="dv">1</span>, <span class="dv">4</span>] <span class="co"># Pr(&gt;|t|)</span></span>
+<span id="cb566-7"><a href="inference-for-simple-linear-regression.html#cb566-7" tabindex="-1"></a></span>
+<span id="cb566-8"><a href="inference-for-simple-linear-regression.html#cb566-8" tabindex="-1"></a>beta_1_hat      <span class="ot">=</span> stop_dist_model_test_info[<span class="dv">2</span>, <span class="dv">1</span>] <span class="co"># Estimate</span></span>
+<span id="cb566-9"><a href="inference-for-simple-linear-regression.html#cb566-9" tabindex="-1"></a>beta_1_hat_se   <span class="ot">=</span> stop_dist_model_test_info[<span class="dv">2</span>, <span class="dv">2</span>] <span class="co"># Std. Error</span></span>
+<span id="cb566-10"><a href="inference-for-simple-linear-regression.html#cb566-10" tabindex="-1"></a>beta_1_hat_t    <span class="ot">=</span> stop_dist_model_test_info[<span class="dv">2</span>, <span class="dv">3</span>] <span class="co"># t value</span></span>
+<span id="cb566-11"><a href="inference-for-simple-linear-regression.html#cb566-11" tabindex="-1"></a>beta_1_hat_pval <span class="ot">=</span> stop_dist_model_test_info[<span class="dv">2</span>, <span class="dv">4</span>] <span class="co"># Pr(&gt;|t|)</span></span></code></pre></div>
 <p>We can then verify some equivalent expressions: the <span class="math inline">\(t\)</span> test statistic for <span class="math inline">\(\hat{\beta}_1\)</span> and the two-sided p-value associated with that test statistic.</p>
-<div class="sourceCode" id="cb567"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb567-1"><a href="inference-for-simple-linear-regression.html#cb567-1" aria-hidden="true" tabindex="-1"></a>(beta_1_hat <span class="sc">-</span> <span class="dv">0</span>) <span class="sc">/</span> beta_1_hat_se</span></code></pre></div>
+<div class="sourceCode" id="cb567"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb567-1"><a href="inference-for-simple-linear-regression.html#cb567-1" tabindex="-1"></a>(beta_1_hat <span class="sc">-</span> <span class="dv">0</span>) <span class="sc">/</span> beta_1_hat_se</span></code></pre></div>
 <pre><code>## [1] 9.46399</code></pre>
-<div class="sourceCode" id="cb569"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb569-1"><a href="inference-for-simple-linear-regression.html#cb569-1" aria-hidden="true" tabindex="-1"></a>beta_1_hat_t</span></code></pre></div>
+<div class="sourceCode" id="cb569"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb569-1"><a href="inference-for-simple-linear-regression.html#cb569-1" tabindex="-1"></a>beta_1_hat_t</span></code></pre></div>
 <pre><code>## [1] 9.46399</code></pre>
-<div class="sourceCode" id="cb571"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb571-1"><a href="inference-for-simple-linear-regression.html#cb571-1" aria-hidden="true" tabindex="-1"></a><span class="dv">2</span> <span class="sc">*</span> <span class="fu">pt</span>(<span class="fu">abs</span>(beta_1_hat_t), <span class="at">df =</span> <span class="fu">length</span>(<span class="fu">resid</span>(stop_dist_model)) <span class="sc">-</span> <span class="dv">2</span>, <span class="at">lower.tail =</span> <span class="cn">FALSE</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb571"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb571-1"><a href="inference-for-simple-linear-regression.html#cb571-1" tabindex="-1"></a><span class="dv">2</span> <span class="sc">*</span> <span class="fu">pt</span>(<span class="fu">abs</span>(beta_1_hat_t), <span class="at">df =</span> <span class="fu">length</span>(<span class="fu">resid</span>(stop_dist_model)) <span class="sc">-</span> <span class="dv">2</span>, <span class="at">lower.tail =</span> <span class="cn">FALSE</span>)</span></code></pre></div>
 <pre><code>## [1] 1.489836e-12</code></pre>
-<div class="sourceCode" id="cb573"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb573-1"><a href="inference-for-simple-linear-regression.html#cb573-1" aria-hidden="true" tabindex="-1"></a>beta_1_hat_pval</span></code></pre></div>
+<div class="sourceCode" id="cb573"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb573-1"><a href="inference-for-simple-linear-regression.html#cb573-1" tabindex="-1"></a>beta_1_hat_pval</span></code></pre></div>
 <pre><code>## [1] 1.489836e-12</code></pre>
 </div>
 <div id="significance-of-regression-t-test" class="section level3 hasAnchor" number="8.6.2">
@@ -997,7 +997,7 @@ <h3><span class="header-section-number">8.6.2</span> Significance of Regression,
 <div id="confidence-intervals-in-r" class="section level3 hasAnchor" number="8.6.3">
 <h3><span class="header-section-number">8.6.3</span> Confidence Intervals in <code>R</code><a href="inference-for-simple-linear-regression.html#confidence-intervals-in-r" class="anchor-section" aria-label="Anchor link to header"></a></h3>
 <p>Using <code>R</code> we can very easily obtain the confidence intervals for <span class="math inline">\(\beta_0\)</span> and <span class="math inline">\(\beta_1\)</span>.</p>
-<div class="sourceCode" id="cb575"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb575-1"><a href="inference-for-simple-linear-regression.html#cb575-1" aria-hidden="true" tabindex="-1"></a><span class="fu">confint</span>(stop_dist_model, <span class="at">level =</span> <span class="fl">0.99</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb575"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb575-1"><a href="inference-for-simple-linear-regression.html#cb575-1" tabindex="-1"></a><span class="fu">confint</span>(stop_dist_model, <span class="at">level =</span> <span class="fl">0.99</span>)</span></code></pre></div>
 <pre><code>##                  0.5 %    99.5 %
 ## (Intercept) -35.706610 0.5484205
 ## speed         2.817919 5.0468988</code></pre>
@@ -1006,26 +1006,26 @@ <h3><span class="header-section-number">8.6.3</span> Confidence Intervals in <co
 <p>Note that this 99% confidence interval does <strong>not</strong> contain the hypothesized value of 0. Since it does not contain 0, it is equivalent to rejecting the test of <span class="math inline">\(H_0: \beta_1 = 0\)</span> vs <span class="math inline">\(H_1: \beta_1 \neq 0\)</span> at <span class="math inline">\(\alpha = 0.01\)</span>, which we had seen previously.</p>
 <p>You should be somewhat suspicious of the confidence interval for <span class="math inline">\(\beta_0\)</span>, as it covers negative values, which correspond to negative stopping distances. Technically the interpretation would be that we are 99% confident that the average stopping distance of a car traveling 0 miles per hour is between -35.7066103 and 0.5484205 feet, but we don’t really believe that, since we are actually certain that it would be non-negative.</p>
 <p>Note, we can extract specific values from this output a number of ways. This code is not run, and instead, you should check how it relates to the output of the code above.</p>
-<div class="sourceCode" id="cb577"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb577-1"><a href="inference-for-simple-linear-regression.html#cb577-1" aria-hidden="true" tabindex="-1"></a><span class="fu">confint</span>(stop_dist_model, <span class="at">level =</span> <span class="fl">0.99</span>)[<span class="dv">1</span>,]</span>
-<span id="cb577-2"><a href="inference-for-simple-linear-regression.html#cb577-2" aria-hidden="true" tabindex="-1"></a><span class="fu">confint</span>(stop_dist_model, <span class="at">level =</span> <span class="fl">0.99</span>)[<span class="dv">1</span>, <span class="dv">1</span>]</span>
-<span id="cb577-3"><a href="inference-for-simple-linear-regression.html#cb577-3" aria-hidden="true" tabindex="-1"></a><span class="fu">confint</span>(stop_dist_model, <span class="at">level =</span> <span class="fl">0.99</span>)[<span class="dv">1</span>, <span class="dv">2</span>]</span>
-<span id="cb577-4"><a href="inference-for-simple-linear-regression.html#cb577-4" aria-hidden="true" tabindex="-1"></a><span class="fu">confint</span>(stop_dist_model, <span class="at">parm =</span> <span class="st">&quot;(Intercept)&quot;</span>, <span class="at">level =</span> <span class="fl">0.99</span>)</span>
-<span id="cb577-5"><a href="inference-for-simple-linear-regression.html#cb577-5" aria-hidden="true" tabindex="-1"></a><span class="fu">confint</span>(stop_dist_model, <span class="at">level =</span> <span class="fl">0.99</span>)[<span class="dv">2</span>,]</span>
-<span id="cb577-6"><a href="inference-for-simple-linear-regression.html#cb577-6" aria-hidden="true" tabindex="-1"></a><span class="fu">confint</span>(stop_dist_model, <span class="at">level =</span> <span class="fl">0.99</span>)[<span class="dv">2</span>, <span class="dv">1</span>]</span>
-<span id="cb577-7"><a href="inference-for-simple-linear-regression.html#cb577-7" aria-hidden="true" tabindex="-1"></a><span class="fu">confint</span>(stop_dist_model, <span class="at">level =</span> <span class="fl">0.99</span>)[<span class="dv">2</span>, <span class="dv">2</span>]</span>
-<span id="cb577-8"><a href="inference-for-simple-linear-regression.html#cb577-8" aria-hidden="true" tabindex="-1"></a><span class="fu">confint</span>(stop_dist_model, <span class="at">parm =</span> <span class="st">&quot;speed&quot;</span>, <span class="at">level =</span> <span class="fl">0.99</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb577"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb577-1"><a href="inference-for-simple-linear-regression.html#cb577-1" tabindex="-1"></a><span class="fu">confint</span>(stop_dist_model, <span class="at">level =</span> <span class="fl">0.99</span>)[<span class="dv">1</span>,]</span>
+<span id="cb577-2"><a href="inference-for-simple-linear-regression.html#cb577-2" tabindex="-1"></a><span class="fu">confint</span>(stop_dist_model, <span class="at">level =</span> <span class="fl">0.99</span>)[<span class="dv">1</span>, <span class="dv">1</span>]</span>
+<span id="cb577-3"><a href="inference-for-simple-linear-regression.html#cb577-3" tabindex="-1"></a><span class="fu">confint</span>(stop_dist_model, <span class="at">level =</span> <span class="fl">0.99</span>)[<span class="dv">1</span>, <span class="dv">2</span>]</span>
+<span id="cb577-4"><a href="inference-for-simple-linear-regression.html#cb577-4" tabindex="-1"></a><span class="fu">confint</span>(stop_dist_model, <span class="at">parm =</span> <span class="st">&quot;(Intercept)&quot;</span>, <span class="at">level =</span> <span class="fl">0.99</span>)</span>
+<span id="cb577-5"><a href="inference-for-simple-linear-regression.html#cb577-5" tabindex="-1"></a><span class="fu">confint</span>(stop_dist_model, <span class="at">level =</span> <span class="fl">0.99</span>)[<span class="dv">2</span>,]</span>
+<span id="cb577-6"><a href="inference-for-simple-linear-regression.html#cb577-6" tabindex="-1"></a><span class="fu">confint</span>(stop_dist_model, <span class="at">level =</span> <span class="fl">0.99</span>)[<span class="dv">2</span>, <span class="dv">1</span>]</span>
+<span id="cb577-7"><a href="inference-for-simple-linear-regression.html#cb577-7" tabindex="-1"></a><span class="fu">confint</span>(stop_dist_model, <span class="at">level =</span> <span class="fl">0.99</span>)[<span class="dv">2</span>, <span class="dv">2</span>]</span>
+<span id="cb577-8"><a href="inference-for-simple-linear-regression.html#cb577-8" tabindex="-1"></a><span class="fu">confint</span>(stop_dist_model, <span class="at">parm =</span> <span class="st">&quot;speed&quot;</span>, <span class="at">level =</span> <span class="fl">0.99</span>)</span></code></pre></div>
 <p>We can also verify that calculations that <code>R</code> is performing for the <span class="math inline">\(\beta_1\)</span> interval.</p>
-<div class="sourceCode" id="cb578"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb578-1"><a href="inference-for-simple-linear-regression.html#cb578-1" aria-hidden="true" tabindex="-1"></a><span class="co"># store estimate</span></span>
-<span id="cb578-2"><a href="inference-for-simple-linear-regression.html#cb578-2" aria-hidden="true" tabindex="-1"></a>beta_1_hat <span class="ot">=</span> <span class="fu">coef</span>(stop_dist_model)[<span class="dv">2</span>]</span>
-<span id="cb578-3"><a href="inference-for-simple-linear-regression.html#cb578-3" aria-hidden="true" tabindex="-1"></a></span>
-<span id="cb578-4"><a href="inference-for-simple-linear-regression.html#cb578-4" aria-hidden="true" tabindex="-1"></a><span class="co"># store standard error</span></span>
-<span id="cb578-5"><a href="inference-for-simple-linear-regression.html#cb578-5" aria-hidden="true" tabindex="-1"></a>beta_1_hat_se <span class="ot">=</span> <span class="fu">summary</span>(stop_dist_model)<span class="sc">$</span>coefficients[<span class="dv">2</span>, <span class="dv">2</span>]</span>
-<span id="cb578-6"><a href="inference-for-simple-linear-regression.html#cb578-6" aria-hidden="true" tabindex="-1"></a></span>
-<span id="cb578-7"><a href="inference-for-simple-linear-regression.html#cb578-7" aria-hidden="true" tabindex="-1"></a><span class="co"># calculate critical value for two-sided 99% CI</span></span>
-<span id="cb578-8"><a href="inference-for-simple-linear-regression.html#cb578-8" aria-hidden="true" tabindex="-1"></a>crit <span class="ot">=</span> <span class="fu">qt</span>(<span class="fl">0.995</span>, <span class="at">df =</span> <span class="fu">length</span>(<span class="fu">resid</span>(stop_dist_model)) <span class="sc">-</span> <span class="dv">2</span>)</span>
-<span id="cb578-9"><a href="inference-for-simple-linear-regression.html#cb578-9" aria-hidden="true" tabindex="-1"></a></span>
-<span id="cb578-10"><a href="inference-for-simple-linear-regression.html#cb578-10" aria-hidden="true" tabindex="-1"></a><span class="co"># est - margin, est + margin</span></span>
-<span id="cb578-11"><a href="inference-for-simple-linear-regression.html#cb578-11" aria-hidden="true" tabindex="-1"></a><span class="fu">c</span>(beta_1_hat <span class="sc">-</span> crit <span class="sc">*</span> beta_1_hat_se, beta_1_hat <span class="sc">+</span> crit <span class="sc">*</span> beta_1_hat_se)</span></code></pre></div>
+<div class="sourceCode" id="cb578"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb578-1"><a href="inference-for-simple-linear-regression.html#cb578-1" tabindex="-1"></a><span class="co"># store estimate</span></span>
+<span id="cb578-2"><a href="inference-for-simple-linear-regression.html#cb578-2" tabindex="-1"></a>beta_1_hat <span class="ot">=</span> <span class="fu">coef</span>(stop_dist_model)[<span class="dv">2</span>]</span>
+<span id="cb578-3"><a href="inference-for-simple-linear-regression.html#cb578-3" tabindex="-1"></a></span>
+<span id="cb578-4"><a href="inference-for-simple-linear-regression.html#cb578-4" tabindex="-1"></a><span class="co"># store standard error</span></span>
+<span id="cb578-5"><a href="inference-for-simple-linear-regression.html#cb578-5" tabindex="-1"></a>beta_1_hat_se <span class="ot">=</span> <span class="fu">summary</span>(stop_dist_model)<span class="sc">$</span>coefficients[<span class="dv">2</span>, <span class="dv">2</span>]</span>
+<span id="cb578-6"><a href="inference-for-simple-linear-regression.html#cb578-6" tabindex="-1"></a></span>
+<span id="cb578-7"><a href="inference-for-simple-linear-regression.html#cb578-7" tabindex="-1"></a><span class="co"># calculate critical value for two-sided 99% CI</span></span>
+<span id="cb578-8"><a href="inference-for-simple-linear-regression.html#cb578-8" tabindex="-1"></a>crit <span class="ot">=</span> <span class="fu">qt</span>(<span class="fl">0.995</span>, <span class="at">df =</span> <span class="fu">length</span>(<span class="fu">resid</span>(stop_dist_model)) <span class="sc">-</span> <span class="dv">2</span>)</span>
+<span id="cb578-9"><a href="inference-for-simple-linear-regression.html#cb578-9" tabindex="-1"></a></span>
+<span id="cb578-10"><a href="inference-for-simple-linear-regression.html#cb578-10" tabindex="-1"></a><span class="co"># est - margin, est + margin</span></span>
+<span id="cb578-11"><a href="inference-for-simple-linear-regression.html#cb578-11" tabindex="-1"></a><span class="fu">c</span>(beta_1_hat <span class="sc">-</span> crit <span class="sc">*</span> beta_1_hat_se, beta_1_hat <span class="sc">+</span> crit <span class="sc">*</span> beta_1_hat_se)</span></code></pre></div>
 <pre><code>##    speed    speed 
 ## 2.817919 5.046899</code></pre>
 </div>
@@ -1062,9 +1062,9 @@ <h2><span class="header-section-number">8.7</span> Confidence Interval for Mean
 \hat{y}(x) \pm t_{\alpha/2, n - 2} \cdot s_e\sqrt{\frac{1}{n}+\frac{(x-\bar{x})^2}{S_{xx}}}
 \]</span></p>
 <p>To find confidence intervals for the mean response using <code>R</code>, we use the <code>predict()</code> function. We give the function our fitted model as well as new data, stored as a data frame. (This is important, so that <code>R</code> knows the name of the predictor variable.) Here, we are finding the confidence interval for the mean stopping distance when a car is travelling 5 miles per hour and when a car is travelling 21 miles per hour.</p>
-<div class="sourceCode" id="cb580"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb580-1"><a href="inference-for-simple-linear-regression.html#cb580-1" aria-hidden="true" tabindex="-1"></a>new_speeds <span class="ot">=</span> <span class="fu">data.frame</span>(<span class="at">speed =</span> <span class="fu">c</span>(<span class="dv">5</span>, <span class="dv">21</span>))</span>
-<span id="cb580-2"><a href="inference-for-simple-linear-regression.html#cb580-2" aria-hidden="true" tabindex="-1"></a><span class="fu">predict</span>(stop_dist_model, <span class="at">newdata =</span> new_speeds, </span>
-<span id="cb580-3"><a href="inference-for-simple-linear-regression.html#cb580-3" aria-hidden="true" tabindex="-1"></a>        <span class="at">interval =</span> <span class="fu">c</span>(<span class="st">&quot;confidence&quot;</span>), <span class="at">level =</span> <span class="fl">0.99</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb580"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb580-1"><a href="inference-for-simple-linear-regression.html#cb580-1" tabindex="-1"></a>new_speeds <span class="ot">=</span> <span class="fu">data.frame</span>(<span class="at">speed =</span> <span class="fu">c</span>(<span class="dv">5</span>, <span class="dv">21</span>))</span>
+<span id="cb580-2"><a href="inference-for-simple-linear-regression.html#cb580-2" tabindex="-1"></a><span class="fu">predict</span>(stop_dist_model, <span class="at">newdata =</span> new_speeds, </span>
+<span id="cb580-3"><a href="inference-for-simple-linear-regression.html#cb580-3" tabindex="-1"></a>        <span class="at">interval =</span> <span class="fu">c</span>(<span class="st">&quot;confidence&quot;</span>), <span class="at">level =</span> <span class="fl">0.99</span>)</span></code></pre></div>
 <pre><code>##         fit       lwr      upr
 ## 1  2.082949 -10.89309 15.05898
 ## 2 65.001489  56.45836 73.54462</code></pre>
@@ -1091,8 +1091,8 @@ <h2><span class="header-section-number">8.8</span> Prediction Interval for New O
 \hat{y}(x) \pm t_{\alpha/2, n - 2} \cdot s_e\sqrt{1 + \frac{1}{n}+\frac{(x-\bar{x})^2}{S_{xx}}}.
 \]</span></p>
 <p>To calculate this for a set of points in <code>R</code> notice there is only a minor change in syntax from finding a confidence interval for the mean response.</p>
-<div class="sourceCode" id="cb582"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb582-1"><a href="inference-for-simple-linear-regression.html#cb582-1" aria-hidden="true" tabindex="-1"></a><span class="fu">predict</span>(stop_dist_model, <span class="at">newdata =</span> new_speeds, </span>
-<span id="cb582-2"><a href="inference-for-simple-linear-regression.html#cb582-2" aria-hidden="true" tabindex="-1"></a>        <span class="at">interval =</span> <span class="fu">c</span>(<span class="st">&quot;prediction&quot;</span>), <span class="at">level =</span> <span class="fl">0.99</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb582"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb582-1"><a href="inference-for-simple-linear-regression.html#cb582-1" tabindex="-1"></a><span class="fu">predict</span>(stop_dist_model, <span class="at">newdata =</span> new_speeds, </span>
+<span id="cb582-2"><a href="inference-for-simple-linear-regression.html#cb582-2" tabindex="-1"></a>        <span class="at">interval =</span> <span class="fu">c</span>(<span class="st">&quot;prediction&quot;</span>), <span class="at">level =</span> <span class="fl">0.99</span>)</span></code></pre></div>
 <pre><code>##         fit       lwr       upr
 ## 1  2.082949 -41.16099  45.32689
 ## 2 65.001489  22.87494 107.12803</code></pre>
@@ -1101,29 +1101,29 @@ <h2><span class="header-section-number">8.8</span> Prediction Interval for New O
 <div id="confidence-and-prediction-bands" class="section level2 hasAnchor" number="8.9">
 <h2><span class="header-section-number">8.9</span> Confidence and Prediction Bands<a href="inference-for-simple-linear-regression.html#confidence-and-prediction-bands" class="anchor-section" aria-label="Anchor link to header"></a></h2>
 <p>Often we will like to plot both confidence intervals for the mean response and prediction intervals for all possible values of <span class="math inline">\(x\)</span>. We calls these confidence and prediction bands.</p>
-<div class="sourceCode" id="cb584"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb584-1"><a href="inference-for-simple-linear-regression.html#cb584-1" aria-hidden="true" tabindex="-1"></a>speed_grid <span class="ot">=</span> <span class="fu">seq</span>(<span class="fu">min</span>(cars<span class="sc">$</span>speed), <span class="fu">max</span>(cars<span class="sc">$</span>speed), <span class="at">by =</span> <span class="fl">0.01</span>)</span>
-<span id="cb584-2"><a href="inference-for-simple-linear-regression.html#cb584-2" aria-hidden="true" tabindex="-1"></a>dist_ci_band <span class="ot">=</span> <span class="fu">predict</span>(stop_dist_model, </span>
-<span id="cb584-3"><a href="inference-for-simple-linear-regression.html#cb584-3" aria-hidden="true" tabindex="-1"></a>                       <span class="at">newdata =</span> <span class="fu">data.frame</span>(<span class="at">speed =</span> speed_grid), </span>
-<span id="cb584-4"><a href="inference-for-simple-linear-regression.html#cb584-4" aria-hidden="true" tabindex="-1"></a>                       <span class="at">interval =</span> <span class="st">&quot;confidence&quot;</span>, <span class="at">level =</span> <span class="fl">0.99</span>)</span>
-<span id="cb584-5"><a href="inference-for-simple-linear-regression.html#cb584-5" aria-hidden="true" tabindex="-1"></a>dist_pi_band <span class="ot">=</span> <span class="fu">predict</span>(stop_dist_model, </span>
-<span id="cb584-6"><a href="inference-for-simple-linear-regression.html#cb584-6" aria-hidden="true" tabindex="-1"></a>                       <span class="at">newdata =</span> <span class="fu">data.frame</span>(<span class="at">speed =</span> speed_grid), </span>
-<span id="cb584-7"><a href="inference-for-simple-linear-regression.html#cb584-7" aria-hidden="true" tabindex="-1"></a>                       <span class="at">interval =</span> <span class="st">&quot;prediction&quot;</span>, <span class="at">level =</span> <span class="fl">0.99</span>) </span>
-<span id="cb584-8"><a href="inference-for-simple-linear-regression.html#cb584-8" aria-hidden="true" tabindex="-1"></a></span>
-<span id="cb584-9"><a href="inference-for-simple-linear-regression.html#cb584-9" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(dist <span class="sc">~</span> speed, <span class="at">data =</span> cars,</span>
-<span id="cb584-10"><a href="inference-for-simple-linear-regression.html#cb584-10" aria-hidden="true" tabindex="-1"></a>     <span class="at">xlab =</span> <span class="st">&quot;Speed (in Miles Per Hour)&quot;</span>,</span>
-<span id="cb584-11"><a href="inference-for-simple-linear-regression.html#cb584-11" aria-hidden="true" tabindex="-1"></a>     <span class="at">ylab =</span> <span class="st">&quot;Stopping Distance (in Feet)&quot;</span>,</span>
-<span id="cb584-12"><a href="inference-for-simple-linear-regression.html#cb584-12" aria-hidden="true" tabindex="-1"></a>     <span class="at">main =</span> <span class="st">&quot;Stopping Distance vs Speed&quot;</span>,</span>
-<span id="cb584-13"><a href="inference-for-simple-linear-regression.html#cb584-13" aria-hidden="true" tabindex="-1"></a>     <span class="at">pch  =</span> <span class="dv">20</span>,</span>
-<span id="cb584-14"><a href="inference-for-simple-linear-regression.html#cb584-14" aria-hidden="true" tabindex="-1"></a>     <span class="at">cex  =</span> <span class="dv">2</span>,</span>
-<span id="cb584-15"><a href="inference-for-simple-linear-regression.html#cb584-15" aria-hidden="true" tabindex="-1"></a>     <span class="at">col  =</span> <span class="st">&quot;grey&quot;</span>,</span>
-<span id="cb584-16"><a href="inference-for-simple-linear-regression.html#cb584-16" aria-hidden="true" tabindex="-1"></a>     <span class="at">ylim =</span> <span class="fu">c</span>(<span class="fu">min</span>(dist_pi_band), <span class="fu">max</span>(dist_pi_band)))</span>
-<span id="cb584-17"><a href="inference-for-simple-linear-regression.html#cb584-17" aria-hidden="true" tabindex="-1"></a><span class="fu">abline</span>(stop_dist_model, <span class="at">lwd =</span> <span class="dv">5</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>)</span>
-<span id="cb584-18"><a href="inference-for-simple-linear-regression.html#cb584-18" aria-hidden="true" tabindex="-1"></a></span>
-<span id="cb584-19"><a href="inference-for-simple-linear-regression.html#cb584-19" aria-hidden="true" tabindex="-1"></a><span class="fu">lines</span>(speed_grid, dist_ci_band[,<span class="st">&quot;lwr&quot;</span>], <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">lwd =</span> <span class="dv">3</span>, <span class="at">lty =</span> <span class="dv">2</span>)</span>
-<span id="cb584-20"><a href="inference-for-simple-linear-regression.html#cb584-20" aria-hidden="true" tabindex="-1"></a><span class="fu">lines</span>(speed_grid, dist_ci_band[,<span class="st">&quot;upr&quot;</span>], <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">lwd =</span> <span class="dv">3</span>, <span class="at">lty =</span> <span class="dv">2</span>)</span>
-<span id="cb584-21"><a href="inference-for-simple-linear-regression.html#cb584-21" aria-hidden="true" tabindex="-1"></a><span class="fu">lines</span>(speed_grid, dist_pi_band[,<span class="st">&quot;lwr&quot;</span>], <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">lwd =</span> <span class="dv">3</span>, <span class="at">lty =</span> <span class="dv">3</span>)</span>
-<span id="cb584-22"><a href="inference-for-simple-linear-regression.html#cb584-22" aria-hidden="true" tabindex="-1"></a><span class="fu">lines</span>(speed_grid, dist_pi_band[,<span class="st">&quot;upr&quot;</span>], <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">lwd =</span> <span class="dv">3</span>, <span class="at">lty =</span> <span class="dv">3</span>)</span>
-<span id="cb584-23"><a href="inference-for-simple-linear-regression.html#cb584-23" aria-hidden="true" tabindex="-1"></a><span class="fu">points</span>(<span class="fu">mean</span>(cars<span class="sc">$</span>speed), <span class="fu">mean</span>(cars<span class="sc">$</span>dist), <span class="at">pch =</span> <span class="st">&quot;+&quot;</span>, <span class="at">cex =</span> <span class="dv">3</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb584"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb584-1"><a href="inference-for-simple-linear-regression.html#cb584-1" tabindex="-1"></a>speed_grid <span class="ot">=</span> <span class="fu">seq</span>(<span class="fu">min</span>(cars<span class="sc">$</span>speed), <span class="fu">max</span>(cars<span class="sc">$</span>speed), <span class="at">by =</span> <span class="fl">0.01</span>)</span>
+<span id="cb584-2"><a href="inference-for-simple-linear-regression.html#cb584-2" tabindex="-1"></a>dist_ci_band <span class="ot">=</span> <span class="fu">predict</span>(stop_dist_model, </span>
+<span id="cb584-3"><a href="inference-for-simple-linear-regression.html#cb584-3" tabindex="-1"></a>                       <span class="at">newdata =</span> <span class="fu">data.frame</span>(<span class="at">speed =</span> speed_grid), </span>
+<span id="cb584-4"><a href="inference-for-simple-linear-regression.html#cb584-4" tabindex="-1"></a>                       <span class="at">interval =</span> <span class="st">&quot;confidence&quot;</span>, <span class="at">level =</span> <span class="fl">0.99</span>)</span>
+<span id="cb584-5"><a href="inference-for-simple-linear-regression.html#cb584-5" tabindex="-1"></a>dist_pi_band <span class="ot">=</span> <span class="fu">predict</span>(stop_dist_model, </span>
+<span id="cb584-6"><a href="inference-for-simple-linear-regression.html#cb584-6" tabindex="-1"></a>                       <span class="at">newdata =</span> <span class="fu">data.frame</span>(<span class="at">speed =</span> speed_grid), </span>
+<span id="cb584-7"><a href="inference-for-simple-linear-regression.html#cb584-7" tabindex="-1"></a>                       <span class="at">interval =</span> <span class="st">&quot;prediction&quot;</span>, <span class="at">level =</span> <span class="fl">0.99</span>) </span>
+<span id="cb584-8"><a href="inference-for-simple-linear-regression.html#cb584-8" tabindex="-1"></a></span>
+<span id="cb584-9"><a href="inference-for-simple-linear-regression.html#cb584-9" tabindex="-1"></a><span class="fu">plot</span>(dist <span class="sc">~</span> speed, <span class="at">data =</span> cars,</span>
+<span id="cb584-10"><a href="inference-for-simple-linear-regression.html#cb584-10" tabindex="-1"></a>     <span class="at">xlab =</span> <span class="st">&quot;Speed (in Miles Per Hour)&quot;</span>,</span>
+<span id="cb584-11"><a href="inference-for-simple-linear-regression.html#cb584-11" tabindex="-1"></a>     <span class="at">ylab =</span> <span class="st">&quot;Stopping Distance (in Feet)&quot;</span>,</span>
+<span id="cb584-12"><a href="inference-for-simple-linear-regression.html#cb584-12" tabindex="-1"></a>     <span class="at">main =</span> <span class="st">&quot;Stopping Distance vs Speed&quot;</span>,</span>
+<span id="cb584-13"><a href="inference-for-simple-linear-regression.html#cb584-13" tabindex="-1"></a>     <span class="at">pch  =</span> <span class="dv">20</span>,</span>
+<span id="cb584-14"><a href="inference-for-simple-linear-regression.html#cb584-14" tabindex="-1"></a>     <span class="at">cex  =</span> <span class="dv">2</span>,</span>
+<span id="cb584-15"><a href="inference-for-simple-linear-regression.html#cb584-15" tabindex="-1"></a>     <span class="at">col  =</span> <span class="st">&quot;grey&quot;</span>,</span>
+<span id="cb584-16"><a href="inference-for-simple-linear-regression.html#cb584-16" tabindex="-1"></a>     <span class="at">ylim =</span> <span class="fu">c</span>(<span class="fu">min</span>(dist_pi_band), <span class="fu">max</span>(dist_pi_band)))</span>
+<span id="cb584-17"><a href="inference-for-simple-linear-regression.html#cb584-17" tabindex="-1"></a><span class="fu">abline</span>(stop_dist_model, <span class="at">lwd =</span> <span class="dv">5</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>)</span>
+<span id="cb584-18"><a href="inference-for-simple-linear-regression.html#cb584-18" tabindex="-1"></a></span>
+<span id="cb584-19"><a href="inference-for-simple-linear-regression.html#cb584-19" tabindex="-1"></a><span class="fu">lines</span>(speed_grid, dist_ci_band[,<span class="st">&quot;lwr&quot;</span>], <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">lwd =</span> <span class="dv">3</span>, <span class="at">lty =</span> <span class="dv">2</span>)</span>
+<span id="cb584-20"><a href="inference-for-simple-linear-regression.html#cb584-20" tabindex="-1"></a><span class="fu">lines</span>(speed_grid, dist_ci_band[,<span class="st">&quot;upr&quot;</span>], <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">lwd =</span> <span class="dv">3</span>, <span class="at">lty =</span> <span class="dv">2</span>)</span>
+<span id="cb584-21"><a href="inference-for-simple-linear-regression.html#cb584-21" tabindex="-1"></a><span class="fu">lines</span>(speed_grid, dist_pi_band[,<span class="st">&quot;lwr&quot;</span>], <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">lwd =</span> <span class="dv">3</span>, <span class="at">lty =</span> <span class="dv">3</span>)</span>
+<span id="cb584-22"><a href="inference-for-simple-linear-regression.html#cb584-22" tabindex="-1"></a><span class="fu">lines</span>(speed_grid, dist_pi_band[,<span class="st">&quot;upr&quot;</span>], <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">lwd =</span> <span class="dv">3</span>, <span class="at">lty =</span> <span class="dv">3</span>)</span>
+<span id="cb584-23"><a href="inference-for-simple-linear-regression.html#cb584-23" tabindex="-1"></a><span class="fu">points</span>(<span class="fu">mean</span>(cars<span class="sc">$</span>speed), <span class="fu">mean</span>(cars<span class="sc">$</span>dist), <span class="at">pch =</span> <span class="st">&quot;+&quot;</span>, <span class="at">cex =</span> <span class="dv">3</span>)</span></code></pre></div>
 <p><img src="slr-inf_files/figure-html/unnamed-chunk-27-1.png" width="960" style="display: block; margin: auto;" /></p>
 <p>Some things to notice:</p>
 <ul>
@@ -1208,7 +1208,7 @@ <h2><span class="header-section-number">8.10</span> Significance of Regression,
 \]</span></p>
 <p>In particular, we will reject the null when the <span class="math inline">\(F\)</span> statistic is large, that is, when there is a low probability that the observations could have come from the null model by chance. We will let <code>R</code> calculate the p-value for us.</p>
 <p>To perform the <span class="math inline">\(F\)</span> test in <code>R</code> you can look at the last row of the output from <code>summary()</code> called <code>F-statistic</code> which gives the value of the test statistic, the relevant degrees of freedom, as well as the p-value of the test.</p>
-<div class="sourceCode" id="cb585"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb585-1"><a href="inference-for-simple-linear-regression.html#cb585-1" aria-hidden="true" tabindex="-1"></a><span class="fu">summary</span>(stop_dist_model)</span></code></pre></div>
+<div class="sourceCode" id="cb585"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb585-1"><a href="inference-for-simple-linear-regression.html#cb585-1" tabindex="-1"></a><span class="fu">summary</span>(stop_dist_model)</span></code></pre></div>
 <pre><code>## 
 ## Call:
 ## lm(formula = dist ~ speed, data = cars)
@@ -1228,7 +1228,7 @@ <h2><span class="header-section-number">8.10</span> Significance of Regression,
 ## Multiple R-squared:  0.6511, Adjusted R-squared:  0.6438 
 ## F-statistic: 89.57 on 1 and 48 DF,  p-value: 1.49e-12</code></pre>
 <p>Additionally, you can use the <code>anova()</code> function to display the information in an ANOVA table.</p>
-<div class="sourceCode" id="cb587"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb587-1"><a href="inference-for-simple-linear-regression.html#cb587-1" aria-hidden="true" tabindex="-1"></a><span class="fu">anova</span>(stop_dist_model)</span></code></pre></div>
+<div class="sourceCode" id="cb587"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb587-1"><a href="inference-for-simple-linear-regression.html#cb587-1" tabindex="-1"></a><span class="fu">anova</span>(stop_dist_model)</span></code></pre></div>
 <pre><code>## Analysis of Variance Table
 ## 
 ## Response: dist
@@ -1239,7 +1239,7 @@ <h2><span class="header-section-number">8.10</span> Significance of Regression,
 ## Signif. codes:  0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1</code></pre>
 <p>This also gives a p-value for the test. You should notice that the p-value from the <span class="math inline">\(t\)</span> test was the same. You might also notice that the value of the test statistic for the <span class="math inline">\(t\)</span> test, <span class="math inline">\(9.46399\)</span>, can be squared to obtain the value of the <span class="math inline">\(F\)</span> statistic, <span class="math inline">\(89.5671065\)</span>.</p>
 <p>Note that there is another equivalent way to do this in <code>R</code>, which we will return to often to compare two models.</p>
-<div class="sourceCode" id="cb589"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb589-1"><a href="inference-for-simple-linear-regression.html#cb589-1" aria-hidden="true" tabindex="-1"></a><span class="fu">anova</span>(<span class="fu">lm</span>(dist <span class="sc">~</span> <span class="dv">1</span>, <span class="at">data =</span> cars), <span class="fu">lm</span>(dist <span class="sc">~</span> speed, <span class="at">data =</span> cars))</span></code></pre></div>
+<div class="sourceCode" id="cb589"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb589-1"><a href="inference-for-simple-linear-regression.html#cb589-1" tabindex="-1"></a><span class="fu">anova</span>(<span class="fu">lm</span>(dist <span class="sc">~</span> <span class="dv">1</span>, <span class="at">data =</span> cars), <span class="fu">lm</span>(dist <span class="sc">~</span> speed, <span class="at">data =</span> cars))</span></code></pre></div>
 <pre><code>## Analysis of Variance Table
 ## 
 ## Model 1: dist ~ 1
@@ -1259,7 +1259,7 @@ <h2><span class="header-section-number">8.11</span> <code>R</code> Markdown<a hr
 <ul>
 <li><a href="slr-inf.Rmd" target="_blank"><code>slr-inf.Rmd</code></a></li>
 </ul>
-<p>The file was created using <code>R</code> version <code>4.3.2</code>.</p>
+<p>The file was created using <code>R</code> version <code>4.4.1</code>.</p>
 
 </div>
 </div>
diff --git a/introduction-to-r.html b/introduction-to-r.html
index 329027c..7d45ca1 100644
--- a/introduction-to-r.html
+++ b/introduction-to-r.html
@@ -6,7 +6,7 @@
   <meta http-equiv="X-UA-Compatible" content="IE=edge" />
   <title>Chapter 2 Introduction to R | Applied Statistics with R</title>
   <meta name="description" content="Chapter 2 Introduction to R | Applied Statistics with R" />
-  <meta name="generator" content="bookdown 0.37 and GitBook 2.6.7" />
+  <meta name="generator" content="bookdown 0.40 and GitBook 2.6.7" />
 
   <meta property="og:title" content="Chapter 2 Introduction to R | Applied Statistics with R" />
   <meta property="og:type" content="book" />
@@ -55,7 +55,7 @@
 
 <style type="text/css">
 pre > code.sourceCode { white-space: pre; position: relative; }
-pre > code.sourceCode > span { display: inline-block; line-height: 1.25; }
+pre > code.sourceCode > span { line-height: 1.25; }
 pre > code.sourceCode > span:empty { height: 1.2em; }
 .sourceCode { overflow: visible; }
 code.sourceCode > span { color: inherit; text-decoration: inherit; }
@@ -472,7 +472,7 @@ <h2><span class="header-section-number">2.1</span> Getting Started<a href="intro
 <ul>
 <li><a href="http://cran.r-project.org/" target="_blank"><code>R</code>, the actual programming language.</a>
 <ul>
-<li>Chose your operating system, and select the most recent version, 4.3.2.</li>
+<li>Chose your operating system, and select the most recent version, 4.4.1.</li>
 </ul></li>
 <li><a href="http://www.rstudio.com/" target="_blank">RStudio, an excellent IDE for working with <code>R</code>.</a>
 <ul>
@@ -654,10 +654,10 @@ <h4>Trigonometry<a href="introduction-to-r.html#trigonometry" class="anchor-sect
 <div id="getting-help" class="section level2 hasAnchor" number="2.3">
 <h2><span class="header-section-number">2.3</span> Getting Help<a href="introduction-to-r.html#getting-help" class="anchor-section" aria-label="Anchor link to header"></a></h2>
 <p>In using <code>R</code> as a calculator, we have seen a number of functions: <code>sqrt()</code>, <code>exp()</code>, <code>log()</code> and <code>sin()</code>. To get documentation about a function in <code>R</code>, simply put a question mark in front of the function name and RStudio will display the documentation, for example:</p>
-<div class="sourceCode" id="cb3"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb3-1"><a href="introduction-to-r.html#cb3-1" aria-hidden="true" tabindex="-1"></a>?log</span>
-<span id="cb3-2"><a href="introduction-to-r.html#cb3-2" aria-hidden="true" tabindex="-1"></a>?sin</span>
-<span id="cb3-3"><a href="introduction-to-r.html#cb3-3" aria-hidden="true" tabindex="-1"></a>?paste</span>
-<span id="cb3-4"><a href="introduction-to-r.html#cb3-4" aria-hidden="true" tabindex="-1"></a>?lm</span></code></pre></div>
+<div class="sourceCode" id="cb3"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb3-1"><a href="introduction-to-r.html#cb3-1" tabindex="-1"></a>?log</span>
+<span id="cb3-2"><a href="introduction-to-r.html#cb3-2" tabindex="-1"></a>?sin</span>
+<span id="cb3-3"><a href="introduction-to-r.html#cb3-3" tabindex="-1"></a>?paste</span>
+<span id="cb3-4"><a href="introduction-to-r.html#cb3-4" tabindex="-1"></a>?lm</span></code></pre></div>
 <p>Frequently one of the most difficult things to do when learning <code>R</code> is asking for help. First, you need to decide to ask for help, then you need to know <em>how</em> to ask for help. Your very first line of defense should be to Google your error message or a short description of your issue. (The ability to solve problems using this method is quickly becoming an extremely valuable skill.) If that fails, and it eventually will, you should ask for help. There are a number of things you should include when emailing an instructor, or posting to a help website such as <a href="http://stats.stackexchange.com/" target="_blank">Stack Exchange</a>.</p>
 <ul>
 <li>Describe what you expect the code to do.</li>
@@ -672,9 +672,9 @@ <h2><span class="header-section-number">2.3</span> Getting Help<a href="introduc
 <h2><span class="header-section-number">2.4</span> Installing Packages<a href="introduction-to-r.html#installing-packages" class="anchor-section" aria-label="Anchor link to header"></a></h2>
 <p><code>R</code> comes with a number of built-in functions and datasets, but one of the main strengths of <code>R</code> as an open-source project is its package system. Packages add additional functions and data. Frequently if you want to do something in <code>R</code>, and it is not available by default, there is a good chance that there is a package that will fulfill your needs.</p>
 <p>To install a package, use the <code>install.packages()</code> function. Think of this as buying a recipe book from the store, bringing it home, and putting it on your shelf.</p>
-<div class="sourceCode" id="cb4"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb4-1"><a href="introduction-to-r.html#cb4-1" aria-hidden="true" tabindex="-1"></a><span class="fu">install.packages</span>(<span class="st">&quot;ggplot2&quot;</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb4"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb4-1"><a href="introduction-to-r.html#cb4-1" tabindex="-1"></a><span class="fu">install.packages</span>(<span class="st">&quot;ggplot2&quot;</span>)</span></code></pre></div>
 <p>Once a package is installed, it must be loaded into your current <code>R</code> session before being used. Think of this as taking the book off of the shelf and opening it up to read.</p>
-<div class="sourceCode" id="cb5"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb5-1"><a href="introduction-to-r.html#cb5-1" aria-hidden="true" tabindex="-1"></a><span class="fu">library</span>(ggplot2)</span></code></pre></div>
+<div class="sourceCode" id="cb5"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb5-1"><a href="introduction-to-r.html#cb5-1" tabindex="-1"></a><span class="fu">library</span>(ggplot2)</span></code></pre></div>
 <p>Once you close <code>R</code>, all the packages are closed and put back on the imaginary shelf. The next time you open <code>R</code>, you do not have to install the package again, but you do have to load any packages you intend to use by invoking <code>library()</code>.</p>
 
 </div>
diff --git a/libs/gitbook-2.6.7/css/plugin-highlight.css b/libs/gitbook-2.6.7/css/plugin-highlight.css
index 2aabd3d..02c0189 100644
--- a/libs/gitbook-2.6.7/css/plugin-highlight.css
+++ b/libs/gitbook-2.6.7/css/plugin-highlight.css
@@ -133,7 +133,7 @@
 .book.color-theme-1 .book-body .page-wrapper .page-inner section.normal code {
   /*
 
-Orginal Style from ethanschoonover.com/solarized (c) Jeremy Hull <sourdrums@gmail.com>
+Original Style from ethanschoonover.com/solarized (c) Jeremy Hull <sourdrums@gmail.com>
 
 */
   /* Solarized Green */
diff --git a/logistic-regression.html b/logistic-regression.html
index b05de8c..f00ff3d 100644
--- a/logistic-regression.html
+++ b/logistic-regression.html
@@ -6,7 +6,7 @@
   <meta http-equiv="X-UA-Compatible" content="IE=edge" />
   <title>Chapter 17 Logistic Regression | Applied Statistics with R</title>
   <meta name="description" content="Chapter 17 Logistic Regression | Applied Statistics with R" />
-  <meta name="generator" content="bookdown 0.37 and GitBook 2.6.7" />
+  <meta name="generator" content="bookdown 0.40 and GitBook 2.6.7" />
 
   <meta property="og:title" content="Chapter 17 Logistic Regression | Applied Statistics with R" />
   <meta property="og:type" content="book" />
@@ -55,7 +55,7 @@
 
 <style type="text/css">
 pre > code.sourceCode { white-space: pre; position: relative; }
-pre > code.sourceCode > span { display: inline-block; line-height: 1.25; }
+pre > code.sourceCode > span { line-height: 1.25; }
 pre > code.sourceCode > span:empty { height: 1.2em; }
 .sourceCode { overflow: visible; }
 code.sourceCode > span { color: inherit; text-decoration: inherit; }
@@ -485,7 +485,7 @@ <h2><span class="header-section-number">17.1</span> Generalized Linear Models<a
 <p>In <em>general</em>, a generalized linear model has three parts:</p>
 <ul>
 <li>A <strong>distribution</strong> of the response conditioned on the predictors. (Technically this distribution needs to be from the <a href="https://en.wikipedia.org/wiki/Exponential_family" target="_blank">exponential family</a> of distributions.)</li>
-<li>A <strong>linear combination</strong> of the <span class="math inline">\(p - 1\)</span> predictors, <span class="math inline">\(\beta_0 + \beta_1 x_1 + \beta_2 x_2 + \ldots + \beta_{p - 1} x_{p - 1}\)</span>, which we write as <span class="math inline">\(\eta({\bf x})\)</span>. That is,</li>
+<li>A <strong>linear combination</strong> of the <span class="math inline">\(p - 1\)</span> predictors, <span class="math inline">\(\beta_0 + \beta_1 x_1 + \beta_2 x_2 + \ldots  + \beta_{p - 1} x_{p - 1}\)</span>, which we write as <span class="math inline">\(\eta({\bf x})\)</span>. That is,</li>
 </ul>
 <p><span class="math display">\[\eta({\bf x}) = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \ldots  + \beta_{p - 1} x_{p - 1}\]</span></p>
 <ul>
@@ -679,13 +679,13 @@ <h3><span class="header-section-number">17.2.2</span> Fitting Issues<a href="log
 </div>
 <div id="simulation-examples" class="section level3 hasAnchor" number="17.2.3">
 <h3><span class="header-section-number">17.2.3</span> Simulation Examples<a href="logistic-regression.html#simulation-examples" class="anchor-section" aria-label="Anchor link to header"></a></h3>
-<div class="sourceCode" id="cb1433"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1433-1"><a href="logistic-regression.html#cb1433-1" aria-hidden="true" tabindex="-1"></a>sim_logistic_data <span class="ot">=</span> <span class="cf">function</span>(<span class="at">sample_size =</span> <span class="dv">25</span>, <span class="at">beta_0 =</span> <span class="sc">-</span><span class="dv">2</span>, <span class="at">beta_1 =</span> <span class="dv">3</span>) {</span>
-<span id="cb1433-2"><a href="logistic-regression.html#cb1433-2" aria-hidden="true" tabindex="-1"></a>  x <span class="ot">=</span> <span class="fu">rnorm</span>(<span class="at">n =</span> sample_size)</span>
-<span id="cb1433-3"><a href="logistic-regression.html#cb1433-3" aria-hidden="true" tabindex="-1"></a>  eta <span class="ot">=</span> beta_0 <span class="sc">+</span> beta_1 <span class="sc">*</span> x</span>
-<span id="cb1433-4"><a href="logistic-regression.html#cb1433-4" aria-hidden="true" tabindex="-1"></a>  p <span class="ot">=</span> <span class="dv">1</span> <span class="sc">/</span> (<span class="dv">1</span> <span class="sc">+</span> <span class="fu">exp</span>(<span class="sc">-</span>eta))</span>
-<span id="cb1433-5"><a href="logistic-regression.html#cb1433-5" aria-hidden="true" tabindex="-1"></a>  y <span class="ot">=</span> <span class="fu">rbinom</span>(<span class="at">n =</span> sample_size, <span class="at">size =</span> <span class="dv">1</span>, <span class="at">prob =</span> p)</span>
-<span id="cb1433-6"><a href="logistic-regression.html#cb1433-6" aria-hidden="true" tabindex="-1"></a>  <span class="fu">data.frame</span>(y, x)</span>
-<span id="cb1433-7"><a href="logistic-regression.html#cb1433-7" aria-hidden="true" tabindex="-1"></a>}</span></code></pre></div>
+<div class="sourceCode" id="cb1433"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1433-1"><a href="logistic-regression.html#cb1433-1" tabindex="-1"></a>sim_logistic_data <span class="ot">=</span> <span class="cf">function</span>(<span class="at">sample_size =</span> <span class="dv">25</span>, <span class="at">beta_0 =</span> <span class="sc">-</span><span class="dv">2</span>, <span class="at">beta_1 =</span> <span class="dv">3</span>) {</span>
+<span id="cb1433-2"><a href="logistic-regression.html#cb1433-2" tabindex="-1"></a>  x <span class="ot">=</span> <span class="fu">rnorm</span>(<span class="at">n =</span> sample_size)</span>
+<span id="cb1433-3"><a href="logistic-regression.html#cb1433-3" tabindex="-1"></a>  eta <span class="ot">=</span> beta_0 <span class="sc">+</span> beta_1 <span class="sc">*</span> x</span>
+<span id="cb1433-4"><a href="logistic-regression.html#cb1433-4" tabindex="-1"></a>  p <span class="ot">=</span> <span class="dv">1</span> <span class="sc">/</span> (<span class="dv">1</span> <span class="sc">+</span> <span class="fu">exp</span>(<span class="sc">-</span>eta))</span>
+<span id="cb1433-5"><a href="logistic-regression.html#cb1433-5" tabindex="-1"></a>  y <span class="ot">=</span> <span class="fu">rbinom</span>(<span class="at">n =</span> sample_size, <span class="at">size =</span> <span class="dv">1</span>, <span class="at">prob =</span> p)</span>
+<span id="cb1433-6"><a href="logistic-regression.html#cb1433-6" tabindex="-1"></a>  <span class="fu">data.frame</span>(y, x)</span>
+<span id="cb1433-7"><a href="logistic-regression.html#cb1433-7" tabindex="-1"></a>}</span></code></pre></div>
 <p>You might think, why not simply use ordinary linear regression? Even with a binary response, our goal is still to model (some function of) <span class="math inline">\(\text{E}[Y \mid {\bf X} = {\bf x}]\)</span>. However, with a binary response coded as <span class="math inline">\(0\)</span> and <span class="math inline">\(1\)</span>, <span class="math inline">\(\text{E}[Y \mid {\bf X} = {\bf x}] = P[Y = 1 \mid {\bf X} = {\bf x}]\)</span> since</p>
 <p><span class="math display">\[
 \begin{aligned}
@@ -706,9 +706,9 @@ <h3><span class="header-section-number">17.2.3</span> Simulation Examples<a href
 \eta({\bf x_i}) &amp;= -2 + 3 x_i
 \end{aligned}
 \]</span></p>
-<div class="sourceCode" id="cb1434"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1434-1"><a href="logistic-regression.html#cb1434-1" aria-hidden="true" tabindex="-1"></a><span class="fu">set.seed</span>(<span class="dv">1</span>)</span>
-<span id="cb1434-2"><a href="logistic-regression.html#cb1434-2" aria-hidden="true" tabindex="-1"></a>example_data <span class="ot">=</span> <span class="fu">sim_logistic_data</span>()</span>
-<span id="cb1434-3"><a href="logistic-regression.html#cb1434-3" aria-hidden="true" tabindex="-1"></a><span class="fu">head</span>(example_data)</span></code></pre></div>
+<div class="sourceCode" id="cb1434"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1434-1"><a href="logistic-regression.html#cb1434-1" tabindex="-1"></a><span class="fu">set.seed</span>(<span class="dv">1</span>)</span>
+<span id="cb1434-2"><a href="logistic-regression.html#cb1434-2" tabindex="-1"></a>example_data <span class="ot">=</span> <span class="fu">sim_logistic_data</span>()</span>
+<span id="cb1434-3"><a href="logistic-regression.html#cb1434-3" tabindex="-1"></a><span class="fu">head</span>(example_data)</span></code></pre></div>
 <pre><code>##   y          x
 ## 1 0 -0.6264538
 ## 2 1  0.1836433
@@ -717,21 +717,21 @@ <h3><span class="header-section-number">17.2.3</span> Simulation Examples<a href
 ## 5 0  0.3295078
 ## 6 0 -0.8204684</code></pre>
 <p>After simulating a dataset, we’ll then fit both ordinary linear regression and logistic regression. Notice that currently the responses variable <code>y</code> is a numeric variable that only takes values <code>0</code> and <code>1</code>. Later we’ll see that we can also fit logistic regression when the response is a factor variable with only two levels. (Generally, having a factor response is preferred, but having a dummy response allows us to make the comparison to using ordinary linear regression.)</p>
-<div class="sourceCode" id="cb1436"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1436-1"><a href="logistic-regression.html#cb1436-1" aria-hidden="true" tabindex="-1"></a><span class="co"># ordinary linear regression</span></span>
-<span id="cb1436-2"><a href="logistic-regression.html#cb1436-2" aria-hidden="true" tabindex="-1"></a>fit_lm  <span class="ot">=</span> <span class="fu">lm</span>(y <span class="sc">~</span> x, <span class="at">data =</span> example_data)</span>
-<span id="cb1436-3"><a href="logistic-regression.html#cb1436-3" aria-hidden="true" tabindex="-1"></a><span class="co"># logistic regression</span></span>
-<span id="cb1436-4"><a href="logistic-regression.html#cb1436-4" aria-hidden="true" tabindex="-1"></a>fit_glm <span class="ot">=</span> <span class="fu">glm</span>(y <span class="sc">~</span> x, <span class="at">data =</span> example_data, <span class="at">family =</span> binomial)</span></code></pre></div>
+<div class="sourceCode" id="cb1436"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1436-1"><a href="logistic-regression.html#cb1436-1" tabindex="-1"></a><span class="co"># ordinary linear regression</span></span>
+<span id="cb1436-2"><a href="logistic-regression.html#cb1436-2" tabindex="-1"></a>fit_lm  <span class="ot">=</span> <span class="fu">lm</span>(y <span class="sc">~</span> x, <span class="at">data =</span> example_data)</span>
+<span id="cb1436-3"><a href="logistic-regression.html#cb1436-3" tabindex="-1"></a><span class="co"># logistic regression</span></span>
+<span id="cb1436-4"><a href="logistic-regression.html#cb1436-4" tabindex="-1"></a>fit_glm <span class="ot">=</span> <span class="fu">glm</span>(y <span class="sc">~</span> x, <span class="at">data =</span> example_data, <span class="at">family =</span> binomial)</span></code></pre></div>
 <p>Notice that the syntax is extremely similar. What’s changed?</p>
 <ul>
 <li><code>lm()</code> has become <code>glm()</code></li>
 <li>We’ve added <code>family = binomial</code> argument</li>
 </ul>
 <p>In a lot of ways, <code>lm()</code> is just a more specific version of <code>glm()</code>. For example</p>
-<div class="sourceCode" id="cb1437"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1437-1"><a href="logistic-regression.html#cb1437-1" aria-hidden="true" tabindex="-1"></a><span class="fu">glm</span>(y <span class="sc">~</span> x, <span class="at">data =</span> example_data)</span></code></pre></div>
+<div class="sourceCode" id="cb1437"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1437-1"><a href="logistic-regression.html#cb1437-1" tabindex="-1"></a><span class="fu">glm</span>(y <span class="sc">~</span> x, <span class="at">data =</span> example_data)</span></code></pre></div>
 <p>would actually fit the ordinary linear regression that we have seen in the past. By default, <code>glm()</code> uses <code>family = gaussian</code> argument. That is, we’re fitting a GLM with a normally distributed response and the identity function as the link.</p>
 <p>The <code>family</code> argument to <code>glm()</code> actually specifies both the distribution and the link function. If not made explicit, the link function is chosen to be the <strong>canonical link function</strong>, which is essentially the most mathematical convenient link function. See <code>?glm</code> and <code>?family</code> for details. For example, the following code explicitly specifies the link function which was previously used by default.</p>
-<div class="sourceCode" id="cb1438"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1438-1"><a href="logistic-regression.html#cb1438-1" aria-hidden="true" tabindex="-1"></a><span class="co"># more detailed call to glm for logistic regression</span></span>
-<span id="cb1438-2"><a href="logistic-regression.html#cb1438-2" aria-hidden="true" tabindex="-1"></a>fit_glm <span class="ot">=</span> <span class="fu">glm</span>(y <span class="sc">~</span> x, <span class="at">data =</span> example_data, <span class="at">family =</span> <span class="fu">binomial</span>(<span class="at">link =</span> <span class="st">&quot;logit&quot;</span>))</span></code></pre></div>
+<div class="sourceCode" id="cb1438"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1438-1"><a href="logistic-regression.html#cb1438-1" tabindex="-1"></a><span class="co"># more detailed call to glm for logistic regression</span></span>
+<span id="cb1438-2"><a href="logistic-regression.html#cb1438-2" tabindex="-1"></a>fit_glm <span class="ot">=</span> <span class="fu">glm</span>(y <span class="sc">~</span> x, <span class="at">data =</span> example_data, <span class="at">family =</span> <span class="fu">binomial</span>(<span class="at">link =</span> <span class="st">&quot;logit&quot;</span>))</span></code></pre></div>
 <p>Making predictions with an object of type <code>glm</code> is slightly different than making predictions after fitting with <code>lm()</code>. In the case of logistic regression, with <code>family = binomial</code>, we have:</p>
 <table>
 <colgroup>
@@ -756,31 +756,31 @@ <h3><span class="header-section-number">17.2.3</span> Simulation Examples<a href
 </tbody>
 </table>
 <p>That is, <code>type = "link"</code> will get you the log odds, while <code>type = "response"</code> will return the estimated mean, in this case, <span class="math inline">\(P[Y = 1 \mid {\bf X} = {\bf x}]\)</span> for each observation.</p>
-<div class="sourceCode" id="cb1439"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1439-1"><a href="logistic-regression.html#cb1439-1" aria-hidden="true" tabindex="-1"></a><span class="fu">predict</span>(fit_glm, <span class="fu">data.frame</span>(<span class="at">x =</span> <span class="fl">1.2</span>)) <span class="co"># type = &quot;link&quot; as default</span></span></code></pre></div>
+<div class="sourceCode" id="cb1439"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1439-1"><a href="logistic-regression.html#cb1439-1" tabindex="-1"></a><span class="fu">predict</span>(fit_glm, <span class="fu">data.frame</span>(<span class="at">x =</span> <span class="fl">1.2</span>)) <span class="co"># type = &quot;link&quot; as default</span></span></code></pre></div>
 <pre><code>##        1 
 ## 2.076166</code></pre>
-<div class="sourceCode" id="cb1441"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1441-1"><a href="logistic-regression.html#cb1441-1" aria-hidden="true" tabindex="-1"></a><span class="fu">predict</span>(fit_glm, <span class="fu">data.frame</span>(<span class="at">x =</span> <span class="fl">1.2</span>), <span class="at">type =</span> <span class="st">&quot;link&quot;</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1441"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1441-1"><a href="logistic-regression.html#cb1441-1" tabindex="-1"></a><span class="fu">predict</span>(fit_glm, <span class="fu">data.frame</span>(<span class="at">x =</span> <span class="fl">1.2</span>), <span class="at">type =</span> <span class="st">&quot;link&quot;</span>)</span></code></pre></div>
 <pre><code>##        1 
 ## 2.076166</code></pre>
-<div class="sourceCode" id="cb1443"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1443-1"><a href="logistic-regression.html#cb1443-1" aria-hidden="true" tabindex="-1"></a><span class="fu">predict</span>(fit_glm, <span class="fu">data.frame</span>(<span class="at">x =</span> <span class="fl">1.2</span>), <span class="at">type =</span> <span class="st">&quot;response&quot;</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1443"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1443-1"><a href="logistic-regression.html#cb1443-1" tabindex="-1"></a><span class="fu">predict</span>(fit_glm, <span class="fu">data.frame</span>(<span class="at">x =</span> <span class="fl">1.2</span>), <span class="at">type =</span> <span class="st">&quot;response&quot;</span>)</span></code></pre></div>
 <pre><code>##        1 
 ## 0.888565</code></pre>
-<div class="sourceCode" id="cb1445"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1445-1"><a href="logistic-regression.html#cb1445-1" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(y <span class="sc">~</span> x, <span class="at">data =</span> example_data, </span>
-<span id="cb1445-2"><a href="logistic-regression.html#cb1445-2" aria-hidden="true" tabindex="-1"></a>     <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">ylab =</span> <span class="st">&quot;Estimated Probability&quot;</span>, </span>
-<span id="cb1445-3"><a href="logistic-regression.html#cb1445-3" aria-hidden="true" tabindex="-1"></a>     <span class="at">main =</span> <span class="st">&quot;Ordinary vs Logistic Regression&quot;</span>)</span>
-<span id="cb1445-4"><a href="logistic-regression.html#cb1445-4" aria-hidden="true" tabindex="-1"></a><span class="fu">grid</span>()</span>
-<span id="cb1445-5"><a href="logistic-regression.html#cb1445-5" aria-hidden="true" tabindex="-1"></a><span class="fu">abline</span>(fit_lm, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>)</span>
-<span id="cb1445-6"><a href="logistic-regression.html#cb1445-6" aria-hidden="true" tabindex="-1"></a><span class="fu">curve</span>(<span class="fu">predict</span>(fit_glm, <span class="fu">data.frame</span>(x), <span class="at">type =</span> <span class="st">&quot;response&quot;</span>), </span>
-<span id="cb1445-7"><a href="logistic-regression.html#cb1445-7" aria-hidden="true" tabindex="-1"></a>      <span class="at">add =</span> <span class="cn">TRUE</span>, <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">lty =</span> <span class="dv">2</span>)</span>
-<span id="cb1445-8"><a href="logistic-regression.html#cb1445-8" aria-hidden="true" tabindex="-1"></a><span class="fu">legend</span>(<span class="st">&quot;topleft&quot;</span>, <span class="fu">c</span>(<span class="st">&quot;Ordinary&quot;</span>, <span class="st">&quot;Logistic&quot;</span>, <span class="st">&quot;Data&quot;</span>), <span class="at">lty =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>, <span class="dv">0</span>), </span>
-<span id="cb1445-9"><a href="logistic-regression.html#cb1445-9" aria-hidden="true" tabindex="-1"></a>       <span class="at">pch =</span> <span class="fu">c</span>(<span class="cn">NA</span>, <span class="cn">NA</span>, <span class="dv">20</span>), <span class="at">lwd =</span> <span class="dv">2</span>, <span class="at">col =</span> <span class="fu">c</span>(<span class="st">&quot;darkorange&quot;</span>, <span class="st">&quot;dodgerblue&quot;</span>, <span class="st">&quot;black&quot;</span>))</span></code></pre></div>
+<div class="sourceCode" id="cb1445"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1445-1"><a href="logistic-regression.html#cb1445-1" tabindex="-1"></a><span class="fu">plot</span>(y <span class="sc">~</span> x, <span class="at">data =</span> example_data, </span>
+<span id="cb1445-2"><a href="logistic-regression.html#cb1445-2" tabindex="-1"></a>     <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">ylab =</span> <span class="st">&quot;Estimated Probability&quot;</span>, </span>
+<span id="cb1445-3"><a href="logistic-regression.html#cb1445-3" tabindex="-1"></a>     <span class="at">main =</span> <span class="st">&quot;Ordinary vs Logistic Regression&quot;</span>)</span>
+<span id="cb1445-4"><a href="logistic-regression.html#cb1445-4" tabindex="-1"></a><span class="fu">grid</span>()</span>
+<span id="cb1445-5"><a href="logistic-regression.html#cb1445-5" tabindex="-1"></a><span class="fu">abline</span>(fit_lm, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>)</span>
+<span id="cb1445-6"><a href="logistic-regression.html#cb1445-6" tabindex="-1"></a><span class="fu">curve</span>(<span class="fu">predict</span>(fit_glm, <span class="fu">data.frame</span>(x), <span class="at">type =</span> <span class="st">&quot;response&quot;</span>), </span>
+<span id="cb1445-7"><a href="logistic-regression.html#cb1445-7" tabindex="-1"></a>      <span class="at">add =</span> <span class="cn">TRUE</span>, <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">lty =</span> <span class="dv">2</span>)</span>
+<span id="cb1445-8"><a href="logistic-regression.html#cb1445-8" tabindex="-1"></a><span class="fu">legend</span>(<span class="st">&quot;topleft&quot;</span>, <span class="fu">c</span>(<span class="st">&quot;Ordinary&quot;</span>, <span class="st">&quot;Logistic&quot;</span>, <span class="st">&quot;Data&quot;</span>), <span class="at">lty =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>, <span class="dv">0</span>), </span>
+<span id="cb1445-9"><a href="logistic-regression.html#cb1445-9" tabindex="-1"></a>       <span class="at">pch =</span> <span class="fu">c</span>(<span class="cn">NA</span>, <span class="cn">NA</span>, <span class="dv">20</span>), <span class="at">lwd =</span> <span class="dv">2</span>, <span class="at">col =</span> <span class="fu">c</span>(<span class="st">&quot;darkorange&quot;</span>, <span class="st">&quot;dodgerblue&quot;</span>, <span class="st">&quot;black&quot;</span>))</span></code></pre></div>
 <p><img src="logistic_files/figure-html/unnamed-chunk-9-1.png" width="672" style="display: block; margin: auto;" /></p>
 <p>Since we only have a single predictor variable, we are able to graphically show this situation. First, note that the data, is plotted using black dots. The response <code>y</code> only takes values <code>0</code> and <code>1</code>.</p>
 <p>Next, we need to discuss the two added lines to the plot. The first, the solid orange line, is the fitted ordinary linear regression.</p>
 <p>The dashed blue curve is the estimated logistic regression. It is helpful to realize that we are not plotting an estimate of <span class="math inline">\(Y\)</span> for either. (Sometimes it might seem that way with ordinary linear regression, but that isn’t what is happening.) For both, we are plotting <span class="math inline">\(\hat{\text{E}}[Y \mid {\bf X} = {\bf x}]\)</span>, the estimated mean, which for a binary response happens to be an estimate of <span class="math inline">\(P[Y = 1 \mid {\bf X} = {\bf x}]\)</span>.</p>
 <p>We immediately see why ordinary linear regression is not a good idea. While it is estimating the mean, we see that it produces estimates that are less than 0! (And in other situations could produce estimates greater than 1!) If the mean is a probability, we don’t want probabilities less than 0 or greater than 1.</p>
 <p>Enter logistic regression. Since the output of the inverse logit function is restricted to be between 0 and 1, our estimates make much more sense as probabilities. Let’s look at our estimated coefficients. (With a lot of rounding, for simplicity.)</p>
-<div class="sourceCode" id="cb1446"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1446-1"><a href="logistic-regression.html#cb1446-1" aria-hidden="true" tabindex="-1"></a><span class="fu">round</span>(<span class="fu">coef</span>(fit_glm), <span class="dv">1</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1446"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1446-1"><a href="logistic-regression.html#cb1446-1" tabindex="-1"></a><span class="fu">round</span>(<span class="fu">coef</span>(fit_glm), <span class="dv">1</span>)</span></code></pre></div>
 <pre><code>## (Intercept)           x 
 ##        -2.3         3.7</code></pre>
 <p>Our estimated model is then:</p>
@@ -811,29 +811,29 @@ <h3><span class="header-section-number">17.2.3</span> Simulation Examples<a href
 \end{aligned}
 \]</span></p>
 <p>In this model, as <span class="math inline">\(x\)</span> increases, the log odds decrease.</p>
-<div class="sourceCode" id="cb1448"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1448-1"><a href="logistic-regression.html#cb1448-1" aria-hidden="true" tabindex="-1"></a><span class="fu">set.seed</span>(<span class="dv">1</span>)</span>
-<span id="cb1448-2"><a href="logistic-regression.html#cb1448-2" aria-hidden="true" tabindex="-1"></a>example_data <span class="ot">=</span> <span class="fu">sim_logistic_data</span>(<span class="at">sample_size =</span> <span class="dv">50</span>, <span class="at">beta_0 =</span> <span class="dv">1</span>, <span class="at">beta_1 =</span> <span class="sc">-</span><span class="dv">4</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1448"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1448-1"><a href="logistic-regression.html#cb1448-1" tabindex="-1"></a><span class="fu">set.seed</span>(<span class="dv">1</span>)</span>
+<span id="cb1448-2"><a href="logistic-regression.html#cb1448-2" tabindex="-1"></a>example_data <span class="ot">=</span> <span class="fu">sim_logistic_data</span>(<span class="at">sample_size =</span> <span class="dv">50</span>, <span class="at">beta_0 =</span> <span class="dv">1</span>, <span class="at">beta_1 =</span> <span class="sc">-</span><span class="dv">4</span>)</span></code></pre></div>
 <p>We again simulate some observations form this model, then fit logistic regression.</p>
-<div class="sourceCode" id="cb1449"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1449-1"><a href="logistic-regression.html#cb1449-1" aria-hidden="true" tabindex="-1"></a>fit_glm <span class="ot">=</span> <span class="fu">glm</span>(y <span class="sc">~</span> x, <span class="at">data =</span> example_data, <span class="at">family =</span> binomial)</span></code></pre></div>
-<div class="sourceCode" id="cb1450"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1450-1"><a href="logistic-regression.html#cb1450-1" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(y <span class="sc">~</span> x, <span class="at">data =</span> example_data, </span>
-<span id="cb1450-2"><a href="logistic-regression.html#cb1450-2" aria-hidden="true" tabindex="-1"></a>     <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">ylab =</span> <span class="st">&quot;Estimated Probability&quot;</span>, </span>
-<span id="cb1450-3"><a href="logistic-regression.html#cb1450-3" aria-hidden="true" tabindex="-1"></a>     <span class="at">main =</span> <span class="st">&quot;Logistic Regression, Decreasing Probability&quot;</span>)</span>
-<span id="cb1450-4"><a href="logistic-regression.html#cb1450-4" aria-hidden="true" tabindex="-1"></a><span class="fu">grid</span>()</span>
-<span id="cb1450-5"><a href="logistic-regression.html#cb1450-5" aria-hidden="true" tabindex="-1"></a><span class="fu">curve</span>(<span class="fu">predict</span>(fit_glm, <span class="fu">data.frame</span>(x), <span class="at">type =</span> <span class="st">&quot;response&quot;</span>), </span>
-<span id="cb1450-6"><a href="logistic-regression.html#cb1450-6" aria-hidden="true" tabindex="-1"></a>      <span class="at">add =</span> <span class="cn">TRUE</span>, <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">lty =</span> <span class="dv">2</span>)</span>
-<span id="cb1450-7"><a href="logistic-regression.html#cb1450-7" aria-hidden="true" tabindex="-1"></a><span class="fu">curve</span>(boot<span class="sc">::</span><span class="fu">inv.logit</span>(<span class="dv">1</span> <span class="sc">-</span> <span class="dv">4</span> <span class="sc">*</span> x), <span class="at">add =</span> <span class="cn">TRUE</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lty =</span> <span class="dv">1</span>)</span>
-<span id="cb1450-8"><a href="logistic-regression.html#cb1450-8" aria-hidden="true" tabindex="-1"></a><span class="fu">legend</span>(<span class="st">&quot;bottomleft&quot;</span>, <span class="fu">c</span>(<span class="st">&quot;True Probability&quot;</span>, <span class="st">&quot;Estimated Probability&quot;</span>, <span class="st">&quot;Data&quot;</span>), <span class="at">lty =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>, <span class="dv">0</span>), </span>
-<span id="cb1450-9"><a href="logistic-regression.html#cb1450-9" aria-hidden="true" tabindex="-1"></a>       <span class="at">pch =</span> <span class="fu">c</span>(<span class="cn">NA</span>, <span class="cn">NA</span>, <span class="dv">20</span>), <span class="at">lwd =</span> <span class="dv">2</span>, <span class="at">col =</span> <span class="fu">c</span>(<span class="st">&quot;darkorange&quot;</span>, <span class="st">&quot;dodgerblue&quot;</span>, <span class="st">&quot;black&quot;</span>))</span></code></pre></div>
+<div class="sourceCode" id="cb1449"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1449-1"><a href="logistic-regression.html#cb1449-1" tabindex="-1"></a>fit_glm <span class="ot">=</span> <span class="fu">glm</span>(y <span class="sc">~</span> x, <span class="at">data =</span> example_data, <span class="at">family =</span> binomial)</span></code></pre></div>
+<div class="sourceCode" id="cb1450"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1450-1"><a href="logistic-regression.html#cb1450-1" tabindex="-1"></a><span class="fu">plot</span>(y <span class="sc">~</span> x, <span class="at">data =</span> example_data, </span>
+<span id="cb1450-2"><a href="logistic-regression.html#cb1450-2" tabindex="-1"></a>     <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">ylab =</span> <span class="st">&quot;Estimated Probability&quot;</span>, </span>
+<span id="cb1450-3"><a href="logistic-regression.html#cb1450-3" tabindex="-1"></a>     <span class="at">main =</span> <span class="st">&quot;Logistic Regression, Decreasing Probability&quot;</span>)</span>
+<span id="cb1450-4"><a href="logistic-regression.html#cb1450-4" tabindex="-1"></a><span class="fu">grid</span>()</span>
+<span id="cb1450-5"><a href="logistic-regression.html#cb1450-5" tabindex="-1"></a><span class="fu">curve</span>(<span class="fu">predict</span>(fit_glm, <span class="fu">data.frame</span>(x), <span class="at">type =</span> <span class="st">&quot;response&quot;</span>), </span>
+<span id="cb1450-6"><a href="logistic-regression.html#cb1450-6" tabindex="-1"></a>      <span class="at">add =</span> <span class="cn">TRUE</span>, <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">lty =</span> <span class="dv">2</span>)</span>
+<span id="cb1450-7"><a href="logistic-regression.html#cb1450-7" tabindex="-1"></a><span class="fu">curve</span>(boot<span class="sc">::</span><span class="fu">inv.logit</span>(<span class="dv">1</span> <span class="sc">-</span> <span class="dv">4</span> <span class="sc">*</span> x), <span class="at">add =</span> <span class="cn">TRUE</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lty =</span> <span class="dv">1</span>)</span>
+<span id="cb1450-8"><a href="logistic-regression.html#cb1450-8" tabindex="-1"></a><span class="fu">legend</span>(<span class="st">&quot;bottomleft&quot;</span>, <span class="fu">c</span>(<span class="st">&quot;True Probability&quot;</span>, <span class="st">&quot;Estimated Probability&quot;</span>, <span class="st">&quot;Data&quot;</span>), <span class="at">lty =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>, <span class="dv">0</span>), </span>
+<span id="cb1450-9"><a href="logistic-regression.html#cb1450-9" tabindex="-1"></a>       <span class="at">pch =</span> <span class="fu">c</span>(<span class="cn">NA</span>, <span class="cn">NA</span>, <span class="dv">20</span>), <span class="at">lwd =</span> <span class="dv">2</span>, <span class="at">col =</span> <span class="fu">c</span>(<span class="st">&quot;darkorange&quot;</span>, <span class="st">&quot;dodgerblue&quot;</span>, <span class="st">&quot;black&quot;</span>))</span></code></pre></div>
 <p><img src="logistic_files/figure-html/unnamed-chunk-13-1.png" width="672" style="display: block; margin: auto;" /></p>
 <p>We see that this time, as <span class="math inline">\(x\)</span> increases, <span class="math inline">\(\hat{p}({\bf x})\)</span> decreases.</p>
 <p>Now let’s look at an example where the estimated probability doesn’t always simply increase or decrease. Much like ordinary linear regression, the linear combination of predictors can contain transformations of predictors (in this case a quadratic term) and interactions.</p>
-<div class="sourceCode" id="cb1451"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1451-1"><a href="logistic-regression.html#cb1451-1" aria-hidden="true" tabindex="-1"></a>sim_quadratic_logistic_data <span class="ot">=</span> <span class="cf">function</span>(<span class="at">sample_size =</span> <span class="dv">25</span>) {</span>
-<span id="cb1451-2"><a href="logistic-regression.html#cb1451-2" aria-hidden="true" tabindex="-1"></a>  x <span class="ot">=</span> <span class="fu">rnorm</span>(<span class="at">n =</span> sample_size)</span>
-<span id="cb1451-3"><a href="logistic-regression.html#cb1451-3" aria-hidden="true" tabindex="-1"></a>  eta <span class="ot">=</span> <span class="sc">-</span><span class="fl">1.5</span> <span class="sc">+</span> <span class="fl">0.5</span> <span class="sc">*</span> x <span class="sc">+</span> x <span class="sc">^</span> <span class="dv">2</span></span>
-<span id="cb1451-4"><a href="logistic-regression.html#cb1451-4" aria-hidden="true" tabindex="-1"></a>  p <span class="ot">=</span> <span class="dv">1</span> <span class="sc">/</span> (<span class="dv">1</span> <span class="sc">+</span> <span class="fu">exp</span>(<span class="sc">-</span>eta))</span>
-<span id="cb1451-5"><a href="logistic-regression.html#cb1451-5" aria-hidden="true" tabindex="-1"></a>  y <span class="ot">=</span> <span class="fu">rbinom</span>(<span class="at">n =</span> sample_size, <span class="at">size =</span> <span class="dv">1</span>, <span class="at">prob =</span> p)</span>
-<span id="cb1451-6"><a href="logistic-regression.html#cb1451-6" aria-hidden="true" tabindex="-1"></a>  <span class="fu">data.frame</span>(y, x)</span>
-<span id="cb1451-7"><a href="logistic-regression.html#cb1451-7" aria-hidden="true" tabindex="-1"></a>}</span></code></pre></div>
+<div class="sourceCode" id="cb1451"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1451-1"><a href="logistic-regression.html#cb1451-1" tabindex="-1"></a>sim_quadratic_logistic_data <span class="ot">=</span> <span class="cf">function</span>(<span class="at">sample_size =</span> <span class="dv">25</span>) {</span>
+<span id="cb1451-2"><a href="logistic-regression.html#cb1451-2" tabindex="-1"></a>  x <span class="ot">=</span> <span class="fu">rnorm</span>(<span class="at">n =</span> sample_size)</span>
+<span id="cb1451-3"><a href="logistic-regression.html#cb1451-3" tabindex="-1"></a>  eta <span class="ot">=</span> <span class="sc">-</span><span class="fl">1.5</span> <span class="sc">+</span> <span class="fl">0.5</span> <span class="sc">*</span> x <span class="sc">+</span> x <span class="sc">^</span> <span class="dv">2</span></span>
+<span id="cb1451-4"><a href="logistic-regression.html#cb1451-4" tabindex="-1"></a>  p <span class="ot">=</span> <span class="dv">1</span> <span class="sc">/</span> (<span class="dv">1</span> <span class="sc">+</span> <span class="fu">exp</span>(<span class="sc">-</span>eta))</span>
+<span id="cb1451-5"><a href="logistic-regression.html#cb1451-5" tabindex="-1"></a>  y <span class="ot">=</span> <span class="fu">rbinom</span>(<span class="at">n =</span> sample_size, <span class="at">size =</span> <span class="dv">1</span>, <span class="at">prob =</span> p)</span>
+<span id="cb1451-6"><a href="logistic-regression.html#cb1451-6" tabindex="-1"></a>  <span class="fu">data.frame</span>(y, x)</span>
+<span id="cb1451-7"><a href="logistic-regression.html#cb1451-7" tabindex="-1"></a>}</span></code></pre></div>
 <p><span class="math display">\[
 \log\left(\frac{p({\bf x})}{1 - p({\bf x})}\right) = -1.5 + 0.5x + x^2.
 \]</span></p>
@@ -845,19 +845,19 @@ <h3><span class="header-section-number">17.2.3</span> Simulation Examples<a href
 \eta({\bf x_i}) &amp;= -1.5 + 0.5x_i + x_i^2
 \end{aligned}
 \]</span></p>
-<div class="sourceCode" id="cb1452"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1452-1"><a href="logistic-regression.html#cb1452-1" aria-hidden="true" tabindex="-1"></a><span class="fu">set.seed</span>(<span class="dv">42</span>)</span>
-<span id="cb1452-2"><a href="logistic-regression.html#cb1452-2" aria-hidden="true" tabindex="-1"></a>example_data <span class="ot">=</span> <span class="fu">sim_quadratic_logistic_data</span>(<span class="at">sample_size =</span> <span class="dv">50</span>)</span></code></pre></div>
-<div class="sourceCode" id="cb1453"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1453-1"><a href="logistic-regression.html#cb1453-1" aria-hidden="true" tabindex="-1"></a>fit_glm <span class="ot">=</span> <span class="fu">glm</span>(y <span class="sc">~</span> x <span class="sc">+</span> <span class="fu">I</span>(x<span class="sc">^</span><span class="dv">2</span>), <span class="at">data =</span> example_data, <span class="at">family =</span> binomial)</span></code></pre></div>
-<div class="sourceCode" id="cb1454"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1454-1"><a href="logistic-regression.html#cb1454-1" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(y <span class="sc">~</span> x, <span class="at">data =</span> example_data, </span>
-<span id="cb1454-2"><a href="logistic-regression.html#cb1454-2" aria-hidden="true" tabindex="-1"></a>     <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">ylab =</span> <span class="st">&quot;Estimated Probability&quot;</span>, </span>
-<span id="cb1454-3"><a href="logistic-regression.html#cb1454-3" aria-hidden="true" tabindex="-1"></a>     <span class="at">main =</span> <span class="st">&quot;Logistic Regression, Quadratic Relationship&quot;</span>)</span>
-<span id="cb1454-4"><a href="logistic-regression.html#cb1454-4" aria-hidden="true" tabindex="-1"></a><span class="fu">grid</span>()</span>
-<span id="cb1454-5"><a href="logistic-regression.html#cb1454-5" aria-hidden="true" tabindex="-1"></a><span class="fu">curve</span>(<span class="fu">predict</span>(fit_glm, <span class="fu">data.frame</span>(x), <span class="at">type =</span> <span class="st">&quot;response&quot;</span>), </span>
-<span id="cb1454-6"><a href="logistic-regression.html#cb1454-6" aria-hidden="true" tabindex="-1"></a>      <span class="at">add =</span> <span class="cn">TRUE</span>, <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">lty =</span> <span class="dv">2</span>)</span>
-<span id="cb1454-7"><a href="logistic-regression.html#cb1454-7" aria-hidden="true" tabindex="-1"></a><span class="fu">curve</span>(boot<span class="sc">::</span><span class="fu">inv.logit</span>(<span class="sc">-</span><span class="fl">1.5</span> <span class="sc">+</span> <span class="fl">0.5</span> <span class="sc">*</span> x <span class="sc">+</span> x <span class="sc">^</span> <span class="dv">2</span>), </span>
-<span id="cb1454-8"><a href="logistic-regression.html#cb1454-8" aria-hidden="true" tabindex="-1"></a>      <span class="at">add =</span> <span class="cn">TRUE</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lty =</span> <span class="dv">1</span>)</span>
-<span id="cb1454-9"><a href="logistic-regression.html#cb1454-9" aria-hidden="true" tabindex="-1"></a><span class="fu">legend</span>(<span class="st">&quot;bottomleft&quot;</span>, <span class="fu">c</span>(<span class="st">&quot;True Probability&quot;</span>, <span class="st">&quot;Estimated Probability&quot;</span>, <span class="st">&quot;Data&quot;</span>), <span class="at">lty =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>, <span class="dv">0</span>), </span>
-<span id="cb1454-10"><a href="logistic-regression.html#cb1454-10" aria-hidden="true" tabindex="-1"></a>       <span class="at">pch =</span> <span class="fu">c</span>(<span class="cn">NA</span>, <span class="cn">NA</span>, <span class="dv">20</span>), <span class="at">lwd =</span> <span class="dv">2</span>, <span class="at">col =</span> <span class="fu">c</span>(<span class="st">&quot;darkorange&quot;</span>, <span class="st">&quot;dodgerblue&quot;</span>, <span class="st">&quot;black&quot;</span>))</span></code></pre></div>
+<div class="sourceCode" id="cb1452"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1452-1"><a href="logistic-regression.html#cb1452-1" tabindex="-1"></a><span class="fu">set.seed</span>(<span class="dv">42</span>)</span>
+<span id="cb1452-2"><a href="logistic-regression.html#cb1452-2" tabindex="-1"></a>example_data <span class="ot">=</span> <span class="fu">sim_quadratic_logistic_data</span>(<span class="at">sample_size =</span> <span class="dv">50</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1453"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1453-1"><a href="logistic-regression.html#cb1453-1" tabindex="-1"></a>fit_glm <span class="ot">=</span> <span class="fu">glm</span>(y <span class="sc">~</span> x <span class="sc">+</span> <span class="fu">I</span>(x<span class="sc">^</span><span class="dv">2</span>), <span class="at">data =</span> example_data, <span class="at">family =</span> binomial)</span></code></pre></div>
+<div class="sourceCode" id="cb1454"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1454-1"><a href="logistic-regression.html#cb1454-1" tabindex="-1"></a><span class="fu">plot</span>(y <span class="sc">~</span> x, <span class="at">data =</span> example_data, </span>
+<span id="cb1454-2"><a href="logistic-regression.html#cb1454-2" tabindex="-1"></a>     <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">ylab =</span> <span class="st">&quot;Estimated Probability&quot;</span>, </span>
+<span id="cb1454-3"><a href="logistic-regression.html#cb1454-3" tabindex="-1"></a>     <span class="at">main =</span> <span class="st">&quot;Logistic Regression, Quadratic Relationship&quot;</span>)</span>
+<span id="cb1454-4"><a href="logistic-regression.html#cb1454-4" tabindex="-1"></a><span class="fu">grid</span>()</span>
+<span id="cb1454-5"><a href="logistic-regression.html#cb1454-5" tabindex="-1"></a><span class="fu">curve</span>(<span class="fu">predict</span>(fit_glm, <span class="fu">data.frame</span>(x), <span class="at">type =</span> <span class="st">&quot;response&quot;</span>), </span>
+<span id="cb1454-6"><a href="logistic-regression.html#cb1454-6" tabindex="-1"></a>      <span class="at">add =</span> <span class="cn">TRUE</span>, <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">lty =</span> <span class="dv">2</span>)</span>
+<span id="cb1454-7"><a href="logistic-regression.html#cb1454-7" tabindex="-1"></a><span class="fu">curve</span>(boot<span class="sc">::</span><span class="fu">inv.logit</span>(<span class="sc">-</span><span class="fl">1.5</span> <span class="sc">+</span> <span class="fl">0.5</span> <span class="sc">*</span> x <span class="sc">+</span> x <span class="sc">^</span> <span class="dv">2</span>), </span>
+<span id="cb1454-8"><a href="logistic-regression.html#cb1454-8" tabindex="-1"></a>      <span class="at">add =</span> <span class="cn">TRUE</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lty =</span> <span class="dv">1</span>)</span>
+<span id="cb1454-9"><a href="logistic-regression.html#cb1454-9" tabindex="-1"></a><span class="fu">legend</span>(<span class="st">&quot;bottomleft&quot;</span>, <span class="fu">c</span>(<span class="st">&quot;True Probability&quot;</span>, <span class="st">&quot;Estimated Probability&quot;</span>, <span class="st">&quot;Data&quot;</span>), <span class="at">lty =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>, <span class="dv">0</span>), </span>
+<span id="cb1454-10"><a href="logistic-regression.html#cb1454-10" tabindex="-1"></a>       <span class="at">pch =</span> <span class="fu">c</span>(<span class="cn">NA</span>, <span class="cn">NA</span>, <span class="dv">20</span>), <span class="at">lwd =</span> <span class="dv">2</span>, <span class="at">col =</span> <span class="fu">c</span>(<span class="st">&quot;darkorange&quot;</span>, <span class="st">&quot;dodgerblue&quot;</span>, <span class="st">&quot;black&quot;</span>))</span></code></pre></div>
 <p><img src="logistic_files/figure-html/unnamed-chunk-17-1.png" width="672" style="display: block; margin: auto;" /></p>
 </div>
 </div>
@@ -936,12 +936,12 @@ <h3><span class="header-section-number">17.3.3</span> Likelihood-Ratio Test<a hr
 <div id="saheart-example" class="section level3 hasAnchor" number="17.3.4">
 <h3><span class="header-section-number">17.3.4</span> <code>SAheart</code> Example<a href="logistic-regression.html#saheart-example" class="anchor-section" aria-label="Anchor link to header"></a></h3>
 <p>To illustrate the use of logistic regression, we will use the <code>SAheart</code> dataset from the <code>bestglm</code> package.</p>
-<div class="sourceCode" id="cb1455"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1455-1"><a href="logistic-regression.html#cb1455-1" aria-hidden="true" tabindex="-1"></a><span class="co"># &quot;leaps&quot; is required for &quot;bestglm&quot;</span></span>
-<span id="cb1455-2"><a href="logistic-regression.html#cb1455-2" aria-hidden="true" tabindex="-1"></a><span class="cf">if</span> (<span class="sc">!</span><span class="fu">require</span>(<span class="st">&quot;leaps&quot;</span>)) <span class="fu">install.packages</span>(<span class="st">&quot;leaps&quot;</span>)</span>
-<span id="cb1455-3"><a href="logistic-regression.html#cb1455-3" aria-hidden="true" tabindex="-1"></a><span class="cf">if</span> (<span class="sc">!</span><span class="fu">require</span>(<span class="st">&quot;bestglm&quot;</span>)) <span class="fu">install.packages</span>(<span class="st">&quot;bestglm&quot;</span>)</span></code></pre></div>
-<div class="sourceCode" id="cb1456"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1456-1"><a href="logistic-regression.html#cb1456-1" aria-hidden="true" tabindex="-1"></a><span class="fu">library</span>(leaps)</span>
-<span id="cb1456-2"><a href="logistic-regression.html#cb1456-2" aria-hidden="true" tabindex="-1"></a><span class="fu">library</span>(bestglm)</span>
-<span id="cb1456-3"><a href="logistic-regression.html#cb1456-3" aria-hidden="true" tabindex="-1"></a><span class="fu">data</span>(<span class="st">&quot;SAheart&quot;</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1455"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1455-1"><a href="logistic-regression.html#cb1455-1" tabindex="-1"></a><span class="co"># &quot;leaps&quot; is required for &quot;bestglm&quot;</span></span>
+<span id="cb1455-2"><a href="logistic-regression.html#cb1455-2" tabindex="-1"></a><span class="cf">if</span> (<span class="sc">!</span><span class="fu">require</span>(<span class="st">&quot;leaps&quot;</span>)) <span class="fu">install.packages</span>(<span class="st">&quot;leaps&quot;</span>)</span>
+<span id="cb1455-3"><a href="logistic-regression.html#cb1455-3" tabindex="-1"></a><span class="cf">if</span> (<span class="sc">!</span><span class="fu">require</span>(<span class="st">&quot;bestglm&quot;</span>)) <span class="fu">install.packages</span>(<span class="st">&quot;bestglm&quot;</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1456"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1456-1"><a href="logistic-regression.html#cb1456-1" tabindex="-1"></a><span class="fu">library</span>(leaps)</span>
+<span id="cb1456-2"><a href="logistic-regression.html#cb1456-2" tabindex="-1"></a><span class="fu">library</span>(bestglm)</span>
+<span id="cb1456-3"><a href="logistic-regression.html#cb1456-3" tabindex="-1"></a><span class="fu">data</span>(<span class="st">&quot;SAheart&quot;</span>)</span></code></pre></div>
 <table>
 <colgroup>
 <col width="6%" />
@@ -1050,16 +1050,16 @@ <h3><span class="header-section-number">17.3.4</span> <code>SAheart</code> Examp
 <p><span class="math display">\[
 \log\left(\frac{P[\texttt{chd} = 1]}{1 - P[\texttt{chd} = 1]}\right) = \beta_0 + \beta_{\texttt{ldl}} x_{\texttt{ldl}}
 \]</span></p>
-<div class="sourceCode" id="cb1457"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1457-1"><a href="logistic-regression.html#cb1457-1" aria-hidden="true" tabindex="-1"></a>chd_mod_ldl <span class="ot">=</span> <span class="fu">glm</span>(chd <span class="sc">~</span> ldl, <span class="at">data =</span> SAheart, <span class="at">family =</span> binomial)</span>
-<span id="cb1457-2"><a href="logistic-regression.html#cb1457-2" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(<span class="fu">jitter</span>(chd, <span class="at">factor =</span> <span class="fl">0.1</span>) <span class="sc">~</span> ldl, <span class="at">data =</span> SAheart, <span class="at">pch =</span> <span class="dv">20</span>, </span>
-<span id="cb1457-3"><a href="logistic-regression.html#cb1457-3" aria-hidden="true" tabindex="-1"></a>     <span class="at">ylab =</span> <span class="st">&quot;Probability of CHD&quot;</span>, <span class="at">xlab =</span> <span class="st">&quot;Low Density Lipoprotein Cholesterol&quot;</span>)</span>
-<span id="cb1457-4"><a href="logistic-regression.html#cb1457-4" aria-hidden="true" tabindex="-1"></a><span class="fu">grid</span>()</span>
-<span id="cb1457-5"><a href="logistic-regression.html#cb1457-5" aria-hidden="true" tabindex="-1"></a><span class="fu">curve</span>(<span class="fu">predict</span>(chd_mod_ldl, <span class="fu">data.frame</span>(<span class="at">ldl =</span> x), <span class="at">type =</span> <span class="st">&quot;response&quot;</span>), </span>
-<span id="cb1457-6"><a href="logistic-regression.html#cb1457-6" aria-hidden="true" tabindex="-1"></a>      <span class="at">add =</span> <span class="cn">TRUE</span>, <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">lty =</span> <span class="dv">2</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1457"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1457-1"><a href="logistic-regression.html#cb1457-1" tabindex="-1"></a>chd_mod_ldl <span class="ot">=</span> <span class="fu">glm</span>(chd <span class="sc">~</span> ldl, <span class="at">data =</span> SAheart, <span class="at">family =</span> binomial)</span>
+<span id="cb1457-2"><a href="logistic-regression.html#cb1457-2" tabindex="-1"></a><span class="fu">plot</span>(<span class="fu">jitter</span>(chd, <span class="at">factor =</span> <span class="fl">0.1</span>) <span class="sc">~</span> ldl, <span class="at">data =</span> SAheart, <span class="at">pch =</span> <span class="dv">20</span>, </span>
+<span id="cb1457-3"><a href="logistic-regression.html#cb1457-3" tabindex="-1"></a>     <span class="at">ylab =</span> <span class="st">&quot;Probability of CHD&quot;</span>, <span class="at">xlab =</span> <span class="st">&quot;Low Density Lipoprotein Cholesterol&quot;</span>)</span>
+<span id="cb1457-4"><a href="logistic-regression.html#cb1457-4" tabindex="-1"></a><span class="fu">grid</span>()</span>
+<span id="cb1457-5"><a href="logistic-regression.html#cb1457-5" tabindex="-1"></a><span class="fu">curve</span>(<span class="fu">predict</span>(chd_mod_ldl, <span class="fu">data.frame</span>(<span class="at">ldl =</span> x), <span class="at">type =</span> <span class="st">&quot;response&quot;</span>), </span>
+<span id="cb1457-6"><a href="logistic-regression.html#cb1457-6" tabindex="-1"></a>      <span class="at">add =</span> <span class="cn">TRUE</span>, <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">lty =</span> <span class="dv">2</span>)</span></code></pre></div>
 <p><img src="logistic_files/figure-html/unnamed-chunk-21-1.png" width="672" style="display: block; margin: auto;" /></p>
 <p>As before, we plot the data in addition to the estimated probabilities. Note that we have “jittered” the data to make it easier to visualize, but the data do only take values <code>0</code> and <code>1</code>.</p>
 <p>As we would expect, this plot indicates that as <code>ldl</code> increases, so does the probability of <code>chd</code>.</p>
-<div class="sourceCode" id="cb1458"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1458-1"><a href="logistic-regression.html#cb1458-1" aria-hidden="true" tabindex="-1"></a><span class="fu">coef</span>(<span class="fu">summary</span>(chd_mod_ldl))</span></code></pre></div>
+<div class="sourceCode" id="cb1458"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1458-1"><a href="logistic-regression.html#cb1458-1" tabindex="-1"></a><span class="fu">coef</span>(<span class="fu">summary</span>(chd_mod_ldl))</span></code></pre></div>
 <pre><code>##               Estimate Std. Error   z value     Pr(&gt;|z|)
 ## (Intercept) -1.9686681 0.27307908 -7.209150 5.630207e-13
 ## ldl          0.2746613 0.05163983  5.318787 1.044615e-07</code></pre>
@@ -1069,16 +1069,16 @@ <h3><span class="header-section-number">17.3.4</span> <code>SAheart</code> Examp
 \]</span></p>
 <p>we use the <code>summary()</code> function as we have done so many times before. Like the <span class="math inline">\(t\)</span>-test for ordinary linear regression, this returns the estimate of the parameter, its standard error, the relevant test statistic (<span class="math inline">\(z\)</span>), and its p-value. Here we have an incredibly low p-value, so we reject the null hypothesis. The <code>ldl</code> variable appears to be a significant predictor.</p>
 <p>When fitting logistic regression, we can use the same formula syntax as ordinary linear regression. So, to fit an additive model using all available predictors, we use:</p>
-<div class="sourceCode" id="cb1460"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1460-1"><a href="logistic-regression.html#cb1460-1" aria-hidden="true" tabindex="-1"></a>chd_mod_additive <span class="ot">=</span> <span class="fu">glm</span>(chd <span class="sc">~</span> ., <span class="at">data =</span> SAheart, <span class="at">family =</span> binomial)</span></code></pre></div>
+<div class="sourceCode" id="cb1460"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1460-1"><a href="logistic-regression.html#cb1460-1" tabindex="-1"></a>chd_mod_additive <span class="ot">=</span> <span class="fu">glm</span>(chd <span class="sc">~</span> ., <span class="at">data =</span> SAheart, <span class="at">family =</span> binomial)</span></code></pre></div>
 <p>We can then use the likelihood-ratio test to compare the two models. Specifically, we are testing</p>
 <p><span class="math display">\[
 H_0: \beta_{\texttt{sbp}} = \beta_{\texttt{tobacco}} = \beta_{\texttt{adiposity}} = \beta_{\texttt{famhist}} = \beta_{\texttt{typea}} = \beta_{\texttt{obesity}} = \beta_{\texttt{alcohol}} = \beta_{\texttt{age}} = 0
 \]</span></p>
 <p>We could manually calculate the test statistic,</p>
-<div class="sourceCode" id="cb1461"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1461-1"><a href="logistic-regression.html#cb1461-1" aria-hidden="true" tabindex="-1"></a><span class="sc">-</span><span class="dv">2</span> <span class="sc">*</span> <span class="fu">as.numeric</span>(<span class="fu">logLik</span>(chd_mod_ldl) <span class="sc">-</span> <span class="fu">logLik</span>(chd_mod_additive))</span></code></pre></div>
+<div class="sourceCode" id="cb1461"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1461-1"><a href="logistic-regression.html#cb1461-1" tabindex="-1"></a><span class="sc">-</span><span class="dv">2</span> <span class="sc">*</span> <span class="fu">as.numeric</span>(<span class="fu">logLik</span>(chd_mod_ldl) <span class="sc">-</span> <span class="fu">logLik</span>(chd_mod_additive))</span></code></pre></div>
 <pre><code>## [1] 92.13879</code></pre>
 <p>Or we could utilize the <code>anova()</code> function. By specifying <code>test = "LRT"</code>, <code>R</code> will use the likelihood-ratio test to compare the two models.</p>
-<div class="sourceCode" id="cb1463"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1463-1"><a href="logistic-regression.html#cb1463-1" aria-hidden="true" tabindex="-1"></a><span class="fu">anova</span>(chd_mod_ldl, chd_mod_additive, <span class="at">test =</span> <span class="st">&quot;LRT&quot;</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1463"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1463-1"><a href="logistic-regression.html#cb1463-1" tabindex="-1"></a><span class="fu">anova</span>(chd_mod_ldl, chd_mod_additive, <span class="at">test =</span> <span class="st">&quot;LRT&quot;</span>)</span></code></pre></div>
 <pre><code>## Analysis of Deviance Table
 ## 
 ## Model 1: chd ~ ldl
@@ -1091,8 +1091,8 @@ <h3><span class="header-section-number">17.3.4</span> <code>SAheart</code> Examp
 ## Signif. codes:  0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1</code></pre>
 <p>We see that the test statistic that we had just calculated appears in the output. The very small p-value suggests that we prefer the larger model.</p>
 <p>While we prefer the additive model compared to the model with only a single predictor, do we actually need all of the predictors in the additive model? To select a subset of predictors, we can use a stepwise procedure as we did with ordinary linear regression. Recall that AIC and BIC were defined in terms of likelihoods. Here we demonstrate using AIC with a backwards selection procedure.</p>
-<div class="sourceCode" id="cb1465"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1465-1"><a href="logistic-regression.html#cb1465-1" aria-hidden="true" tabindex="-1"></a>chd_mod_selected <span class="ot">=</span> <span class="fu">step</span>(chd_mod_additive, <span class="at">trace =</span> <span class="dv">0</span>)</span>
-<span id="cb1465-2"><a href="logistic-regression.html#cb1465-2" aria-hidden="true" tabindex="-1"></a><span class="fu">coef</span>(chd_mod_selected)</span></code></pre></div>
+<div class="sourceCode" id="cb1465"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1465-1"><a href="logistic-regression.html#cb1465-1" tabindex="-1"></a>chd_mod_selected <span class="ot">=</span> <span class="fu">step</span>(chd_mod_additive, <span class="at">trace =</span> <span class="dv">0</span>)</span>
+<span id="cb1465-2"><a href="logistic-regression.html#cb1465-2" tabindex="-1"></a><span class="fu">coef</span>(chd_mod_selected)</span></code></pre></div>
 <pre><code>##    (Intercept)        tobacco            ldl famhistPresent          typea 
 ##    -6.44644451     0.08037533     0.16199164     0.90817526     0.03711521 
 ##            age 
@@ -1101,7 +1101,7 @@ <h3><span class="header-section-number">17.3.4</span> <code>SAheart</code> Examp
 <p><span class="math display">\[
 H_0: \beta_{\texttt{sbp}} = \beta_{\texttt{adiposity}} = \beta_{\texttt{obesity}} = \beta_{\texttt{alcohol}} = 0
 \]</span></p>
-<div class="sourceCode" id="cb1467"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1467-1"><a href="logistic-regression.html#cb1467-1" aria-hidden="true" tabindex="-1"></a><span class="fu">anova</span>(chd_mod_selected, chd_mod_additive, <span class="at">test =</span> <span class="st">&quot;LRT&quot;</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1467"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1467-1"><a href="logistic-regression.html#cb1467-1" tabindex="-1"></a><span class="fu">anova</span>(chd_mod_selected, chd_mod_additive, <span class="at">test =</span> <span class="st">&quot;LRT&quot;</span>)</span></code></pre></div>
 <pre><code>## Analysis of Deviance Table
 ## 
 ## Model 1: chd ~ tobacco + ldl + famhist + typea + age
@@ -1115,7 +1115,7 @@ <h3><span class="header-section-number">17.3.4</span> <code>SAheart</code> Examp
 <div id="confidence-intervals-1" class="section level3 hasAnchor" number="17.3.5">
 <h3><span class="header-section-number">17.3.5</span> Confidence Intervals<a href="logistic-regression.html#confidence-intervals-1" class="anchor-section" aria-label="Anchor link to header"></a></h3>
 <p>We can create confidence intervals for the <span class="math inline">\(\beta\)</span> parameters using the <code>confint()</code> function as we did with ordinary linear regression.</p>
-<div class="sourceCode" id="cb1469"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1469-1"><a href="logistic-regression.html#cb1469-1" aria-hidden="true" tabindex="-1"></a><span class="fu">confint</span>(chd_mod_selected, <span class="at">level =</span> <span class="fl">0.99</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1469"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1469-1"><a href="logistic-regression.html#cb1469-1" tabindex="-1"></a><span class="fu">confint</span>(chd_mod_selected, <span class="at">level =</span> <span class="fl">0.99</span>)</span></code></pre></div>
 <pre><code>## Waiting for profiling to be done...</code></pre>
 <pre><code>##                       0.5 %      99.5 %
 ## (Intercept)    -8.941825274 -4.18278990
@@ -1146,20 +1146,20 @@ <h3><span class="header-section-number">17.3.6</span> Confidence Intervals for M
 \left(\text{logit}^{-1}(\hat{\eta}({\bf x}) - z_{\alpha/2} \cdot \text{SE}[\hat{\eta}({\bf x})] ), \ \text{logit}^{-1}(\hat{\eta}({\bf x}) + z_{\alpha/2} \cdot \text{SE}[\hat{\eta}({\bf x})])\right)
 \]</span></p>
 <p>To demonstrate creating these intervals, we’ll consider a new observation.</p>
-<div class="sourceCode" id="cb1472"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1472-1"><a href="logistic-regression.html#cb1472-1" aria-hidden="true" tabindex="-1"></a>new_obs <span class="ot">=</span> <span class="fu">data.frame</span>(</span>
-<span id="cb1472-2"><a href="logistic-regression.html#cb1472-2" aria-hidden="true" tabindex="-1"></a>  <span class="at">sbp =</span> <span class="fl">148.0</span>,</span>
-<span id="cb1472-3"><a href="logistic-regression.html#cb1472-3" aria-hidden="true" tabindex="-1"></a>  <span class="at">tobacco =</span> <span class="dv">5</span>,</span>
-<span id="cb1472-4"><a href="logistic-regression.html#cb1472-4" aria-hidden="true" tabindex="-1"></a>  <span class="at">ldl =</span> <span class="dv">12</span>,</span>
-<span id="cb1472-5"><a href="logistic-regression.html#cb1472-5" aria-hidden="true" tabindex="-1"></a>  <span class="at">adiposity =</span> <span class="fl">31.23</span>,</span>
-<span id="cb1472-6"><a href="logistic-regression.html#cb1472-6" aria-hidden="true" tabindex="-1"></a>  <span class="at">famhist =</span> <span class="st">&quot;Present&quot;</span>,</span>
-<span id="cb1472-7"><a href="logistic-regression.html#cb1472-7" aria-hidden="true" tabindex="-1"></a>  <span class="at">typea =</span> <span class="dv">47</span>,</span>
-<span id="cb1472-8"><a href="logistic-regression.html#cb1472-8" aria-hidden="true" tabindex="-1"></a>  <span class="at">obesity =</span> <span class="fl">28.50</span>,</span>
-<span id="cb1472-9"><a href="logistic-regression.html#cb1472-9" aria-hidden="true" tabindex="-1"></a>  <span class="at">alcohol =</span> <span class="fl">23.89</span>,</span>
-<span id="cb1472-10"><a href="logistic-regression.html#cb1472-10" aria-hidden="true" tabindex="-1"></a>  <span class="at">age =</span> <span class="dv">60</span></span>
-<span id="cb1472-11"><a href="logistic-regression.html#cb1472-11" aria-hidden="true" tabindex="-1"></a>)</span></code></pre></div>
+<div class="sourceCode" id="cb1472"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1472-1"><a href="logistic-regression.html#cb1472-1" tabindex="-1"></a>new_obs <span class="ot">=</span> <span class="fu">data.frame</span>(</span>
+<span id="cb1472-2"><a href="logistic-regression.html#cb1472-2" tabindex="-1"></a>  <span class="at">sbp =</span> <span class="fl">148.0</span>,</span>
+<span id="cb1472-3"><a href="logistic-regression.html#cb1472-3" tabindex="-1"></a>  <span class="at">tobacco =</span> <span class="dv">5</span>,</span>
+<span id="cb1472-4"><a href="logistic-regression.html#cb1472-4" tabindex="-1"></a>  <span class="at">ldl =</span> <span class="dv">12</span>,</span>
+<span id="cb1472-5"><a href="logistic-regression.html#cb1472-5" tabindex="-1"></a>  <span class="at">adiposity =</span> <span class="fl">31.23</span>,</span>
+<span id="cb1472-6"><a href="logistic-regression.html#cb1472-6" tabindex="-1"></a>  <span class="at">famhist =</span> <span class="st">&quot;Present&quot;</span>,</span>
+<span id="cb1472-7"><a href="logistic-regression.html#cb1472-7" tabindex="-1"></a>  <span class="at">typea =</span> <span class="dv">47</span>,</span>
+<span id="cb1472-8"><a href="logistic-regression.html#cb1472-8" tabindex="-1"></a>  <span class="at">obesity =</span> <span class="fl">28.50</span>,</span>
+<span id="cb1472-9"><a href="logistic-regression.html#cb1472-9" tabindex="-1"></a>  <span class="at">alcohol =</span> <span class="fl">23.89</span>,</span>
+<span id="cb1472-10"><a href="logistic-regression.html#cb1472-10" tabindex="-1"></a>  <span class="at">age =</span> <span class="dv">60</span></span>
+<span id="cb1472-11"><a href="logistic-regression.html#cb1472-11" tabindex="-1"></a>)</span></code></pre></div>
 <p>First, we’ll use the <code>predict()</code> function to obtain <span class="math inline">\(\hat{\eta}({\bf x})\)</span> for this observation.</p>
-<div class="sourceCode" id="cb1473"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1473-1"><a href="logistic-regression.html#cb1473-1" aria-hidden="true" tabindex="-1"></a>eta_hat <span class="ot">=</span> <span class="fu">predict</span>(chd_mod_selected, new_obs, <span class="at">se.fit =</span> <span class="cn">TRUE</span>, <span class="at">type =</span> <span class="st">&quot;link&quot;</span>)</span>
-<span id="cb1473-2"><a href="logistic-regression.html#cb1473-2" aria-hidden="true" tabindex="-1"></a>eta_hat</span></code></pre></div>
+<div class="sourceCode" id="cb1473"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1473-1"><a href="logistic-regression.html#cb1473-1" tabindex="-1"></a>eta_hat <span class="ot">=</span> <span class="fu">predict</span>(chd_mod_selected, new_obs, <span class="at">se.fit =</span> <span class="cn">TRUE</span>, <span class="at">type =</span> <span class="st">&quot;link&quot;</span>)</span>
+<span id="cb1473-2"><a href="logistic-regression.html#cb1473-2" tabindex="-1"></a>eta_hat</span></code></pre></div>
 <pre><code>## $fit
 ##        1 
 ## 1.579545 
@@ -1170,14 +1170,14 @@ <h3><span class="header-section-number">17.3.6</span> Confidence Intervals for M
 ## $residual.scale
 ## [1] 1</code></pre>
 <p>By setting <code>se.fit = TRUE</code>, <code>R</code> also computes <span class="math inline">\(\text{SE}[\hat{\eta}({\bf x})]\)</span>. Note that we used <code>type = "link"</code>, but this is actually a default value. We added it here to stress that the output from <code>predict()</code> will be the value of the link function.</p>
-<div class="sourceCode" id="cb1475"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1475-1"><a href="logistic-regression.html#cb1475-1" aria-hidden="true" tabindex="-1"></a>z_crit <span class="ot">=</span> <span class="fu">qnorm</span>(<span class="fl">0.975</span>)</span>
-<span id="cb1475-2"><a href="logistic-regression.html#cb1475-2" aria-hidden="true" tabindex="-1"></a><span class="fu">round</span>(z_crit, <span class="dv">2</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1475"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1475-1"><a href="logistic-regression.html#cb1475-1" tabindex="-1"></a>z_crit <span class="ot">=</span> <span class="fu">qnorm</span>(<span class="fl">0.975</span>)</span>
+<span id="cb1475-2"><a href="logistic-regression.html#cb1475-2" tabindex="-1"></a><span class="fu">round</span>(z_crit, <span class="dv">2</span>)</span></code></pre></div>
 <pre><code>## [1] 1.96</code></pre>
 <p>After obtaining the correct critical value, we can easily create a <span class="math inline">\(95\%\)</span> confidence interval for <span class="math inline">\(\eta({\bf x})\)</span>.</p>
-<div class="sourceCode" id="cb1477"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1477-1"><a href="logistic-regression.html#cb1477-1" aria-hidden="true" tabindex="-1"></a>eta_hat<span class="sc">$</span>fit <span class="sc">+</span> <span class="fu">c</span>(<span class="sc">-</span><span class="dv">1</span>, <span class="dv">1</span>) <span class="sc">*</span> z_crit <span class="sc">*</span> eta_hat<span class="sc">$</span>se.fit</span></code></pre></div>
+<div class="sourceCode" id="cb1477"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1477-1"><a href="logistic-regression.html#cb1477-1" tabindex="-1"></a>eta_hat<span class="sc">$</span>fit <span class="sc">+</span> <span class="fu">c</span>(<span class="sc">-</span><span class="dv">1</span>, <span class="dv">1</span>) <span class="sc">*</span> z_crit <span class="sc">*</span> eta_hat<span class="sc">$</span>se.fit</span></code></pre></div>
 <pre><code>## [1] 0.7730599 2.3860302</code></pre>
 <p>Now we simply need to apply the correct transformation to make this a confidence interval for <span class="math inline">\(p({\bf x})\)</span>, the probability of coronary heart disease for this observation. Note that the <code>boot</code> package contains functions <code>logit()</code> and <code>inv.logit()</code> which are the logit and inverse logit transformations, respectively.</p>
-<div class="sourceCode" id="cb1479"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1479-1"><a href="logistic-regression.html#cb1479-1" aria-hidden="true" tabindex="-1"></a>boot<span class="sc">::</span><span class="fu">inv.logit</span>(eta_hat<span class="sc">$</span>fit <span class="sc">+</span> <span class="fu">c</span>(<span class="sc">-</span><span class="dv">1</span>, <span class="dv">1</span>) <span class="sc">*</span> z_crit <span class="sc">*</span> eta_hat<span class="sc">$</span>se.fit)</span></code></pre></div>
+<div class="sourceCode" id="cb1479"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1479-1"><a href="logistic-regression.html#cb1479-1" tabindex="-1"></a>boot<span class="sc">::</span><span class="fu">inv.logit</span>(eta_hat<span class="sc">$</span>fit <span class="sc">+</span> <span class="fu">c</span>(<span class="sc">-</span><span class="dv">1</span>, <span class="dv">1</span>) <span class="sc">*</span> z_crit <span class="sc">*</span> eta_hat<span class="sc">$</span>se.fit)</span></code></pre></div>
 <pre><code>## [1] 0.6841824 0.9157558</code></pre>
 <p>Notice, as we would expect, the bounds of this interval are both between 0 and 1. Also, since both bounds of the interval for <span class="math inline">\(\eta({\bf x})\)</span> are positive, both bounds of the interval for <span class="math inline">\(p({\bf x})\)</span> are greater than 0.5.</p>
 </div>
@@ -1187,9 +1187,9 @@ <h3><span class="header-section-number">17.3.7</span> Formula Syntax<a href="log
 <div id="interactions-1" class="section level4 hasAnchor" number="17.3.7.1">
 <h4><span class="header-section-number">17.3.7.1</span> Interactions<a href="logistic-regression.html#interactions-1" class="anchor-section" aria-label="Anchor link to header"></a></h4>
 <p>Let’s add an interaction between LDL and family history for the model we selected.</p>
-<div class="sourceCode" id="cb1481"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1481-1"><a href="logistic-regression.html#cb1481-1" aria-hidden="true" tabindex="-1"></a>chd_mod_interaction <span class="ot">=</span> <span class="fu">glm</span>(chd <span class="sc">~</span> alcohol <span class="sc">+</span> ldl <span class="sc">+</span> famhist <span class="sc">+</span> typea <span class="sc">+</span> age <span class="sc">+</span> ldl<span class="sc">:</span>famhist, </span>
-<span id="cb1481-2"><a href="logistic-regression.html#cb1481-2" aria-hidden="true" tabindex="-1"></a>                          <span class="at">data =</span> SAheart, <span class="at">family =</span> binomial)</span>
-<span id="cb1481-3"><a href="logistic-regression.html#cb1481-3" aria-hidden="true" tabindex="-1"></a><span class="fu">summary</span>(chd_mod_interaction)</span></code></pre></div>
+<div class="sourceCode" id="cb1481"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1481-1"><a href="logistic-regression.html#cb1481-1" tabindex="-1"></a>chd_mod_interaction <span class="ot">=</span> <span class="fu">glm</span>(chd <span class="sc">~</span> alcohol <span class="sc">+</span> ldl <span class="sc">+</span> famhist <span class="sc">+</span> typea <span class="sc">+</span> age <span class="sc">+</span> ldl<span class="sc">:</span>famhist, </span>
+<span id="cb1481-2"><a href="logistic-regression.html#cb1481-2" tabindex="-1"></a>                          <span class="at">data =</span> SAheart, <span class="at">family =</span> binomial)</span>
+<span id="cb1481-3"><a href="logistic-regression.html#cb1481-3" tabindex="-1"></a><span class="fu">summary</span>(chd_mod_interaction)</span></code></pre></div>
 <pre><code>## 
 ## Call:
 ## glm(formula = chd ~ alcohol + ldl + famhist + typea + age + ldl:famhist, 
@@ -1219,9 +1219,9 @@ <h4><span class="header-section-number">17.3.7.1</span> Interactions<a href="log
 <div id="polynomial-terms" class="section level4 hasAnchor" number="17.3.7.2">
 <h4><span class="header-section-number">17.3.7.2</span> Polynomial Terms<a href="logistic-regression.html#polynomial-terms" class="anchor-section" aria-label="Anchor link to header"></a></h4>
 <p>Let’s take the previous model, and now add a polynomial term.</p>
-<div class="sourceCode" id="cb1483"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1483-1"><a href="logistic-regression.html#cb1483-1" aria-hidden="true" tabindex="-1"></a>chd_mod_int_quad <span class="ot">=</span> <span class="fu">glm</span>(chd <span class="sc">~</span> alcohol <span class="sc">+</span> ldl <span class="sc">+</span> famhist <span class="sc">+</span> typea <span class="sc">+</span> age <span class="sc">+</span> ldl<span class="sc">:</span>famhist <span class="sc">+</span> <span class="fu">I</span>(ldl<span class="sc">^</span><span class="dv">2</span>),</span>
-<span id="cb1483-2"><a href="logistic-regression.html#cb1483-2" aria-hidden="true" tabindex="-1"></a>                       <span class="at">data =</span> SAheart, <span class="at">family =</span> binomial)</span>
-<span id="cb1483-3"><a href="logistic-regression.html#cb1483-3" aria-hidden="true" tabindex="-1"></a><span class="fu">summary</span>(chd_mod_int_quad)</span></code></pre></div>
+<div class="sourceCode" id="cb1483"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1483-1"><a href="logistic-regression.html#cb1483-1" tabindex="-1"></a>chd_mod_int_quad <span class="ot">=</span> <span class="fu">glm</span>(chd <span class="sc">~</span> alcohol <span class="sc">+</span> ldl <span class="sc">+</span> famhist <span class="sc">+</span> typea <span class="sc">+</span> age <span class="sc">+</span> ldl<span class="sc">:</span>famhist <span class="sc">+</span> <span class="fu">I</span>(ldl<span class="sc">^</span><span class="dv">2</span>),</span>
+<span id="cb1483-2"><a href="logistic-regression.html#cb1483-2" tabindex="-1"></a>                       <span class="at">data =</span> SAheart, <span class="at">family =</span> binomial)</span>
+<span id="cb1483-3"><a href="logistic-regression.html#cb1483-3" tabindex="-1"></a><span class="fu">summary</span>(chd_mod_int_quad)</span></code></pre></div>
 <pre><code>## 
 ## Call:
 ## glm(formula = chd ~ alcohol + ldl + famhist + typea + age + ldl:famhist + 
@@ -1267,11 +1267,11 @@ <h4><span class="header-section-number">17.3.7.2</span> Polynomial Terms<a href=
 <h3><span class="header-section-number">17.3.8</span> Deviance<a href="logistic-regression.html#deviance" class="anchor-section" aria-label="Anchor link to header"></a></h3>
 <p>You have probably noticed that the output from <code>summary()</code> is also very similar to that of ordinary linear regression. One difference, is the “deviance” being reported. The <code>Null deviance</code> is the deviance for the null model, that is, a model with no predictors. The <code>Residual deviance</code> is the deviance for the model that was fit.</p>
 <p><a href="https://en.wikipedia.org/wiki/Deviance_(statistics)" target="_blank"><strong>Deviance</strong></a> compares the model to a saturated model. (Without repeated observations, a saturated model is a model that fits perfectly, using a parameter for each observation.) Essentially, deviance is a generalized <em>residual sum of squares</em> for GLMs. Like RSS, deviance decreases as the model complexity increases.</p>
-<div class="sourceCode" id="cb1485"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1485-1"><a href="logistic-regression.html#cb1485-1" aria-hidden="true" tabindex="-1"></a><span class="fu">deviance</span>(chd_mod_ldl)</span></code></pre></div>
+<div class="sourceCode" id="cb1485"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1485-1"><a href="logistic-regression.html#cb1485-1" tabindex="-1"></a><span class="fu">deviance</span>(chd_mod_ldl)</span></code></pre></div>
 <pre><code>## [1] 564.2788</code></pre>
-<div class="sourceCode" id="cb1487"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1487-1"><a href="logistic-regression.html#cb1487-1" aria-hidden="true" tabindex="-1"></a><span class="fu">deviance</span>(chd_mod_selected)</span></code></pre></div>
+<div class="sourceCode" id="cb1487"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1487-1"><a href="logistic-regression.html#cb1487-1" tabindex="-1"></a><span class="fu">deviance</span>(chd_mod_selected)</span></code></pre></div>
 <pre><code>## [1] 475.6856</code></pre>
-<div class="sourceCode" id="cb1489"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1489-1"><a href="logistic-regression.html#cb1489-1" aria-hidden="true" tabindex="-1"></a><span class="fu">deviance</span>(chd_mod_additive)</span></code></pre></div>
+<div class="sourceCode" id="cb1489"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1489-1"><a href="logistic-regression.html#cb1489-1" tabindex="-1"></a><span class="fu">deviance</span>(chd_mod_additive)</span></code></pre></div>
 <pre><code>## [1] 472.14</code></pre>
 <p>Note that these are nested, and we see that deviance does decrease as the model size becomes larger. So while a lower deviance is better, if the model becomes too big, it may be overfitting. Note that <code>R</code> also outputs AIC in the summary, which will penalize according to model size, to prevent overfitting.</p>
 </div>
@@ -1316,10 +1316,10 @@ <h2><span class="header-section-number">17.4</span> Classification<a href="logis
 <div id="spam-example" class="section level3 hasAnchor" number="17.4.1">
 <h3><span class="header-section-number">17.4.1</span> <code>spam</code> Example<a href="logistic-regression.html#spam-example" class="anchor-section" aria-label="Anchor link to header"></a></h3>
 <p>To illustrate the use of logistic regression as a classifier, we will use the <code>spam</code> dataset from the <code>kernlab</code> package.</p>
-<div class="sourceCode" id="cb1491"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1491-1"><a href="logistic-regression.html#cb1491-1" aria-hidden="true" tabindex="-1"></a><span class="co"># install.packages(&quot;kernlab&quot;)</span></span>
-<span id="cb1491-2"><a href="logistic-regression.html#cb1491-2" aria-hidden="true" tabindex="-1"></a><span class="fu">library</span>(kernlab)</span>
-<span id="cb1491-3"><a href="logistic-regression.html#cb1491-3" aria-hidden="true" tabindex="-1"></a><span class="fu">data</span>(<span class="st">&quot;spam&quot;</span>)</span>
-<span id="cb1491-4"><a href="logistic-regression.html#cb1491-4" aria-hidden="true" tabindex="-1"></a>tibble<span class="sc">::</span><span class="fu">as.tibble</span>(spam)</span></code></pre></div>
+<div class="sourceCode" id="cb1491"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1491-1"><a href="logistic-regression.html#cb1491-1" tabindex="-1"></a><span class="co"># install.packages(&quot;kernlab&quot;)</span></span>
+<span id="cb1491-2"><a href="logistic-regression.html#cb1491-2" tabindex="-1"></a><span class="fu">library</span>(kernlab)</span>
+<span id="cb1491-3"><a href="logistic-regression.html#cb1491-3" tabindex="-1"></a><span class="fu">data</span>(<span class="st">&quot;spam&quot;</span>)</span>
+<span id="cb1491-4"><a href="logistic-regression.html#cb1491-4" tabindex="-1"></a>tibble<span class="sc">::</span><span class="fu">as.tibble</span>(spam)</span></code></pre></div>
 <pre><code>## Warning: `as.tibble()` was deprecated in tibble 2.0.0.
 ## ℹ Please use `as_tibble()` instead.
 ## ℹ The signature and semantics have changed, see `?as_tibble`.
@@ -1348,30 +1348,30 @@ <h3><span class="header-section-number">17.4.1</span> <code>spam</code> Example<
 ## #   num1999 &lt;dbl&gt;, parts &lt;dbl&gt;, pm &lt;dbl&gt;, direct &lt;dbl&gt;, cs &lt;dbl&gt;, …</code></pre>
 <p>This dataset, created in the late 1990s at Hewlett-Packard Labs, contains 4601 emails, of which 1813 are considered spam. The remaining are not spam. (Which for simplicity, we might call, ham.) Additional details can be obtained by using <code>?spam</code> or by visiting the <a href="https://archive.ics.uci.edu/ml/datasets/spambase" target="_blank">UCI Machine Learning Repository</a>.</p>
 <p>The response variable, <code>type</code>, is a <strong>factor</strong> with levels that label each email as <code>spam</code> or <code>nonspam</code>. When fitting models, <code>nonspam</code> will be the reference level, <span class="math inline">\(Y = 0\)</span>, as it comes first alphabetically.</p>
-<div class="sourceCode" id="cb1494"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1494-1"><a href="logistic-regression.html#cb1494-1" aria-hidden="true" tabindex="-1"></a><span class="fu">is.factor</span>(spam<span class="sc">$</span>type)</span></code></pre></div>
+<div class="sourceCode" id="cb1494"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1494-1"><a href="logistic-regression.html#cb1494-1" tabindex="-1"></a><span class="fu">is.factor</span>(spam<span class="sc">$</span>type)</span></code></pre></div>
 <pre><code>## [1] TRUE</code></pre>
-<div class="sourceCode" id="cb1496"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1496-1"><a href="logistic-regression.html#cb1496-1" aria-hidden="true" tabindex="-1"></a><span class="fu">levels</span>(spam<span class="sc">$</span>type)</span></code></pre></div>
+<div class="sourceCode" id="cb1496"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1496-1"><a href="logistic-regression.html#cb1496-1" tabindex="-1"></a><span class="fu">levels</span>(spam<span class="sc">$</span>type)</span></code></pre></div>
 <pre><code>## [1] &quot;nonspam&quot; &quot;spam&quot;</code></pre>
 <p>Many of the predictors (often called features in machine learning) are engineered based on the emails. For example, <code>charDollar</code> is the number of times an email contains the <code>$</code> character. Some variables are highly specific to this dataset, for example <code>george</code> and <code>num650</code>. (The name and area code for one of the researchers whose emails were used.) We should keep in mind that this dataset was created based on emails sent to academic type researchers in the 1990s. Any results we derive probably won’t generalize to modern emails for the general public.</p>
 <p>To get started, we’ll first test-train split the data.</p>
-<div class="sourceCode" id="cb1498"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1498-1"><a href="logistic-regression.html#cb1498-1" aria-hidden="true" tabindex="-1"></a><span class="fu">set.seed</span>(<span class="dv">42</span>)</span>
-<span id="cb1498-2"><a href="logistic-regression.html#cb1498-2" aria-hidden="true" tabindex="-1"></a><span class="co"># spam_idx = sample(nrow(spam), round(nrow(spam) / 2))</span></span>
-<span id="cb1498-3"><a href="logistic-regression.html#cb1498-3" aria-hidden="true" tabindex="-1"></a>spam_idx <span class="ot">=</span> <span class="fu">sample</span>(<span class="fu">nrow</span>(spam), <span class="dv">1000</span>)</span>
-<span id="cb1498-4"><a href="logistic-regression.html#cb1498-4" aria-hidden="true" tabindex="-1"></a>spam_trn <span class="ot">=</span> spam[spam_idx, ]</span>
-<span id="cb1498-5"><a href="logistic-regression.html#cb1498-5" aria-hidden="true" tabindex="-1"></a>spam_tst <span class="ot">=</span> spam[<span class="sc">-</span>spam_idx, ]</span></code></pre></div>
+<div class="sourceCode" id="cb1498"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1498-1"><a href="logistic-regression.html#cb1498-1" tabindex="-1"></a><span class="fu">set.seed</span>(<span class="dv">42</span>)</span>
+<span id="cb1498-2"><a href="logistic-regression.html#cb1498-2" tabindex="-1"></a><span class="co"># spam_idx = sample(nrow(spam), round(nrow(spam) / 2))</span></span>
+<span id="cb1498-3"><a href="logistic-regression.html#cb1498-3" tabindex="-1"></a>spam_idx <span class="ot">=</span> <span class="fu">sample</span>(<span class="fu">nrow</span>(spam), <span class="dv">1000</span>)</span>
+<span id="cb1498-4"><a href="logistic-regression.html#cb1498-4" tabindex="-1"></a>spam_trn <span class="ot">=</span> spam[spam_idx, ]</span>
+<span id="cb1498-5"><a href="logistic-regression.html#cb1498-5" tabindex="-1"></a>spam_tst <span class="ot">=</span> spam[<span class="sc">-</span>spam_idx, ]</span></code></pre></div>
 <p>We’ve used a somewhat small train set relative to the total size of the dataset. In practice it should likely be larger, but this is simply to keep training time low for illustration and rendering of this document.</p>
-<div class="sourceCode" id="cb1499"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1499-1"><a href="logistic-regression.html#cb1499-1" aria-hidden="true" tabindex="-1"></a>fit_caps <span class="ot">=</span> <span class="fu">glm</span>(type <span class="sc">~</span> capitalTotal, </span>
-<span id="cb1499-2"><a href="logistic-regression.html#cb1499-2" aria-hidden="true" tabindex="-1"></a>               <span class="at">data =</span> spam_trn, <span class="at">family =</span> binomial)</span>
-<span id="cb1499-3"><a href="logistic-regression.html#cb1499-3" aria-hidden="true" tabindex="-1"></a>fit_selected <span class="ot">=</span> <span class="fu">glm</span>(type <span class="sc">~</span> edu <span class="sc">+</span> money <span class="sc">+</span> capitalTotal <span class="sc">+</span> charDollar, </span>
-<span id="cb1499-4"><a href="logistic-regression.html#cb1499-4" aria-hidden="true" tabindex="-1"></a>                   <span class="at">data =</span> spam_trn, <span class="at">family =</span> binomial)</span>
-<span id="cb1499-5"><a href="logistic-regression.html#cb1499-5" aria-hidden="true" tabindex="-1"></a>fit_additive <span class="ot">=</span> <span class="fu">glm</span>(type <span class="sc">~</span> ., </span>
-<span id="cb1499-6"><a href="logistic-regression.html#cb1499-6" aria-hidden="true" tabindex="-1"></a>                   <span class="at">data =</span> spam_trn, <span class="at">family =</span> binomial)</span>
-<span id="cb1499-7"><a href="logistic-regression.html#cb1499-7" aria-hidden="true" tabindex="-1"></a>fit_over <span class="ot">=</span> <span class="fu">glm</span>(type <span class="sc">~</span> capitalTotal <span class="sc">*</span> (.), </span>
-<span id="cb1499-8"><a href="logistic-regression.html#cb1499-8" aria-hidden="true" tabindex="-1"></a>               <span class="at">data =</span> spam_trn, <span class="at">family =</span> binomial, <span class="at">maxit =</span> <span class="dv">75</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1499"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1499-1"><a href="logistic-regression.html#cb1499-1" tabindex="-1"></a>fit_caps <span class="ot">=</span> <span class="fu">glm</span>(type <span class="sc">~</span> capitalTotal, </span>
+<span id="cb1499-2"><a href="logistic-regression.html#cb1499-2" tabindex="-1"></a>               <span class="at">data =</span> spam_trn, <span class="at">family =</span> binomial)</span>
+<span id="cb1499-3"><a href="logistic-regression.html#cb1499-3" tabindex="-1"></a>fit_selected <span class="ot">=</span> <span class="fu">glm</span>(type <span class="sc">~</span> edu <span class="sc">+</span> money <span class="sc">+</span> capitalTotal <span class="sc">+</span> charDollar, </span>
+<span id="cb1499-4"><a href="logistic-regression.html#cb1499-4" tabindex="-1"></a>                   <span class="at">data =</span> spam_trn, <span class="at">family =</span> binomial)</span>
+<span id="cb1499-5"><a href="logistic-regression.html#cb1499-5" tabindex="-1"></a>fit_additive <span class="ot">=</span> <span class="fu">glm</span>(type <span class="sc">~</span> ., </span>
+<span id="cb1499-6"><a href="logistic-regression.html#cb1499-6" tabindex="-1"></a>                   <span class="at">data =</span> spam_trn, <span class="at">family =</span> binomial)</span>
+<span id="cb1499-7"><a href="logistic-regression.html#cb1499-7" tabindex="-1"></a>fit_over <span class="ot">=</span> <span class="fu">glm</span>(type <span class="sc">~</span> capitalTotal <span class="sc">*</span> (.), </span>
+<span id="cb1499-8"><a href="logistic-regression.html#cb1499-8" tabindex="-1"></a>               <span class="at">data =</span> spam_trn, <span class="at">family =</span> binomial, <span class="at">maxit =</span> <span class="dv">75</span>)</span></code></pre></div>
 <p>We’ll fit four logistic regressions, each more complex than the previous. Note that we’re suppressing two warnings. The first we briefly mentioned previously.</p>
 <pre><code>## Warning: glm.fit: fitted probabilities numerically 0 or 1 occurred</code></pre>
 <p>Note that, when we receive this warning, we should be highly suspicious of the parameter estimates.</p>
-<div class="sourceCode" id="cb1501"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1501-1"><a href="logistic-regression.html#cb1501-1" aria-hidden="true" tabindex="-1"></a><span class="fu">coef</span>(fit_selected)</span></code></pre></div>
+<div class="sourceCode" id="cb1501"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1501-1"><a href="logistic-regression.html#cb1501-1" tabindex="-1"></a><span class="fu">coef</span>(fit_selected)</span></code></pre></div>
 <pre><code>##   (Intercept)           edu         money  capitalTotal    charDollar 
 ## -1.1199744712 -1.9837988840  0.9784675298  0.0007757011 11.5772904667</code></pre>
 <p>However, the model can still be used to create a classifier, and we will evaluate that classifier on its own merits.</p>
@@ -1393,14 +1393,14 @@ <h3><span class="header-section-number">17.4.2</span> Evaluating Classifiers<a h
 \end{cases}
 \]</span></p>
 <p>When using this metric on the training data, it will have the same issues as RSS did for ordinary linear regression, that is, it will only go down.</p>
-<div class="sourceCode" id="cb1504"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1504-1"><a href="logistic-regression.html#cb1504-1" aria-hidden="true" tabindex="-1"></a><span class="co"># training misclassification rate</span></span>
-<span id="cb1504-2"><a href="logistic-regression.html#cb1504-2" aria-hidden="true" tabindex="-1"></a><span class="fu">mean</span>(<span class="fu">ifelse</span>(<span class="fu">predict</span>(fit_caps) <span class="sc">&gt;</span> <span class="dv">0</span>, <span class="st">&quot;spam&quot;</span>, <span class="st">&quot;nonspam&quot;</span>) <span class="sc">!=</span> spam_trn<span class="sc">$</span>type)</span></code></pre></div>
+<div class="sourceCode" id="cb1504"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1504-1"><a href="logistic-regression.html#cb1504-1" tabindex="-1"></a><span class="co"># training misclassification rate</span></span>
+<span id="cb1504-2"><a href="logistic-regression.html#cb1504-2" tabindex="-1"></a><span class="fu">mean</span>(<span class="fu">ifelse</span>(<span class="fu">predict</span>(fit_caps) <span class="sc">&gt;</span> <span class="dv">0</span>, <span class="st">&quot;spam&quot;</span>, <span class="st">&quot;nonspam&quot;</span>) <span class="sc">!=</span> spam_trn<span class="sc">$</span>type)</span></code></pre></div>
 <pre><code>## [1] 0.339</code></pre>
-<div class="sourceCode" id="cb1506"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1506-1"><a href="logistic-regression.html#cb1506-1" aria-hidden="true" tabindex="-1"></a><span class="fu">mean</span>(<span class="fu">ifelse</span>(<span class="fu">predict</span>(fit_selected) <span class="sc">&gt;</span> <span class="dv">0</span>, <span class="st">&quot;spam&quot;</span>, <span class="st">&quot;nonspam&quot;</span>) <span class="sc">!=</span> spam_trn<span class="sc">$</span>type)</span></code></pre></div>
+<div class="sourceCode" id="cb1506"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1506-1"><a href="logistic-regression.html#cb1506-1" tabindex="-1"></a><span class="fu">mean</span>(<span class="fu">ifelse</span>(<span class="fu">predict</span>(fit_selected) <span class="sc">&gt;</span> <span class="dv">0</span>, <span class="st">&quot;spam&quot;</span>, <span class="st">&quot;nonspam&quot;</span>) <span class="sc">!=</span> spam_trn<span class="sc">$</span>type)</span></code></pre></div>
 <pre><code>## [1] 0.224</code></pre>
-<div class="sourceCode" id="cb1508"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1508-1"><a href="logistic-regression.html#cb1508-1" aria-hidden="true" tabindex="-1"></a><span class="fu">mean</span>(<span class="fu">ifelse</span>(<span class="fu">predict</span>(fit_additive) <span class="sc">&gt;</span> <span class="dv">0</span>, <span class="st">&quot;spam&quot;</span>, <span class="st">&quot;nonspam&quot;</span>) <span class="sc">!=</span> spam_trn<span class="sc">$</span>type)</span></code></pre></div>
+<div class="sourceCode" id="cb1508"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1508-1"><a href="logistic-regression.html#cb1508-1" tabindex="-1"></a><span class="fu">mean</span>(<span class="fu">ifelse</span>(<span class="fu">predict</span>(fit_additive) <span class="sc">&gt;</span> <span class="dv">0</span>, <span class="st">&quot;spam&quot;</span>, <span class="st">&quot;nonspam&quot;</span>) <span class="sc">!=</span> spam_trn<span class="sc">$</span>type)</span></code></pre></div>
 <pre><code>## [1] 0.066</code></pre>
-<div class="sourceCode" id="cb1510"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1510-1"><a href="logistic-regression.html#cb1510-1" aria-hidden="true" tabindex="-1"></a><span class="fu">mean</span>(<span class="fu">ifelse</span>(<span class="fu">predict</span>(fit_over) <span class="sc">&gt;</span> <span class="dv">0</span>, <span class="st">&quot;spam&quot;</span>, <span class="st">&quot;nonspam&quot;</span>) <span class="sc">!=</span> spam_trn<span class="sc">$</span>type)</span></code></pre></div>
+<div class="sourceCode" id="cb1510"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1510-1"><a href="logistic-regression.html#cb1510-1" tabindex="-1"></a><span class="fu">mean</span>(<span class="fu">ifelse</span>(<span class="fu">predict</span>(fit_over) <span class="sc">&gt;</span> <span class="dv">0</span>, <span class="st">&quot;spam&quot;</span>, <span class="st">&quot;nonspam&quot;</span>) <span class="sc">!=</span> spam_trn<span class="sc">$</span>type)</span></code></pre></div>
 <pre><code>## [1] 0.118</code></pre>
 <p>Because of this, training data isn’t useful for evaluating, as it would suggest that we should always use the largest possible model, when in reality, that model is likely overfitting. Recall, a model that is too complex will overfit. A model that is too simple will underfit. (We’re looking for something in the middle.)</p>
 <p>To overcome this, we’ll use cross-validation as we did with ordinary linear regression, but this time we’ll cross-validate the misclassification rate. To do so, we’ll use the <code>cv.glm()</code> function from the <code>boot</code> library. It takes arguments for the data (in this case training), a model fit via <code>glm()</code>, and <code>K</code>, the number of folds. It returns <code>delta</code>, a vector of the average mean-squared error (MSE) and the bias-corrected MSE. See <code>?cv.glm</code> for details.</p>
@@ -1412,41 +1412,41 @@ <h3><span class="header-section-number">17.4.2</span> Evaluating Classifiers<a h
 <li>Evaluate misclassification rate on held-out data</li>
 </ul>
 <p>The 5-fold cross-validated misclassification rate will be the average of these misclassification rates. By only needing to refit the model 5 times, instead of <span class="math inline">\(n\)</span> times, we will save a lot of computation time.</p>
-<div class="sourceCode" id="cb1512"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1512-1"><a href="logistic-regression.html#cb1512-1" aria-hidden="true" tabindex="-1"></a><span class="fu">library</span>(boot)</span>
-<span id="cb1512-2"><a href="logistic-regression.html#cb1512-2" aria-hidden="true" tabindex="-1"></a><span class="fu">set.seed</span>(<span class="dv">1</span>)</span>
-<span id="cb1512-3"><a href="logistic-regression.html#cb1512-3" aria-hidden="true" tabindex="-1"></a><span class="fu">cv.glm</span>(spam_trn, fit_caps, <span class="at">K =</span> <span class="dv">5</span>)<span class="sc">$</span>delta[<span class="dv">1</span>]</span></code></pre></div>
+<div class="sourceCode" id="cb1512"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1512-1"><a href="logistic-regression.html#cb1512-1" tabindex="-1"></a><span class="fu">library</span>(boot)</span>
+<span id="cb1512-2"><a href="logistic-regression.html#cb1512-2" tabindex="-1"></a><span class="fu">set.seed</span>(<span class="dv">1</span>)</span>
+<span id="cb1512-3"><a href="logistic-regression.html#cb1512-3" tabindex="-1"></a><span class="fu">cv.glm</span>(spam_trn, fit_caps, <span class="at">K =</span> <span class="dv">5</span>)<span class="sc">$</span>delta[<span class="dv">1</span>]</span></code></pre></div>
 <pre><code>## [1] 0.2166961</code></pre>
-<div class="sourceCode" id="cb1514"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1514-1"><a href="logistic-regression.html#cb1514-1" aria-hidden="true" tabindex="-1"></a><span class="fu">cv.glm</span>(spam_trn, fit_selected, <span class="at">K =</span> <span class="dv">5</span>)<span class="sc">$</span>delta[<span class="dv">1</span>]</span></code></pre></div>
+<div class="sourceCode" id="cb1514"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1514-1"><a href="logistic-regression.html#cb1514-1" tabindex="-1"></a><span class="fu">cv.glm</span>(spam_trn, fit_selected, <span class="at">K =</span> <span class="dv">5</span>)<span class="sc">$</span>delta[<span class="dv">1</span>]</span></code></pre></div>
 <pre><code>## [1] 0.1587043</code></pre>
-<div class="sourceCode" id="cb1516"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1516-1"><a href="logistic-regression.html#cb1516-1" aria-hidden="true" tabindex="-1"></a><span class="fu">cv.glm</span>(spam_trn, fit_additive, <span class="at">K =</span> <span class="dv">5</span>)<span class="sc">$</span>delta[<span class="dv">1</span>]</span></code></pre></div>
+<div class="sourceCode" id="cb1516"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1516-1"><a href="logistic-regression.html#cb1516-1" tabindex="-1"></a><span class="fu">cv.glm</span>(spam_trn, fit_additive, <span class="at">K =</span> <span class="dv">5</span>)<span class="sc">$</span>delta[<span class="dv">1</span>]</span></code></pre></div>
 <pre><code>## [1] 0.08684467</code></pre>
-<div class="sourceCode" id="cb1518"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1518-1"><a href="logistic-regression.html#cb1518-1" aria-hidden="true" tabindex="-1"></a><span class="fu">cv.glm</span>(spam_trn, fit_over, <span class="at">K =</span> <span class="dv">5</span>)<span class="sc">$</span>delta[<span class="dv">1</span>]</span></code></pre></div>
-<pre><code>## [1] 0.146</code></pre>
+<div class="sourceCode" id="cb1518"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1518-1"><a href="logistic-regression.html#cb1518-1" tabindex="-1"></a><span class="fu">cv.glm</span>(spam_trn, fit_over, <span class="at">K =</span> <span class="dv">5</span>)<span class="sc">$</span>delta[<span class="dv">1</span>]</span></code></pre></div>
+<pre><code>## [1] 0.153</code></pre>
 <p>Note that we’re suppressing warnings again here. (Now there would be a lot more, since were fitting a total of 20 models.)</p>
 <p>Based on these results, <code>fit_caps</code> and <code>fit_selected</code> are underfitting relative to <code>fit_additive</code>. Similarly, <code>fit_over</code> is overfitting relative to <code>fit_additive</code>. Thus, based on these results, we prefer the classifier created based on the logistic regression fit and stored in <code>fit_additive</code>.</p>
 <p>Going forward, to evaluate and report on the efficacy of this classifier, we’ll use the test dataset. We’re going to take the position that the test data set should <strong>never</strong> be used in training, which is why we used cross-validation within the training dataset to select a model. Even though cross-validation uses hold-out sets to generate metrics, at some point all of the data is used for training.</p>
 <p>To quickly summarize how well this classifier works, we’ll create a confusion matrix.</p>
-<div class="figure">
-<img src="images/confusion.png" alt="" />
-<p class="caption">Confusion Matrix</p>
+<div class="float">
+<img src="images/confusion.png" alt="Confusion Matrix" />
+<div class="figcaption">Confusion Matrix</div>
 </div>
 <p>It further breaks down the classification errors into false positives and false negatives.</p>
-<div class="sourceCode" id="cb1520"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1520-1"><a href="logistic-regression.html#cb1520-1" aria-hidden="true" tabindex="-1"></a>make_conf_mat <span class="ot">=</span> <span class="cf">function</span>(predicted, actual) {</span>
-<span id="cb1520-2"><a href="logistic-regression.html#cb1520-2" aria-hidden="true" tabindex="-1"></a>  <span class="fu">table</span>(<span class="at">predicted =</span> predicted, <span class="at">actual =</span> actual)</span>
-<span id="cb1520-3"><a href="logistic-regression.html#cb1520-3" aria-hidden="true" tabindex="-1"></a>}</span></code></pre></div>
+<div class="sourceCode" id="cb1520"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1520-1"><a href="logistic-regression.html#cb1520-1" tabindex="-1"></a>make_conf_mat <span class="ot">=</span> <span class="cf">function</span>(predicted, actual) {</span>
+<span id="cb1520-2"><a href="logistic-regression.html#cb1520-2" tabindex="-1"></a>  <span class="fu">table</span>(<span class="at">predicted =</span> predicted, <span class="at">actual =</span> actual)</span>
+<span id="cb1520-3"><a href="logistic-regression.html#cb1520-3" tabindex="-1"></a>}</span></code></pre></div>
 <p>Let’s explicitly store the predicted values of our classifier on the test dataset.</p>
-<div class="sourceCode" id="cb1521"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1521-1"><a href="logistic-regression.html#cb1521-1" aria-hidden="true" tabindex="-1"></a>spam_tst_pred <span class="ot">=</span> <span class="fu">ifelse</span>(<span class="fu">predict</span>(fit_additive, spam_tst) <span class="sc">&gt;</span> <span class="dv">0</span>, </span>
-<span id="cb1521-2"><a href="logistic-regression.html#cb1521-2" aria-hidden="true" tabindex="-1"></a>                       <span class="st">&quot;spam&quot;</span>, </span>
-<span id="cb1521-3"><a href="logistic-regression.html#cb1521-3" aria-hidden="true" tabindex="-1"></a>                       <span class="st">&quot;nonspam&quot;</span>)</span>
-<span id="cb1521-4"><a href="logistic-regression.html#cb1521-4" aria-hidden="true" tabindex="-1"></a>spam_tst_pred <span class="ot">=</span> <span class="fu">ifelse</span>(<span class="fu">predict</span>(fit_additive, spam_tst, <span class="at">type =</span> <span class="st">&quot;response&quot;</span>) <span class="sc">&gt;</span> <span class="fl">0.5</span>, </span>
-<span id="cb1521-5"><a href="logistic-regression.html#cb1521-5" aria-hidden="true" tabindex="-1"></a>                       <span class="st">&quot;spam&quot;</span>, </span>
-<span id="cb1521-6"><a href="logistic-regression.html#cb1521-6" aria-hidden="true" tabindex="-1"></a>                       <span class="st">&quot;nonspam&quot;</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1521"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1521-1"><a href="logistic-regression.html#cb1521-1" tabindex="-1"></a>spam_tst_pred <span class="ot">=</span> <span class="fu">ifelse</span>(<span class="fu">predict</span>(fit_additive, spam_tst) <span class="sc">&gt;</span> <span class="dv">0</span>, </span>
+<span id="cb1521-2"><a href="logistic-regression.html#cb1521-2" tabindex="-1"></a>                       <span class="st">&quot;spam&quot;</span>, </span>
+<span id="cb1521-3"><a href="logistic-regression.html#cb1521-3" tabindex="-1"></a>                       <span class="st">&quot;nonspam&quot;</span>)</span>
+<span id="cb1521-4"><a href="logistic-regression.html#cb1521-4" tabindex="-1"></a>spam_tst_pred <span class="ot">=</span> <span class="fu">ifelse</span>(<span class="fu">predict</span>(fit_additive, spam_tst, <span class="at">type =</span> <span class="st">&quot;response&quot;</span>) <span class="sc">&gt;</span> <span class="fl">0.5</span>, </span>
+<span id="cb1521-5"><a href="logistic-regression.html#cb1521-5" tabindex="-1"></a>                       <span class="st">&quot;spam&quot;</span>, </span>
+<span id="cb1521-6"><a href="logistic-regression.html#cb1521-6" tabindex="-1"></a>                       <span class="st">&quot;nonspam&quot;</span>)</span></code></pre></div>
 <p>The previous two lines of code produce the same output, that is the same predictions, since</p>
 <p><span class="math display">\[
 \eta({\bf x}) = 0 \iff p({\bf x}) = 0.5
 \]</span>
 Now we’ll use these predictions to create a confusion matrix.</p>
-<div class="sourceCode" id="cb1522"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1522-1"><a href="logistic-regression.html#cb1522-1" aria-hidden="true" tabindex="-1"></a>(<span class="at">conf_mat_50 =</span> <span class="fu">make_conf_mat</span>(<span class="at">predicted =</span> spam_tst_pred, <span class="at">actual =</span> spam_tst<span class="sc">$</span>type))</span></code></pre></div>
+<div class="sourceCode" id="cb1522"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1522-1"><a href="logistic-regression.html#cb1522-1" tabindex="-1"></a>(<span class="at">conf_mat_50 =</span> <span class="fu">make_conf_mat</span>(<span class="at">predicted =</span> spam_tst_pred, <span class="at">actual =</span> spam_tst<span class="sc">$</span>type))</span></code></pre></div>
 <pre><code>##          actual
 ## predicted nonspam spam
 ##   nonspam    2057  157
@@ -1454,16 +1454,16 @@ <h3><span class="header-section-number">17.4.2</span> Evaluating Classifiers<a h
 <p><span class="math display">\[
 \text{Prev} = \frac{\text{P}}{\text{Total Obs}}= \frac{\text{TP + FN}}{\text{Total Obs}}
 \]</span></p>
-<div class="sourceCode" id="cb1524"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1524-1"><a href="logistic-regression.html#cb1524-1" aria-hidden="true" tabindex="-1"></a><span class="fu">table</span>(spam_tst<span class="sc">$</span>type) <span class="sc">/</span> <span class="fu">nrow</span>(spam_tst)</span></code></pre></div>
+<div class="sourceCode" id="cb1524"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1524-1"><a href="logistic-regression.html#cb1524-1" tabindex="-1"></a><span class="fu">table</span>(spam_tst<span class="sc">$</span>type) <span class="sc">/</span> <span class="fu">nrow</span>(spam_tst)</span></code></pre></div>
 <pre><code>## 
 ##   nonspam      spam 
 ## 0.6064982 0.3935018</code></pre>
 <p>First, note that to be a reasonable classifier, it needs to outperform the obvious classifier of simply classifying all observations to the majority class. In this case, classifying everything as non-spam for a test misclassification rate of 0.3935018</p>
 <p>Next, we can see that using the classifier created from <code>fit_additive</code>, only a total of <span class="math inline">\(127 + 157 = 284\)</span> from the total of 3601 emails in the test set are misclassified. Overall, the accuracy in the test set it</p>
-<div class="sourceCode" id="cb1526"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1526-1"><a href="logistic-regression.html#cb1526-1" aria-hidden="true" tabindex="-1"></a><span class="fu">mean</span>(spam_tst_pred <span class="sc">==</span> spam_tst<span class="sc">$</span>type)</span></code></pre></div>
+<div class="sourceCode" id="cb1526"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1526-1"><a href="logistic-regression.html#cb1526-1" tabindex="-1"></a><span class="fu">mean</span>(spam_tst_pred <span class="sc">==</span> spam_tst<span class="sc">$</span>type)</span></code></pre></div>
 <pre><code>## [1] 0.921133</code></pre>
 <p>In other words, the test misclassification is</p>
-<div class="sourceCode" id="cb1528"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1528-1"><a href="logistic-regression.html#cb1528-1" aria-hidden="true" tabindex="-1"></a><span class="fu">mean</span>(spam_tst_pred <span class="sc">!=</span> spam_tst<span class="sc">$</span>type)</span></code></pre></div>
+<div class="sourceCode" id="cb1528"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1528-1"><a href="logistic-regression.html#cb1528-1" tabindex="-1"></a><span class="fu">mean</span>(spam_tst_pred <span class="sc">!=</span> spam_tst<span class="sc">$</span>type)</span></code></pre></div>
 <pre><code>## [1] 0.07886698</code></pre>
 <p>This seems like a decent classifier…</p>
 <p>However, are all errors created equal? In this case, absolutely not. The 127 non-spam emails that were marked as spam (false positives) are a problem. We can’t allow important information, say, a job offer, to miss our inbox and get sent to the spam folder. On the other hand, the 157 spam email that would make it to an inbox (false negatives) are easily dealt with, just delete them.</p>
@@ -1473,20 +1473,20 @@ <h3><span class="header-section-number">17.4.2</span> Evaluating Classifiers<a h
 \text{Sens} = \text{True Positive Rate} = \frac{\text{TP}}{\text{P}} = \frac{\text{TP}}{\text{TP + FN}}
 \]</span></p>
 <p>Here we have an <code>R</code> function to calculate the sensitivity based on the confusion matrix. Note that this function is good for illustrative purposes, but is easily broken. (Think about what happens if there are no “positives” predicted.)</p>
-<div class="sourceCode" id="cb1530"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1530-1"><a href="logistic-regression.html#cb1530-1" aria-hidden="true" tabindex="-1"></a>get_sens <span class="ot">=</span> <span class="cf">function</span>(conf_mat) {</span>
-<span id="cb1530-2"><a href="logistic-regression.html#cb1530-2" aria-hidden="true" tabindex="-1"></a>  conf_mat[<span class="dv">2</span>, <span class="dv">2</span>] <span class="sc">/</span> <span class="fu">sum</span>(conf_mat[, <span class="dv">2</span>])</span>
-<span id="cb1530-3"><a href="logistic-regression.html#cb1530-3" aria-hidden="true" tabindex="-1"></a>}</span></code></pre></div>
+<div class="sourceCode" id="cb1530"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1530-1"><a href="logistic-regression.html#cb1530-1" tabindex="-1"></a>get_sens <span class="ot">=</span> <span class="cf">function</span>(conf_mat) {</span>
+<span id="cb1530-2"><a href="logistic-regression.html#cb1530-2" tabindex="-1"></a>  conf_mat[<span class="dv">2</span>, <span class="dv">2</span>] <span class="sc">/</span> <span class="fu">sum</span>(conf_mat[, <span class="dv">2</span>])</span>
+<span id="cb1530-3"><a href="logistic-regression.html#cb1530-3" tabindex="-1"></a>}</span></code></pre></div>
 <p><strong>Specificity</strong> is essentially the true negative rate. So when specificity is high, the number of false positives is low.</p>
 <p><span class="math display">\[
 \text{Spec} = \text{True Negative Rate} = \frac{\text{TN}}{\text{N}} = \frac{\text{TN}}{\text{TN + FP}}
 \]</span></p>
-<div class="sourceCode" id="cb1531"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1531-1"><a href="logistic-regression.html#cb1531-1" aria-hidden="true" tabindex="-1"></a>get_spec <span class="ot">=</span>  <span class="cf">function</span>(conf_mat) {</span>
-<span id="cb1531-2"><a href="logistic-regression.html#cb1531-2" aria-hidden="true" tabindex="-1"></a>  conf_mat[<span class="dv">1</span>, <span class="dv">1</span>] <span class="sc">/</span> <span class="fu">sum</span>(conf_mat[, <span class="dv">1</span>])</span>
-<span id="cb1531-3"><a href="logistic-regression.html#cb1531-3" aria-hidden="true" tabindex="-1"></a>}</span></code></pre></div>
+<div class="sourceCode" id="cb1531"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1531-1"><a href="logistic-regression.html#cb1531-1" tabindex="-1"></a>get_spec <span class="ot">=</span>  <span class="cf">function</span>(conf_mat) {</span>
+<span id="cb1531-2"><a href="logistic-regression.html#cb1531-2" tabindex="-1"></a>  conf_mat[<span class="dv">1</span>, <span class="dv">1</span>] <span class="sc">/</span> <span class="fu">sum</span>(conf_mat[, <span class="dv">1</span>])</span>
+<span id="cb1531-3"><a href="logistic-regression.html#cb1531-3" tabindex="-1"></a>}</span></code></pre></div>
 <p>We calculate both based on the confusion matrix we had created for our classifier.</p>
-<div class="sourceCode" id="cb1532"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1532-1"><a href="logistic-regression.html#cb1532-1" aria-hidden="true" tabindex="-1"></a><span class="fu">get_sens</span>(conf_mat_50)</span></code></pre></div>
+<div class="sourceCode" id="cb1532"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1532-1"><a href="logistic-regression.html#cb1532-1" tabindex="-1"></a><span class="fu">get_sens</span>(conf_mat_50)</span></code></pre></div>
 <pre><code>## [1] 0.8892025</code></pre>
-<div class="sourceCode" id="cb1534"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1534-1"><a href="logistic-regression.html#cb1534-1" aria-hidden="true" tabindex="-1"></a><span class="fu">get_spec</span>(conf_mat_50)</span></code></pre></div>
+<div class="sourceCode" id="cb1534"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1534-1"><a href="logistic-regression.html#cb1534-1" tabindex="-1"></a><span class="fu">get_spec</span>(conf_mat_50)</span></code></pre></div>
 <pre><code>## [1] 0.9418498</code></pre>
 <p>Recall that we had created this classifier using a probability of <span class="math inline">\(0.5\)</span> as a “cutoff” for how observations should be classified. Now we’ll modify this cutoff. We’ll see that by modifying the cutoff, <span class="math inline">\(c\)</span>, we can improve sensitivity or specificity at the expense of the overall accuracy (misclassification rate).</p>
 <p><span class="math display">\[
@@ -1498,34 +1498,34 @@ <h3><span class="header-section-number">17.4.2</span> Evaluating Classifiers<a h
 \]</span></p>
 <p>Additionally, if we change the cutoff to improve sensitivity, we’ll decrease specificity, and vice versa.</p>
 <p>First let’s see what happens when we lower the cutoff from <span class="math inline">\(0.5\)</span> to <span class="math inline">\(0.1\)</span> to create a new classifier, and thus new predictions.</p>
-<div class="sourceCode" id="cb1536"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1536-1"><a href="logistic-regression.html#cb1536-1" aria-hidden="true" tabindex="-1"></a>spam_tst_pred_10 <span class="ot">=</span> <span class="fu">ifelse</span>(<span class="fu">predict</span>(fit_additive, spam_tst, <span class="at">type =</span> <span class="st">&quot;response&quot;</span>) <span class="sc">&gt;</span> <span class="fl">0.1</span>, </span>
-<span id="cb1536-2"><a href="logistic-regression.html#cb1536-2" aria-hidden="true" tabindex="-1"></a>                          <span class="st">&quot;spam&quot;</span>, </span>
-<span id="cb1536-3"><a href="logistic-regression.html#cb1536-3" aria-hidden="true" tabindex="-1"></a>                          <span class="st">&quot;nonspam&quot;</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1536"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1536-1"><a href="logistic-regression.html#cb1536-1" tabindex="-1"></a>spam_tst_pred_10 <span class="ot">=</span> <span class="fu">ifelse</span>(<span class="fu">predict</span>(fit_additive, spam_tst, <span class="at">type =</span> <span class="st">&quot;response&quot;</span>) <span class="sc">&gt;</span> <span class="fl">0.1</span>, </span>
+<span id="cb1536-2"><a href="logistic-regression.html#cb1536-2" tabindex="-1"></a>                          <span class="st">&quot;spam&quot;</span>, </span>
+<span id="cb1536-3"><a href="logistic-regression.html#cb1536-3" tabindex="-1"></a>                          <span class="st">&quot;nonspam&quot;</span>)</span></code></pre></div>
 <p>This is essentially <em>decreasing</em> the threshold for an email to be labeled as spam, so far <em>more</em> emails will be labeled as spam. We see that in the following confusion matrix.</p>
-<div class="sourceCode" id="cb1537"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1537-1"><a href="logistic-regression.html#cb1537-1" aria-hidden="true" tabindex="-1"></a>(<span class="at">conf_mat_10 =</span> <span class="fu">make_conf_mat</span>(<span class="at">predicted =</span> spam_tst_pred_10, <span class="at">actual =</span> spam_tst<span class="sc">$</span>type))</span></code></pre></div>
+<div class="sourceCode" id="cb1537"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1537-1"><a href="logistic-regression.html#cb1537-1" tabindex="-1"></a>(<span class="at">conf_mat_10 =</span> <span class="fu">make_conf_mat</span>(<span class="at">predicted =</span> spam_tst_pred_10, <span class="at">actual =</span> spam_tst<span class="sc">$</span>type))</span></code></pre></div>
 <pre><code>##          actual
 ## predicted nonspam spam
 ##   nonspam    1583   29
 ##   spam        601 1388</code></pre>
 <p>Unfortunately, while this does greatly reduce false negatives, false positives have almost quadrupled. We see this reflected in the sensitivity and specificity.</p>
-<div class="sourceCode" id="cb1539"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1539-1"><a href="logistic-regression.html#cb1539-1" aria-hidden="true" tabindex="-1"></a><span class="fu">get_sens</span>(conf_mat_10)</span></code></pre></div>
+<div class="sourceCode" id="cb1539"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1539-1"><a href="logistic-regression.html#cb1539-1" tabindex="-1"></a><span class="fu">get_sens</span>(conf_mat_10)</span></code></pre></div>
 <pre><code>## [1] 0.9795342</code></pre>
-<div class="sourceCode" id="cb1541"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1541-1"><a href="logistic-regression.html#cb1541-1" aria-hidden="true" tabindex="-1"></a><span class="fu">get_spec</span>(conf_mat_10)</span></code></pre></div>
+<div class="sourceCode" id="cb1541"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1541-1"><a href="logistic-regression.html#cb1541-1" tabindex="-1"></a><span class="fu">get_spec</span>(conf_mat_10)</span></code></pre></div>
 <pre><code>## [1] 0.7248168</code></pre>
 <p>This classifier, using <span class="math inline">\(0.1\)</span> instead of <span class="math inline">\(0.5\)</span> has a higher sensitivity, but a much lower specificity. Clearly, we should have moved the cutoff in the other direction. Let’s try <span class="math inline">\(0.9\)</span>.</p>
-<div class="sourceCode" id="cb1543"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1543-1"><a href="logistic-regression.html#cb1543-1" aria-hidden="true" tabindex="-1"></a>spam_tst_pred_90 <span class="ot">=</span> <span class="fu">ifelse</span>(<span class="fu">predict</span>(fit_additive, spam_tst, <span class="at">type =</span> <span class="st">&quot;response&quot;</span>) <span class="sc">&gt;</span> <span class="fl">0.9</span>, </span>
-<span id="cb1543-2"><a href="logistic-regression.html#cb1543-2" aria-hidden="true" tabindex="-1"></a>                          <span class="st">&quot;spam&quot;</span>, </span>
-<span id="cb1543-3"><a href="logistic-regression.html#cb1543-3" aria-hidden="true" tabindex="-1"></a>                          <span class="st">&quot;nonspam&quot;</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1543"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1543-1"><a href="logistic-regression.html#cb1543-1" tabindex="-1"></a>spam_tst_pred_90 <span class="ot">=</span> <span class="fu">ifelse</span>(<span class="fu">predict</span>(fit_additive, spam_tst, <span class="at">type =</span> <span class="st">&quot;response&quot;</span>) <span class="sc">&gt;</span> <span class="fl">0.9</span>, </span>
+<span id="cb1543-2"><a href="logistic-regression.html#cb1543-2" tabindex="-1"></a>                          <span class="st">&quot;spam&quot;</span>, </span>
+<span id="cb1543-3"><a href="logistic-regression.html#cb1543-3" tabindex="-1"></a>                          <span class="st">&quot;nonspam&quot;</span>)</span></code></pre></div>
 <p>This is essentially <em>increasing</em> the threshold for an email to be labeled as spam, so far <em>fewer</em> emails will be labeled as spam. Again, we see that in the following confusion matrix.</p>
-<div class="sourceCode" id="cb1544"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1544-1"><a href="logistic-regression.html#cb1544-1" aria-hidden="true" tabindex="-1"></a>(<span class="at">conf_mat_90 =</span> <span class="fu">make_conf_mat</span>(<span class="at">predicted =</span> spam_tst_pred_90, <span class="at">actual =</span> spam_tst<span class="sc">$</span>type))</span></code></pre></div>
+<div class="sourceCode" id="cb1544"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1544-1"><a href="logistic-regression.html#cb1544-1" tabindex="-1"></a>(<span class="at">conf_mat_90 =</span> <span class="fu">make_conf_mat</span>(<span class="at">predicted =</span> spam_tst_pred_90, <span class="at">actual =</span> spam_tst<span class="sc">$</span>type))</span></code></pre></div>
 <pre><code>##          actual
 ## predicted nonspam spam
 ##   nonspam    2136  537
 ##   spam         48  880</code></pre>
 <p>This is the result we’re looking for. We have far fewer false positives. While sensitivity is greatly reduced, specificity has gone up.</p>
-<div class="sourceCode" id="cb1546"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1546-1"><a href="logistic-regression.html#cb1546-1" aria-hidden="true" tabindex="-1"></a><span class="fu">get_sens</span>(conf_mat_90)</span></code></pre></div>
+<div class="sourceCode" id="cb1546"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1546-1"><a href="logistic-regression.html#cb1546-1" tabindex="-1"></a><span class="fu">get_sens</span>(conf_mat_90)</span></code></pre></div>
 <pre><code>## [1] 0.6210303</code></pre>
-<div class="sourceCode" id="cb1548"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1548-1"><a href="logistic-regression.html#cb1548-1" aria-hidden="true" tabindex="-1"></a><span class="fu">get_spec</span>(conf_mat_90)</span></code></pre></div>
+<div class="sourceCode" id="cb1548"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1548-1"><a href="logistic-regression.html#cb1548-1" tabindex="-1"></a><span class="fu">get_spec</span>(conf_mat_90)</span></code></pre></div>
 <pre><code>## [1] 0.978022</code></pre>
 <p>While this is far fewer false positives, is it acceptable though? Still probably not. Also, don’t forget, this would actually be a terrible spam detector today since this is based on data from a very different era of the internet, for a very specific set of people. Spam has changed a lot since the 90s! (Ironically, machine learning is probably partially to blame.)</p>
 <p>This chapter has provided a rather quick introduction to classification, and thus, machine learning. For a more complete coverage of machine learning, <a href="http://www-bcf.usc.edu/~gareth/ISL/" target="_blank">An Introduction to Statistical Learning</a> is a highly recommended resource. Additionally, <a href="https://daviddalpiaz.github.io/r4sl/" target="_blank"><code>R</code> for Statistical Learning</a> has been written as a supplement which provides additional detail on how to perform these methods using <code>R</code>. The <a href="https://daviddalpiaz.github.io/r4sl/classification-overview.html" target="_blank">classification</a> and <a href="https://daviddalpiaz.github.io/r4sl/logistic-regression.html" target="_blank">logistic regression</a> chapters might be useful.</p>
@@ -1538,7 +1538,7 @@ <h2><span class="header-section-number">17.5</span> <code>R</code> Markdown<a hr
 <ul>
 <li><a href="logistic.Rmd" target="_blank"><code>logistic.Rmd</code></a></li>
 </ul>
-<p>The file was created using <code>R</code> version <code>4.3.2</code>.</p>
+<p>The file was created using <code>R</code> version <code>4.4.1</code>.</p>
 
 </div>
 </div>
diff --git a/logistic.md b/logistic.md
index 144cc43..3723d95 100644
--- a/logistic.md
+++ b/logistic.md
@@ -222,7 +222,7 @@ When this happens, the model is still "fit," but there are consequences, namely,
 ### Simulation Examples
 
 
-```r
+``` r
 sim_logistic_data = function(sample_size = 25, beta_0 = -2, beta_1 = 3) {
   x = rnorm(n = sample_size)
   eta = beta_0 + beta_1 * x
@@ -260,7 +260,7 @@ p_i &= p({\bf x_i}) = \frac{1}{1 + e^{-\eta({\bf x_i})}} \\
 $$
 
 
-```r
+``` r
 set.seed(1)
 example_data = sim_logistic_data()
 head(example_data)
@@ -279,7 +279,7 @@ head(example_data)
 After simulating a dataset, we'll then fit both ordinary linear regression and logistic regression. Notice that currently the responses variable `y` is a numeric variable that only takes values `0` and `1`. Later we'll see that we can also fit logistic regression when the response is a factor variable with only two levels. (Generally, having a factor response is preferred, but having a dummy response allows us to make the comparison to using ordinary linear regression.)
 
 
-```r
+``` r
 # ordinary linear regression
 fit_lm  = lm(y ~ x, data = example_data)
 # logistic regression
@@ -294,7 +294,7 @@ Notice that the syntax is extremely similar. What's changed?
 In a lot of ways, `lm()` is just a more specific version of `glm()`. For example
 
 
-```r
+``` r
 glm(y ~ x, data = example_data)
 ```
 
@@ -303,7 +303,7 @@ would actually fit the ordinary linear regression that we have seen in the past.
 The `family` argument to `glm()` actually specifies both the distribution and the link function. If not made explicit, the link function is chosen to be the **canonical link function**, which is essentially the most mathematical convenient link function. See `?glm` and `?family` for details. For example, the following code explicitly specifies the link function which was previously used by default.
 
 
-```r
+``` r
 # more detailed call to glm for logistic regression
 fit_glm = glm(y ~ x, data = example_data, family = binomial(link = "logit"))
 ```
@@ -318,7 +318,7 @@ Making predictions with an object of type  `glm` is slightly different than maki
 That is, `type = "link"` will get you the log odds, while `type = "response"` will return the estimated mean, in this case, $P[Y = 1 \mid {\bf X} = {\bf x}]$ for each observation.
 
 
-```r
+``` r
 predict(fit_glm, data.frame(x = 1.2)) # type = "link" as default
 ```
 
@@ -327,7 +327,7 @@ predict(fit_glm, data.frame(x = 1.2)) # type = "link" as default
 ## 2.076166
 ```
 
-```r
+``` r
 predict(fit_glm, data.frame(x = 1.2), type = "link")
 ```
 
@@ -336,7 +336,7 @@ predict(fit_glm, data.frame(x = 1.2), type = "link")
 ## 2.076166
 ```
 
-```r
+``` r
 predict(fit_glm, data.frame(x = 1.2), type = "response")
 ```
 
@@ -346,7 +346,7 @@ predict(fit_glm, data.frame(x = 1.2), type = "response")
 ```
 
 
-```r
+``` r
 plot(y ~ x, data = example_data, 
      pch = 20, ylab = "Estimated Probability", 
      main = "Ordinary vs Logistic Regression")
@@ -373,7 +373,7 @@ We immediately see why ordinary linear regression is not a good idea. While it i
 Enter logistic regression. Since the output of the inverse logit function is restricted to be between 0 and 1, our estimates make much more sense as probabilities. Let's look at our estimated coefficients. (With a lot of rounding, for simplicity.)
 
 
-```r
+``` r
 round(coef(fit_glm), 1)
 ```
 
@@ -423,7 +423,7 @@ $$
 In this model, as $x$ increases, the log odds decrease.
 
 
-```r
+``` r
 set.seed(1)
 example_data = sim_logistic_data(sample_size = 50, beta_0 = 1, beta_1 = -4)
 ```
@@ -431,12 +431,12 @@ example_data = sim_logistic_data(sample_size = 50, beta_0 = 1, beta_1 = -4)
 We again simulate some observations form this model, then fit logistic regression.
 
 
-```r
+``` r
 fit_glm = glm(y ~ x, data = example_data, family = binomial)
 ```
 
 
-```r
+``` r
 plot(y ~ x, data = example_data, 
      pch = 20, ylab = "Estimated Probability", 
      main = "Logistic Regression, Decreasing Probability")
@@ -457,7 +457,7 @@ We see that this time, as $x$ increases, $\hat{p}({\bf x})$ decreases.
 Now let's look at an example where the estimated probability doesn't always simply increase or decrease. Much like ordinary linear regression, the linear combination of predictors can contain transformations of predictors (in this case a quadratic term) and interactions.
 
 
-```r
+``` r
 sim_quadratic_logistic_data = function(sample_size = 25) {
   x = rnorm(n = sample_size)
   eta = -1.5 + 0.5 * x + x ^ 2
@@ -482,18 +482,18 @@ p_i &= p({\bf x_i}) = \frac{1}{1 + e^{-\eta({\bf x_i})}} \\
 $$
 
 
-```r
+``` r
 set.seed(42)
 example_data = sim_quadratic_logistic_data(sample_size = 50)
 ```
 
 
-```r
+``` r
 fit_glm = glm(y ~ x + I(x^2), data = example_data, family = binomial)
 ```
 
 
-```r
+``` r
 plot(y ~ x, data = example_data, 
      pch = 20, ylab = "Estimated Probability", 
      main = "Logistic Regression, Quadratic Relationship")
@@ -615,14 +615,14 @@ The Likelihood-Ratio Test is actually a rather general test, however, here we ha
 To illustrate the use of logistic regression, we will use the `SAheart` dataset from the `bestglm` package. 
 
 
-```r
+``` r
 # "leaps" is required for "bestglm"
 if (!require("leaps")) install.packages("leaps")
 if (!require("bestglm")) install.packages("bestglm")
 ```
 
 
-```r
+``` r
 library(leaps)
 library(bestglm)
 data("SAheart")
@@ -658,7 +658,7 @@ $$
 $$
 
 
-```r
+``` r
 chd_mod_ldl = glm(chd ~ ldl, data = SAheart, family = binomial)
 plot(jitter(chd, factor = 0.1) ~ ldl, data = SAheart, pch = 20, 
      ylab = "Probability of CHD", xlab = "Low Density Lipoprotein Cholesterol")
@@ -676,7 +676,7 @@ As before, we plot the data in addition to the estimated probabilities. Note tha
 As we would expect, this plot indicates that as `ldl` increases, so does the probability of `chd`.
 
 
-```r
+``` r
 coef(summary(chd_mod_ldl))
 ```
 
@@ -697,7 +697,7 @@ we use the `summary()` function as we have done so many times before. Like the $
 When fitting logistic regression, we can use the same formula syntax as ordinary linear regression. So, to fit an additive model using all available predictors, we use:
 
 
-```r
+``` r
 chd_mod_additive = glm(chd ~ ., data = SAheart, family = binomial)
 ```
 
@@ -710,7 +710,7 @@ $$
 We could manually calculate the test statistic,
 
 
-```r
+``` r
 -2 * as.numeric(logLik(chd_mod_ldl) - logLik(chd_mod_additive))
 ```
 
@@ -721,7 +721,7 @@ We could manually calculate the test statistic,
 Or we could utilize the `anova()` function. By specifying `test = "LRT"`, `R` will use the likelihood-ratio test to compare the two models.
 
 
-```r
+``` r
 anova(chd_mod_ldl, chd_mod_additive, test = "LRT")
 ```
 
@@ -743,7 +743,7 @@ We see that the test statistic that we had just calculated appears in the output
 While we prefer the additive model compared to the model with only a single predictor, do we actually need all of the predictors in the additive model? To select a subset of predictors, we can use a stepwise procedure as we did with ordinary linear regression. Recall that AIC and BIC were defined in terms of likelihoods. Here we demonstrate using AIC with a backwards selection procedure.
 
 
-```r
+``` r
 chd_mod_selected = step(chd_mod_additive, trace = 0)
 coef(chd_mod_selected)
 ```
@@ -762,7 +762,7 @@ H_0: \beta_{\texttt{sbp}} = \beta_{\texttt{adiposity}} = \beta_{\texttt{obesity}
 $$
 
 
-```r
+``` r
 anova(chd_mod_selected, chd_mod_additive, test = "LRT")
 ```
 
@@ -784,7 +784,7 @@ Here it seems that we would prefer the selected model.
 We can create confidence intervals for the $\beta$ parameters using the `confint()` function as we did with ordinary linear regression.
 
 
-```r
+``` r
 confint(chd_mod_selected, level = 0.99)
 ```
 
@@ -835,7 +835,7 @@ $$
 To demonstrate creating these intervals, we'll consider a new observation.
 
 
-```r
+``` r
 new_obs = data.frame(
   sbp = 148.0,
   tobacco = 5,
@@ -852,7 +852,7 @@ new_obs = data.frame(
 First, we'll use the `predict()` function to obtain $\hat{\eta}({\bf x})$ for this observation.
 
 
-```r
+``` r
 eta_hat = predict(chd_mod_selected, new_obs, se.fit = TRUE, type = "link")
 eta_hat
 ```
@@ -872,7 +872,7 @@ eta_hat
 By setting `se.fit = TRUE`, `R` also computes $\text{SE}[\hat{\eta}({\bf x})]$. Note that we used `type = "link"`, but this is actually a default value. We added it here to stress that the output from `predict()` will be the value of the link function.
 
 
-```r
+``` r
 z_crit = qnorm(0.975)
 round(z_crit, 2)
 ```
@@ -884,7 +884,7 @@ round(z_crit, 2)
 After obtaining the correct critical value, we can easily create a $95\%$ confidence interval for $\eta({\bf x})$.
 
 
-```r
+``` r
 eta_hat$fit + c(-1, 1) * z_crit * eta_hat$se.fit
 ```
 
@@ -895,7 +895,7 @@ eta_hat$fit + c(-1, 1) * z_crit * eta_hat$se.fit
 Now we simply need to apply the correct transformation to make this a confidence interval for $p({\bf x})$, the probability of coronary heart disease for this observation. Note that the `boot` package contains functions `logit()` and `inv.logit()` which are the logit and inverse logit transformations, respectively.
 
 
-```r
+``` r
 boot::inv.logit(eta_hat$fit + c(-1, 1) * z_crit * eta_hat$se.fit)
 ```
 
@@ -914,7 +914,7 @@ Without really thinking about it, we've been using our previous knowledge of `R`
 Let's add an interaction between LDL and family history for the model we selected.
 
 
-```r
+``` r
 chd_mod_interaction = glm(chd ~ alcohol + ldl + famhist + typea + age + ldl:famhist, 
                           data = SAheart, family = binomial)
 summary(chd_mod_interaction)
@@ -954,7 +954,7 @@ Based on the $z$-test seen in the above summary, this interaction is significant
 Let's take the previous model, and now add a polynomial term.
 
 
-```r
+``` r
 chd_mod_int_quad = glm(chd ~ alcohol + ldl + famhist + typea + age + ldl:famhist + I(ldl^2),
                        data = SAheart, family = binomial)
 summary(chd_mod_int_quad)
@@ -1013,7 +1013,7 @@ You have probably noticed that the output from `summary()` is also very similar
 [**Deviance**](https://en.wikipedia.org/wiki/Deviance_(statistics)){target="_blank"} compares the model to a saturated model. (Without repeated observations, a saturated model is a model that fits perfectly, using a parameter for each observation.) Essentially, deviance is a generalized *residual sum of squares* for GLMs. Like RSS, deviance decreases as the model complexity increases.
 
 
-```r
+``` r
 deviance(chd_mod_ldl)
 ```
 
@@ -1021,7 +1021,7 @@ deviance(chd_mod_ldl)
 ## [1] 564.2788
 ```
 
-```r
+``` r
 deviance(chd_mod_selected)
 ```
 
@@ -1029,7 +1029,7 @@ deviance(chd_mod_selected)
 ## [1] 475.6856
 ```
 
-```r
+``` r
 deviance(chd_mod_additive)
 ```
 
@@ -1096,7 +1096,7 @@ Logistic regression is just one of many ways that these probabilities could be e
 To illustrate the use of logistic regression as a classifier, we will use the `spam` dataset from the `kernlab` package. 
 
 
-```r
+``` r
 # install.packages("kernlab")
 library(kernlab)
 data("spam")
@@ -1140,7 +1140,7 @@ This dataset, created in the late 1990s at Hewlett-Packard Labs, contains 4601 e
 The response variable, `type`, is a **factor** with levels that label each email as `spam` or `nonspam`. When fitting models, `nonspam` will be the reference level, $Y = 0$, as it comes first alphabetically.
 
 
-```r
+``` r
 is.factor(spam$type)
 ```
 
@@ -1148,7 +1148,7 @@ is.factor(spam$type)
 ## [1] TRUE
 ```
 
-```r
+``` r
 levels(spam$type)
 ```
 
@@ -1161,7 +1161,7 @@ Many of the predictors (often called features in machine learning) are engineere
 To get started, we'll first test-train split the data.
 
 
-```r
+``` r
 set.seed(42)
 # spam_idx = sample(nrow(spam), round(nrow(spam) / 2))
 spam_idx = sample(nrow(spam), 1000)
@@ -1172,7 +1172,7 @@ spam_tst = spam[-spam_idx, ]
 We've used a somewhat small train set relative to the total size of the dataset. In practice it should likely be larger, but this is simply to keep training time low for illustration and rendering of this document.
 
 
-```r
+``` r
 fit_caps = glm(type ~ capitalTotal, 
                data = spam_trn, family = binomial)
 fit_selected = glm(type ~ edu + money + capitalTotal + charDollar, 
@@ -1193,7 +1193,7 @@ We'll fit four logistic regressions, each more complex than the previous. Note t
 Note that, when we receive this warning, we should be highly suspicious of the parameter estimates.
 
 
-```r
+``` r
 coef(fit_selected)
 ```
 
@@ -1232,7 +1232,7 @@ $$
 When using this metric on the training data, it will have the same issues as RSS did for ordinary linear regression, that is, it will only go down.
 
 
-```r
+``` r
 # training misclassification rate
 mean(ifelse(predict(fit_caps) > 0, "spam", "nonspam") != spam_trn$type)
 ```
@@ -1241,7 +1241,7 @@ mean(ifelse(predict(fit_caps) > 0, "spam", "nonspam") != spam_trn$type)
 ## [1] 0.339
 ```
 
-```r
+``` r
 mean(ifelse(predict(fit_selected) > 0, "spam", "nonspam") != spam_trn$type)
 ```
 
@@ -1249,7 +1249,7 @@ mean(ifelse(predict(fit_selected) > 0, "spam", "nonspam") != spam_trn$type)
 ## [1] 0.224
 ```
 
-```r
+``` r
 mean(ifelse(predict(fit_additive) > 0, "spam", "nonspam") != spam_trn$type)
 ```
 
@@ -1257,7 +1257,7 @@ mean(ifelse(predict(fit_additive) > 0, "spam", "nonspam") != spam_trn$type)
 ## [1] 0.066
 ```
 
-```r
+``` r
 mean(ifelse(predict(fit_over) > 0, "spam", "nonspam") != spam_trn$type)
 ```
 
@@ -1280,7 +1280,7 @@ Essentially we'll repeat the following process 5 times:
 The 5-fold cross-validated misclassification rate will be the average of these misclassification rates. By only needing to refit the model 5 times, instead of $n$ times, we will save a lot of computation time.
 
 
-```r
+``` r
 library(boot)
 set.seed(1)
 cv.glm(spam_trn, fit_caps, K = 5)$delta[1]
@@ -1290,7 +1290,7 @@ cv.glm(spam_trn, fit_caps, K = 5)$delta[1]
 ## [1] 0.2166961
 ```
 
-```r
+``` r
 cv.glm(spam_trn, fit_selected, K = 5)$delta[1]
 ```
 
@@ -1298,7 +1298,7 @@ cv.glm(spam_trn, fit_selected, K = 5)$delta[1]
 ## [1] 0.1587043
 ```
 
-```r
+``` r
 cv.glm(spam_trn, fit_additive, K = 5)$delta[1]
 ```
 
@@ -1306,12 +1306,12 @@ cv.glm(spam_trn, fit_additive, K = 5)$delta[1]
 ## [1] 0.08684467
 ```
 
-```r
+``` r
 cv.glm(spam_trn, fit_over, K = 5)$delta[1]
 ```
 
 ```
-## [1] 0.146
+## [1] 0.153
 ```
  
 Note that we're suppressing warnings again here. (Now there would be a lot more, since were fitting a total of 20 models.)
@@ -1327,7 +1327,7 @@ To quickly summarize how well this classifier works, we'll create a confusion ma
 It further breaks down the classification errors into false positives and false negatives.
 
 
-```r
+``` r
 make_conf_mat = function(predicted, actual) {
   table(predicted = predicted, actual = actual)
 }
@@ -1336,7 +1336,7 @@ make_conf_mat = function(predicted, actual) {
 Let's explicitly store the predicted values of our classifier on the test dataset.
 
 
-```r
+``` r
 spam_tst_pred = ifelse(predict(fit_additive, spam_tst) > 0, 
                        "spam", 
                        "nonspam")
@@ -1353,7 +1353,7 @@ $$
 Now we'll use these predictions to create a confusion matrix.
 
 
-```r
+``` r
 (conf_mat_50 = make_conf_mat(predicted = spam_tst_pred, actual = spam_tst$type))
 ```
 
@@ -1369,7 +1369,7 @@ $$
 $$
 
 
-```r
+``` r
 table(spam_tst$type) / nrow(spam_tst)
 ```
 
@@ -1384,7 +1384,7 @@ First, note that to be a reasonable classifier, it needs to outperform the obvio
 Next, we can see that using the classifier created from `fit_additive`, only a total of $127 + 157 = 284$ from the total of 3601 emails in the test set are misclassified. Overall, the accuracy in the test set it
 
 
-```r
+``` r
 mean(spam_tst_pred == spam_tst$type)
 ```
 
@@ -1395,7 +1395,7 @@ mean(spam_tst_pred == spam_tst$type)
 In other words, the test misclassification is
 
 
-```r
+``` r
 mean(spam_tst_pred != spam_tst$type)
 ```
 
@@ -1418,7 +1418,7 @@ $$
 Here we have an `R` function to calculate the sensitivity based on the confusion matrix. Note that this function is good for illustrative purposes, but is easily broken. (Think about what happens if there are no "positives" predicted.)
 
 
-```r
+``` r
 get_sens = function(conf_mat) {
   conf_mat[2, 2] / sum(conf_mat[, 2])
 }
@@ -1431,7 +1431,7 @@ $$
 $$
 
 
-```r
+``` r
 get_spec =  function(conf_mat) {
   conf_mat[1, 1] / sum(conf_mat[, 1])
 }
@@ -1440,7 +1440,7 @@ get_spec =  function(conf_mat) {
 We calculate both based on the confusion matrix we had created for our classifier. 
 
 
-```r
+``` r
 get_sens(conf_mat_50)
 ```
 
@@ -1448,7 +1448,7 @@ get_sens(conf_mat_50)
 ## [1] 0.8892025
 ```
 
-```r
+``` r
 get_spec(conf_mat_50)
 ```
 
@@ -1471,7 +1471,7 @@ Additionally, if we change the cutoff to improve sensitivity, we'll decrease spe
 First let's see what happens when we lower the cutoff from $0.5$ to $0.1$ to create a new classifier, and thus new predictions.
 
 
-```r
+``` r
 spam_tst_pred_10 = ifelse(predict(fit_additive, spam_tst, type = "response") > 0.1, 
                           "spam", 
                           "nonspam")
@@ -1480,7 +1480,7 @@ spam_tst_pred_10 = ifelse(predict(fit_additive, spam_tst, type = "response") > 0
 This is essentially *decreasing* the threshold for an email to be labeled as spam, so far *more* emails will be labeled as spam. We see that in the following confusion matrix.
 
 
-```r
+``` r
 (conf_mat_10 = make_conf_mat(predicted = spam_tst_pred_10, actual = spam_tst$type))
 ```
 
@@ -1494,7 +1494,7 @@ This is essentially *decreasing* the threshold for an email to be labeled as spa
 Unfortunately, while this does greatly reduce false negatives, false positives have almost quadrupled. We see this reflected in the sensitivity and specificity.
 
 
-```r
+``` r
 get_sens(conf_mat_10)
 ```
 
@@ -1502,7 +1502,7 @@ get_sens(conf_mat_10)
 ## [1] 0.9795342
 ```
 
-```r
+``` r
 get_spec(conf_mat_10)
 ```
 
@@ -1513,7 +1513,7 @@ get_spec(conf_mat_10)
 This classifier, using $0.1$ instead of $0.5$ has a higher sensitivity, but a much lower specificity. Clearly, we should have moved the cutoff in the other direction. Let's try $0.9$.
 
 
-```r
+``` r
 spam_tst_pred_90 = ifelse(predict(fit_additive, spam_tst, type = "response") > 0.9, 
                           "spam", 
                           "nonspam")
@@ -1522,7 +1522,7 @@ spam_tst_pred_90 = ifelse(predict(fit_additive, spam_tst, type = "response") > 0
 This is essentially *increasing* the threshold for an email to be labeled as spam, so far *fewer* emails will be labeled as spam. Again, we see that in the following confusion matrix.
 
 
-```r
+``` r
 (conf_mat_90 = make_conf_mat(predicted = spam_tst_pred_90, actual = spam_tst$type))
 ```
 
@@ -1536,7 +1536,7 @@ This is essentially *increasing* the threshold for an email to be labeled as spa
 This is the result we're looking for. We have far fewer false positives. While sensitivity is greatly reduced, specificity has gone up.
 
 
-```r
+``` r
 get_sens(conf_mat_90)
 ```
 
@@ -1544,7 +1544,7 @@ get_sens(conf_mat_90)
 ## [1] 0.6210303
 ```
 
-```r
+``` r
 get_spec(conf_mat_90)
 ```
 
@@ -1564,4 +1564,4 @@ The `R` Markdown file for this chapter can be found here:
 
 - [`logistic.Rmd`](logistic.Rmd){target="_blank"}
 
-The file was created using `R` version `4.3.2`.
+The file was created using `R` version `4.4.1`.
diff --git a/logistic_files/figure-html/unnamed-chunk-13-1.png b/logistic_files/figure-html/unnamed-chunk-13-1.png
index 308bc04..4ff3829 100644
Binary files a/logistic_files/figure-html/unnamed-chunk-13-1.png and b/logistic_files/figure-html/unnamed-chunk-13-1.png differ
diff --git a/logistic_files/figure-html/unnamed-chunk-17-1.png b/logistic_files/figure-html/unnamed-chunk-17-1.png
index e220977..3efe9d8 100644
Binary files a/logistic_files/figure-html/unnamed-chunk-17-1.png and b/logistic_files/figure-html/unnamed-chunk-17-1.png differ
diff --git a/logistic_files/figure-html/unnamed-chunk-21-1.png b/logistic_files/figure-html/unnamed-chunk-21-1.png
index 9027cda..81a0ef6 100644
Binary files a/logistic_files/figure-html/unnamed-chunk-21-1.png and b/logistic_files/figure-html/unnamed-chunk-21-1.png differ
diff --git a/logistic_files/figure-html/unnamed-chunk-9-1.png b/logistic_files/figure-html/unnamed-chunk-9-1.png
index e4a382b..63b396d 100644
Binary files a/logistic_files/figure-html/unnamed-chunk-9-1.png and b/logistic_files/figure-html/unnamed-chunk-9-1.png differ
diff --git a/mlr.md b/mlr.md
index b7b0e2d..4727e73 100644
--- a/mlr.md
+++ b/mlr.md
@@ -25,7 +25,7 @@ where $\epsilon_i \sim N(0, \sigma^2)$.
 However, it is rarely the case that a dataset will have a single predictor variable. It is also rarely the case that a response variable will only depend on a single variable. So in this chapter, we will extend our current linear model to allow a response to depend on *multiple* predictors.
 
 
-```r
+``` r
 # read the data from the web
 autompg = read.table(
   "http://archive.ics.uci.edu/ml/machine-learning-databases/auto-mpg/auto-mpg.data",
@@ -109,7 +109,7 @@ n \beta_0 + \beta_1 \sum_{i = 1}^{n} x_{i1} + \beta_2 \sum_{i = 1}^{n} x_{i2} &=
 We now have three equations and three variables, which we could solve, or we could simply let `R` solve for us.
 
 
-```r
+``` r
 mpg_model = lm(mpg ~ wt + year, data = autompg)
 coef(mpg_model)
 ```
@@ -248,7 +248,7 @@ We can then solve this expression by multiplying both sides by the inverse of $X
 To verify that this is what `R` has done for us in the case of two predictors, we create an $X$ matrix. Note that the first column is all 1s, and the remaining columns contain the data.
 
 
-```r
+``` r
 n = nrow(autompg)
 p = length(coef(mpg_model))
 X = cbind(rep(1, n), autompg$wt, autompg$year)
@@ -264,7 +264,7 @@ y = autompg$mpg
 ## [3,]   0.761401955
 ```
 
-```r
+``` r
 coef(mpg_model)
 ```
 
@@ -330,7 +330,7 @@ s_e = \sqrt{\frac{\sum_{i=1}^n (y_i - \hat{y}_i)^2}{n - p}}.
 In `R`, we could directly access $s_e$ for a fitted model, as we have seen before.
 
 
-```r
+``` r
 summary(mpg_model)$sigma
 ```
 
@@ -341,7 +341,7 @@ summary(mpg_model)$sigma
 And we can now verify that our math above is indeed calculating the same quantities.
 
 
-```r
+``` r
 y_hat = X %*% solve(t(X) %*% X) %*% t(X) %*% y
 e     = y - y_hat
 sqrt(t(e) %*% e / (n - p))
@@ -352,7 +352,7 @@ sqrt(t(e) %*% e / (n - p))
 ## [1,] 3.431367
 ```
 
-```r
+``` r
 sqrt(sum((y - y_hat) ^ 2) / (n - p))
 ```
 
@@ -365,7 +365,7 @@ sqrt(sum((y - y_hat) ^ 2) / (n - p))
 As we can see in the output below, the results of calling `summary()` are similar to SLR, but there are some differences, most obviously a new row for the added predictor variable.
 
 
-```r
+``` r
 summary(mpg_model)
 ```
 
@@ -530,7 +530,7 @@ H_0: \beta_1 = 0 \quad \text{vs} \quad H_1: \beta_1 \neq 0
 can be found in the `summary()` output, in particular:
 
 
-```r
+``` r
 summary(mpg_model)$coef
 ```
 
@@ -573,7 +573,7 @@ and the sampling distribution of $\hat{\beta}_j$ is Normal, then we can easily c
 We can find these in `R` using the same method as before. Now there will simply be additional rows for the additional $\beta$.
 
 
-```r
+``` r
 confint(mpg_model, level = 0.99)
 ```
 
@@ -631,7 +631,7 @@ Putting it all together, we obtain a confidence interval for the mean response.
 The math has changed a bit, but the process in `R` remains almost identical. Here, we create a data frame for two additional cars. One car that weighs 3500 pounds produced in 1976, as well as a second car that weighs 5000 pounds which was produced in 1981.
 
 
-```r
+``` r
 new_cars = data.frame(wt = c(3500, 5000), year = c(76, 81))
 new_cars
 ```
@@ -645,7 +645,7 @@ new_cars
 We can then use the `predict()` function with `interval = "confidence"` to obtain intervals for the mean fuel efficiency for both new cars. Again, it is important to make the data passed to `newdata` a data frame, so that `R` knows which values are for which variables.
 
 
-```r
+``` r
 predict(mpg_model, newdata = new_cars, interval = "confidence", level = 0.99)
 ```
 
@@ -660,7 +660,7 @@ predict(mpg_model, newdata = new_cars, interval = "confidence", level = 0.99)
 A word of caution here: one of these estimates is good while one is suspect.
 
 
-```r
+``` r
 new_cars$wt
 ```
 
@@ -668,7 +668,7 @@ new_cars$wt
 ## [1] 3500 5000
 ```
 
-```r
+``` r
 range(autompg$wt)
 ```
 
@@ -679,7 +679,7 @@ range(autompg$wt)
 Note that both of the weights of the new cars are within the range of observed values.
 
 
-```r
+``` r
 new_cars$year
 ```
 
@@ -687,7 +687,7 @@ new_cars$year
 ## [1] 76 81
 ```
 
-```r
+``` r
 range(autompg$year)
 ```
 
@@ -698,7 +698,7 @@ range(autompg$year)
 As are the years of each of the new cars.
 
 
-```r
+``` r
 plot(year ~ wt, data = autompg, pch = 20, col = "dodgerblue", cex = 1.5)
 points(new_cars, col = "darkorange", cex = 3, pch = "X")
 ```
@@ -712,7 +712,7 @@ However, we have to consider weight and year together now. And based on the abov
 Shifting gears back to the new data pair that can be reasonably estimated, we do a quick verification of some of the mathematics in `R`.
 
 
-```r
+``` r
 x0 = c(1, 3500, 76)
 x0 %*% beta_hat
 ```
@@ -755,7 +755,7 @@ x_{0} = \begin{bmatrix}
 Also note that, using a particular value for $x_0$, we can essentially extract certain $\hat{\beta}_j$ values.
 
 
-```r
+``` r
 beta_hat
 ```
 
@@ -766,7 +766,7 @@ beta_hat
 ## [3,]   0.761401955
 ```
 
-```r
+``` r
 x0 = c(0, 0, 1)
 x0 %*% beta_hat
 ```
@@ -811,7 +811,7 @@ Then we arrive at our updated prediction interval for MLR.
 \]
 
 
-```r
+``` r
 new_cars
 ```
 
@@ -821,7 +821,7 @@ new_cars
 ## 2 5000   81
 ```
 
-```r
+``` r
 predict(mpg_model, newdata = new_cars, interval = "prediction", level = 0.99)
 ```
 
@@ -848,7 +848,7 @@ That is,
 This means that we can still calculate $R^2$ in the same manner as before, which `R` continues to do automatically.
 
 
-```r
+``` r
 summary(mpg_model)$r.squared
 ```
 
@@ -924,7 +924,7 @@ In this case,
 That is, in the null model, we use neither of the predictors, whereas in the full (alternative) model, at least one of the predictors is useful.
 
 
-```r
+``` r
 null_mpg_model = lm(mpg ~ 1, data = autompg)
 full_mpg_model = lm(mpg ~ wt + year, data = autompg)
 anova(null_mpg_model, full_mpg_model)
@@ -945,7 +945,7 @@ anova(null_mpg_model, full_mpg_model)
 First, notice that `R` does not display the results in the same manner as the table above. More important than the layout of the table are its contents. We see that the value of the $F$ statistic is 815.55, and the p-value is extremely low, so we reject the null hypothesis at any reasonable $\alpha$ and say that the regression is significant. At least one of `wt` or `year` has a useful linear relationship with `mpg`.
 
 
-```r
+``` r
 summary(mpg_model)
 ```
 
@@ -978,7 +978,7 @@ Also, note that none of the individual $t$-tests are equivalent to the $F$-test
 We can also verify the sums of squares and degrees of freedom directly in `R`. You should match these to the table from `R` and use this to match `R`'s output to the written table above. 
 
 
-```r
+``` r
 # SSReg
 sum((fitted(full_mpg_model) - fitted(null_mpg_model)) ^ 2)
 ```
@@ -987,7 +987,7 @@ sum((fitted(full_mpg_model) - fitted(null_mpg_model)) ^ 2)
 ## [1] 19205.03
 ```
 
-```r
+``` r
 # SSE
 sum(resid(full_mpg_model) ^ 2)
 ```
@@ -996,7 +996,7 @@ sum(resid(full_mpg_model) ^ 2)
 ## [1] 4556.646
 ```
 
-```r
+``` r
 # SST
 sum(resid(null_mpg_model) ^ 2)
 ```
@@ -1005,7 +1005,7 @@ sum(resid(null_mpg_model) ^ 2)
 ## [1] 23761.67
 ```
 
-```r
+``` r
 # Degrees of Freedom: Regression
 length(coef(full_mpg_model)) - length(coef(null_mpg_model))
 ```
@@ -1014,7 +1014,7 @@ length(coef(full_mpg_model)) - length(coef(null_mpg_model))
 ## [1] 2
 ```
 
-```r
+``` r
 # Degrees of Freedom: Error
 length(resid(full_mpg_model)) - length(coef(full_mpg_model))
 ```
@@ -1023,7 +1023,7 @@ length(resid(full_mpg_model)) - length(coef(full_mpg_model))
 ## [1] 387
 ```
 
-```r
+``` r
 # Degrees of Freedom: Total
 length(resid(null_mpg_model)) - length(coef(null_mpg_model))
 ```
@@ -1077,7 +1077,7 @@ Notice that the row for "Diff" compares the sum of the squared differences of th
 For example, the `autompg` dataset has a number of additional variables that we have yet to use.
 
 
-```r
+``` r
 names(autompg)
 ```
 
@@ -1101,7 +1101,7 @@ The alternative is simply that at least one of the $\beta_{j}$ from the null is
 To perform this test in `R` we first define both models, then give them to the `anova()` commands.
 
 
-```r
+``` r
 null_mpg_model = lm(mpg ~ wt + year, data = autompg)
 #full_mpg_model = lm(mpg ~ wt + year + cyl + disp + hp + acc, data = autompg)
 full_mpg_model = lm(mpg ~ ., data = autompg)
@@ -1125,7 +1125,7 @@ Here we see that the value of the $F$ statistic is 0.553, and the p-value is ver
 Again, we verify the sums of squares and degrees of freedom directly in `R`. You should match these to the table from `R`, and use this to match `R`'s output to the written table above. 
 
 
-```r
+``` r
 # SSDiff
 sum((fitted(full_mpg_model) - fitted(null_mpg_model)) ^ 2)
 ```
@@ -1134,7 +1134,7 @@ sum((fitted(full_mpg_model) - fitted(null_mpg_model)) ^ 2)
 ## [1] 26.17981
 ```
 
-```r
+``` r
 # SSE (For Full)
 sum(resid(full_mpg_model) ^ 2)
 ```
@@ -1143,7 +1143,7 @@ sum(resid(full_mpg_model) ^ 2)
 ## [1] 4530.466
 ```
 
-```r
+``` r
 # SST (For Null)
 sum(resid(null_mpg_model) ^ 2)
 ```
@@ -1152,7 +1152,7 @@ sum(resid(null_mpg_model) ^ 2)
 ## [1] 4556.646
 ```
 
-```r
+``` r
 # Degrees of Freedom: Diff
 length(coef(full_mpg_model)) - length(coef(null_mpg_model))
 ```
@@ -1161,7 +1161,7 @@ length(coef(full_mpg_model)) - length(coef(null_mpg_model))
 ## [1] 4
 ```
 
-```r
+``` r
 # Degrees of Freedom: Full
 length(resid(full_mpg_model)) - length(coef(full_mpg_model))
 ```
@@ -1170,7 +1170,7 @@ length(resid(full_mpg_model)) - length(coef(full_mpg_model))
 ## [1] 383
 ```
 
-```r
+``` r
 # Degrees of Freedom: Null
 length(resid(null_mpg_model)) - length(coef(null_mpg_model))
 ```
@@ -1190,7 +1190,7 @@ Y_i = 5 + -2 x_{i1} + 6 x_{i2} + \epsilon_i, \qquad i = 1, 2, \ldots, n
 where $\epsilon_i \sim N(0, \sigma^2 = 16)$. Here we have two predictors, so $p = 3$.
 
 
-```r
+``` r
 set.seed(1337)
 n = 100 # sample size
 p = 3
@@ -1204,7 +1204,7 @@ sigma  = 4
 As is the norm with regression, the $x$ values are considered fixed and known quantities, so we will simulate those first, and they remain the same for the rest of the simulation study. Also note we create an `x0` which is all `1`, which we need to create our `X` matrix. If you look at the matrix formulation of regression, this unit vector of all `1`s is a "predictor" that puts the intercept into the model. We also calculate the `C` matrix for later use.
 
 
-```r
+``` r
 x0 = rep(1, n)
 x1 = sample(seq(1, 10, length = n))
 x2 = sample(seq(1, 10, length = n))
@@ -1215,7 +1215,7 @@ C = solve(t(X) %*% X)
 We then simulate the response according the model above. Lastly, we place the two predictors and response into a data frame. Note that we do **not** place `x0` in the data frame. This is a result of `R` adding an intercept by default.
 
 
-```r
+``` r
 eps      = rnorm(n, mean = 0, sd = sigma)
 y        = beta_0 + beta_1 * x1 + beta_2 * x2 + eps
 sim_data = data.frame(x1, x2, y)
@@ -1233,7 +1233,7 @@ We then calculate
 \]
 
 
-```r
+``` r
 (beta_hat = C %*% t(X) %*% y)
 ```
 
@@ -1247,7 +1247,7 @@ We then calculate
 Notice that these values are the same as the coefficients found using `lm()` in `R`.
 
 
-```r
+``` r
 coef(lm(y ~ x1 + x2, data = sim_data))
 ```
 
@@ -1259,7 +1259,7 @@ coef(lm(y ~ x1 + x2, data = sim_data))
 Also, these values are close to what we would expect.
 
 
-```r
+``` r
 c(beta_0, beta_1, beta_2)
 ```
 
@@ -1270,7 +1270,7 @@ c(beta_0, beta_1, beta_2)
 We then calculated the fitted values in order to calculate $s_e$, which we see is the same as the `sigma` which is returned by `summary()`.
 
 
-```r
+``` r
 y_hat = X %*% beta_hat
 (s_e = sqrt(sum((y - y_hat) ^ 2) / (n - p)))
 ```
@@ -1279,7 +1279,7 @@ y_hat = X %*% beta_hat
 ## [1] 4.294307
 ```
 
-```r
+``` r
 summary(lm(y ~ x1 + x2, data = sim_data))$sigma
 ```
 
@@ -1308,7 +1308,7 @@ In this case,
 Note that $C_{22}$ corresponds to the element in the **third** row and **third** column since $\beta_2$ is the **third** parameter in the model and because `R` is indexed starting at `1`. However, we index the $C$ matrix starting at `0` to match the diagonal elements to the corresponding $\beta_j$.
 
 
-```r
+``` r
 C[3, 3]
 ```
 
@@ -1316,7 +1316,7 @@ C[3, 3]
 ## [1] 0.00145343
 ```
 
-```r
+``` r
 C[2 + 1, 2 + 1]
 ```
 
@@ -1324,7 +1324,7 @@ C[2 + 1, 2 + 1]
 ## [1] 0.00145343
 ```
 
-```r
+``` r
 sigma ^ 2 * C[2 + 1, 2 + 1]
 ```
 
@@ -1335,7 +1335,7 @@ sigma ^ 2 * C[2 + 1, 2 + 1]
 We now perform the simulation a large number of times. Each time, we update the `y` variable in the data frame, leaving the `x` variables the same. We then fit a model, and store $\hat{\beta}_2$.
 
 
-```r
+``` r
 num_sims = 10000
 beta_hat_2 = rep(0, num_sims)
 for(i in 1:num_sims) {
@@ -1349,7 +1349,7 @@ for(i in 1:num_sims) {
 We then see that the mean of the simulated values is close to the true value of $\beta_2$.
 
 
-```r
+``` r
 mean(beta_hat_2)
 ```
 
@@ -1357,7 +1357,7 @@ mean(beta_hat_2)
 ## [1] 5.999723
 ```
 
-```r
+``` r
 beta_2
 ```
 
@@ -1372,7 +1372,7 @@ We also see that the variance of the simulated values is close to the true varia
 \]
 
 
-```r
+``` r
 var(beta_hat_2)
 ```
 
@@ -1380,7 +1380,7 @@ var(beta_hat_2)
 ## [1] 0.02343408
 ```
 
-```r
+``` r
 sigma ^ 2 * C[2 + 1, 2 + 1]
 ```
 
@@ -1391,7 +1391,7 @@ sigma ^ 2 * C[2 + 1, 2 + 1]
 The standard deviations found from the simulated data and the parent population are also very close.
 
 
-```r
+``` r
 sd(beta_hat_2)
 ```
 
@@ -1399,7 +1399,7 @@ sd(beta_hat_2)
 ## [1] 0.1530819
 ```
 
-```r
+``` r
 sqrt(sigma ^ 2 * C[2 + 1, 2 + 1])
 ```
 
@@ -1410,7 +1410,7 @@ sqrt(sigma ^ 2 * C[2 + 1, 2 + 1])
 Lastly, we plot a histogram of the *simulated values*, and overlay the *true distribution*.
 
 
-```r
+``` r
 hist(beta_hat_2, prob = TRUE, breaks = 20, 
      xlab = expression(hat(beta)[2]), main = "", border = "dodgerblue")
 curve(dnorm(x, mean = beta_2, sd = sqrt(sigma ^ 2 * C[2 + 1, 2 + 1])), 
@@ -1426,7 +1426,7 @@ This looks good! The simulation-based histogram appears to be Normal with mean 6
 One last check, we verify the $68 - 95 - 99.7$ rule.
 
 
-```r
+``` r
 sd_bh2 = sqrt(sigma ^ 2 * C[2 + 1, 2 + 1])
 # We expect these to be: 0.68, 0.95, 0.997
 mean(beta_2 - 1 * sd_bh2 < beta_hat_2 & beta_hat_2 < beta_2 + 1 * sd_bh2)
@@ -1436,7 +1436,7 @@ mean(beta_2 - 1 * sd_bh2 < beta_hat_2 & beta_hat_2 < beta_2 + 1 * sd_bh2)
 ## [1] 0.6807
 ```
 
-```r
+``` r
 mean(beta_2 - 2 * sd_bh2 < beta_hat_2 & beta_hat_2 < beta_2 + 2 * sd_bh2)
 ```
 
@@ -1444,7 +1444,7 @@ mean(beta_2 - 2 * sd_bh2 < beta_hat_2 & beta_hat_2 < beta_2 + 2 * sd_bh2)
 ## [1] 0.9529
 ```
 
-```r
+``` r
 mean(beta_2 - 3 * sd_bh2 < beta_hat_2 & beta_hat_2 < beta_2 + 3 * sd_bh2)
 ```
 
@@ -1458,4 +1458,4 @@ The `R` Markdown file for this chapter can be found here:
 
 - [`mlr.Rmd`](mlr.Rmd){target="_blank"}
 
-The file was created using `R` version `4.3.2`.
+The file was created using `R` version `4.4.1`.
diff --git a/mlr_files/figure-html/unnamed-chunk-30-1.png b/mlr_files/figure-html/unnamed-chunk-30-1.png
index 7b151d5..0e3eebf 100644
Binary files a/mlr_files/figure-html/unnamed-chunk-30-1.png and b/mlr_files/figure-html/unnamed-chunk-30-1.png differ
diff --git a/model-building.html b/model-building.html
index e55b84b..d5edc1a 100644
--- a/model-building.html
+++ b/model-building.html
@@ -6,7 +6,7 @@
   <meta http-equiv="X-UA-Compatible" content="IE=edge" />
   <title>Chapter 10 Model Building | Applied Statistics with R</title>
   <meta name="description" content="Chapter 10 Model Building | Applied Statistics with R" />
-  <meta name="generator" content="bookdown 0.37 and GitBook 2.6.7" />
+  <meta name="generator" content="bookdown 0.40 and GitBook 2.6.7" />
 
   <meta property="og:title" content="Chapter 10 Model Building | Applied Statistics with R" />
   <meta property="og:type" content="book" />
@@ -55,7 +55,7 @@
 
 <style type="text/css">
 pre > code.sourceCode { white-space: pre; position: relative; }
-pre > code.sourceCode > span { display: inline-block; line-height: 1.25; }
+pre > code.sourceCode > span { line-height: 1.25; }
 pre > code.sourceCode > span:empty { height: 1.2em; }
 .sourceCode { overflow: visible; }
 code.sourceCode > span { color: inherit; text-decoration: inherit; }
@@ -565,7 +565,7 @@ <h3><span class="header-section-number">10.1.4</span> Assumed Model, Fitted Mode
 \]</span></p>
 <p>assumes that <span class="math inline">\(y\)</span> is a linear combination of <span class="math inline">\(x_{1}\)</span>, <span class="math inline">\(x_{3}\)</span>, and <span class="math inline">\(x_{5}\)</span> as well as some noise. This assumes that the effect of <span class="math inline">\(x_{1}\)</span> on <span class="math inline">\(y\)</span> is <span class="math inline">\(\beta_1\)</span>, which is the same for all values of <span class="math inline">\(x_{3}\)</span> and <span class="math inline">\(x_{5}\)</span>. That is, we are using the <em>family</em> of linear models with a particular <em>form</em>.</p>
 <p>Suppose we then <em>fit</em> this model to some data and obtain the <strong>fitted model</strong>. For example, in <code>R</code> we would use</p>
-<div class="sourceCode" id="cb707"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb707-1"><a href="model-building.html#cb707-1" aria-hidden="true" tabindex="-1"></a>fit <span class="ot">=</span> <span class="fu">lm</span>(y <span class="sc">~</span> x1 <span class="sc">+</span> x3 <span class="sc">+</span> x5, <span class="at">data =</span> some_data)</span></code></pre></div>
+<div class="sourceCode" id="cb707"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb707-1"><a href="model-building.html#cb707-1" tabindex="-1"></a>fit <span class="ot">=</span> <span class="fu">lm</span>(y <span class="sc">~</span> x1 <span class="sc">+</span> x3 <span class="sc">+</span> x5, <span class="at">data =</span> some_data)</span></code></pre></div>
 <p>This is <code>R</code>’s way of saying the <em>family</em> is <em>linear</em> and specifying the <em>form</em> from above. An additive model with the specified predictors as well as an intercept. We then obtain</p>
 <p><span class="math display">\[
 \hat{y} = 1.5 + 0.9 x_{1} + 1.1 x_{3} + 2.3 x_{5}.
@@ -601,7 +601,7 @@ <h3><span class="header-section-number">10.2.1</span> Explanation<a href="model-
 <h4><span class="header-section-number">10.2.1.1</span> Correlation and Causation<a href="model-building.html#correlation-and-causation" class="anchor-section" aria-label="Anchor link to header"></a></h4>
 <p>A word of caution when using a model to <em>explain</em> a relationship. There are two terms often used to describe a relationship between two variables: <em>causation</em> and <em>correlation</em>. <a href="https://xkcd.com/552/" target="_blank">Correlation</a> is often also referred to as association.</p>
 <p>Just because two variables are correlated does not necessarily mean that one causes the other. For example, consider modeling <code>mpg</code> as only a function of <code>hp</code>.</p>
-<div class="sourceCode" id="cb708"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb708-1"><a href="model-building.html#cb708-1" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(mpg <span class="sc">~</span> hp, <span class="at">data =</span> autompg, <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">cex =</span> <span class="fl">1.5</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb708"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb708-1"><a href="model-building.html#cb708-1" tabindex="-1"></a><span class="fu">plot</span>(mpg <span class="sc">~</span> hp, <span class="at">data =</span> autompg, <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">cex =</span> <span class="fl">1.5</span>)</span></code></pre></div>
 <p><img src="model-building_files/figure-html/unnamed-chunk-5-1.png" width="672" style="display: block; margin: auto;" /></p>
 <p>Does an increase in horsepower cause a drop in fuel efficiency? Or, perhaps the causality is reversed and an increase in fuel efficiency causes a decrease in horsepower. Or, perhaps there is a third variable that explains both!</p>
 <p>The issue here is that we have <strong>observational</strong> data. With observational data, we can only detect <em>associations</em>. To speak with confidence about <em>causality</em>, we would need to run <strong>experiments</strong>. Often, this decision is made for us, before we ever see data, so we can only modify our interpretation.</p>
@@ -682,7 +682,7 @@ <h2><span class="header-section-number">10.4</span> <code>R</code> Markdown<a hr
 <ul>
 <li><a href="model-building.Rmd" target="_blank"><code>model-building.Rmd</code></a></li>
 </ul>
-<p>The file was created using <code>R</code> version <code>4.3.2</code>.</p>
+<p>The file was created using <code>R</code> version <code>4.4.1</code>.</p>
 
 </div>
 </div>
diff --git a/model-building.md b/model-building.md
index ebefb40..e318d99 100644
--- a/model-building.md
+++ b/model-building.md
@@ -137,7 +137,7 @@ assumes that $y$ is a linear combination of $x_{1}$, $x_{3}$, and $x_{5}$ as wel
 Suppose we then *fit* this model to some data and obtain the **fitted model**. For example, in `R` we would use
 
 
-```r
+``` r
 fit = lm(y ~ x1 + x3 + x5, data = some_data)
 ```
 
@@ -196,7 +196,7 @@ A word of caution when using a model to *explain* a relationship. There are two
 Just because two variables are correlated does not necessarily mean that one causes the other. For example, consider modeling `mpg` as only a function of `hp`.
 
 
-```r
+``` r
 plot(mpg ~ hp, data = autompg, col = "dodgerblue", pch = 20, cex = 1.5)
 ```
 
@@ -275,4 +275,4 @@ The `R` Markdown file for this chapter can be found here:
 
 - [`model-building.Rmd`](model-building.Rmd){target="_blank"}
 
-The file was created using `R` version `4.3.2`.
+The file was created using `R` version `4.4.1`.
diff --git a/model-diagnostics.html b/model-diagnostics.html
index b2e05d5..c8af872 100644
--- a/model-diagnostics.html
+++ b/model-diagnostics.html
@@ -6,7 +6,7 @@
   <meta http-equiv="X-UA-Compatible" content="IE=edge" />
   <title>Chapter 13 Model Diagnostics | Applied Statistics with R</title>
   <meta name="description" content="Chapter 13 Model Diagnostics | Applied Statistics with R" />
-  <meta name="generator" content="bookdown 0.37 and GitBook 2.6.7" />
+  <meta name="generator" content="bookdown 0.40 and GitBook 2.6.7" />
 
   <meta property="og:title" content="Chapter 13 Model Diagnostics | Applied Statistics with R" />
   <meta property="og:type" content="book" />
@@ -55,7 +55,7 @@
 
 <style type="text/css">
 pre > code.sourceCode { white-space: pre; position: relative; }
-pre > code.sourceCode > span { display: inline-block; line-height: 1.25; }
+pre > code.sourceCode > span { line-height: 1.25; }
 pre > code.sourceCode > span:empty { height: 1.2em; }
 .sourceCode { overflow: visible; }
 code.sourceCode > span { color: inherit; text-decoration: inherit; }
@@ -558,30 +558,30 @@ <h2><span class="header-section-number">13.2</span> Checking Assumptions<a href=
 <p><span class="math display">\[
 \text{Model 3:} \quad Y = 3 + 5x^2 + \epsilon, \quad \epsilon \sim N(0, 25)
 \]</span></p>
-<div class="sourceCode" id="cb886"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb886-1"><a href="model-diagnostics.html#cb886-1" aria-hidden="true" tabindex="-1"></a>sim_1 <span class="ot">=</span> <span class="cf">function</span>(<span class="at">sample_size =</span> <span class="dv">500</span>) {</span>
-<span id="cb886-2"><a href="model-diagnostics.html#cb886-2" aria-hidden="true" tabindex="-1"></a>  x <span class="ot">=</span> <span class="fu">runif</span>(<span class="at">n =</span> sample_size) <span class="sc">*</span> <span class="dv">5</span></span>
-<span id="cb886-3"><a href="model-diagnostics.html#cb886-3" aria-hidden="true" tabindex="-1"></a>  y <span class="ot">=</span> <span class="dv">3</span> <span class="sc">+</span> <span class="dv">5</span> <span class="sc">*</span> x <span class="sc">+</span> <span class="fu">rnorm</span>(<span class="at">n =</span> sample_size, <span class="at">mean =</span> <span class="dv">0</span>, <span class="at">sd =</span> <span class="dv">1</span>)</span>
-<span id="cb886-4"><a href="model-diagnostics.html#cb886-4" aria-hidden="true" tabindex="-1"></a>  <span class="fu">data.frame</span>(x, y)</span>
-<span id="cb886-5"><a href="model-diagnostics.html#cb886-5" aria-hidden="true" tabindex="-1"></a>}</span>
-<span id="cb886-6"><a href="model-diagnostics.html#cb886-6" aria-hidden="true" tabindex="-1"></a></span>
-<span id="cb886-7"><a href="model-diagnostics.html#cb886-7" aria-hidden="true" tabindex="-1"></a>sim_2 <span class="ot">=</span> <span class="cf">function</span>(<span class="at">sample_size =</span> <span class="dv">500</span>) {</span>
-<span id="cb886-8"><a href="model-diagnostics.html#cb886-8" aria-hidden="true" tabindex="-1"></a>  x <span class="ot">=</span> <span class="fu">runif</span>(<span class="at">n =</span> sample_size) <span class="sc">*</span> <span class="dv">5</span></span>
-<span id="cb886-9"><a href="model-diagnostics.html#cb886-9" aria-hidden="true" tabindex="-1"></a>  y <span class="ot">=</span> <span class="dv">3</span> <span class="sc">+</span> <span class="dv">5</span> <span class="sc">*</span> x <span class="sc">+</span> <span class="fu">rnorm</span>(<span class="at">n =</span> sample_size, <span class="at">mean =</span> <span class="dv">0</span>, <span class="at">sd =</span> x)</span>
-<span id="cb886-10"><a href="model-diagnostics.html#cb886-10" aria-hidden="true" tabindex="-1"></a>  <span class="fu">data.frame</span>(x, y)</span>
-<span id="cb886-11"><a href="model-diagnostics.html#cb886-11" aria-hidden="true" tabindex="-1"></a>}</span>
-<span id="cb886-12"><a href="model-diagnostics.html#cb886-12" aria-hidden="true" tabindex="-1"></a></span>
-<span id="cb886-13"><a href="model-diagnostics.html#cb886-13" aria-hidden="true" tabindex="-1"></a>sim_3 <span class="ot">=</span> <span class="cf">function</span>(<span class="at">sample_size =</span> <span class="dv">500</span>) {</span>
-<span id="cb886-14"><a href="model-diagnostics.html#cb886-14" aria-hidden="true" tabindex="-1"></a>  x <span class="ot">=</span> <span class="fu">runif</span>(<span class="at">n =</span> sample_size) <span class="sc">*</span> <span class="dv">5</span></span>
-<span id="cb886-15"><a href="model-diagnostics.html#cb886-15" aria-hidden="true" tabindex="-1"></a>  y <span class="ot">=</span> <span class="dv">3</span> <span class="sc">+</span> <span class="dv">5</span> <span class="sc">*</span> x <span class="sc">^</span> <span class="dv">2</span> <span class="sc">+</span> <span class="fu">rnorm</span>(<span class="at">n =</span> sample_size, <span class="at">mean =</span> <span class="dv">0</span>, <span class="at">sd =</span> <span class="dv">5</span>)</span>
-<span id="cb886-16"><a href="model-diagnostics.html#cb886-16" aria-hidden="true" tabindex="-1"></a>  <span class="fu">data.frame</span>(x, y)</span>
-<span id="cb886-17"><a href="model-diagnostics.html#cb886-17" aria-hidden="true" tabindex="-1"></a>}</span></code></pre></div>
+<div class="sourceCode" id="cb886"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb886-1"><a href="model-diagnostics.html#cb886-1" tabindex="-1"></a>sim_1 <span class="ot">=</span> <span class="cf">function</span>(<span class="at">sample_size =</span> <span class="dv">500</span>) {</span>
+<span id="cb886-2"><a href="model-diagnostics.html#cb886-2" tabindex="-1"></a>  x <span class="ot">=</span> <span class="fu">runif</span>(<span class="at">n =</span> sample_size) <span class="sc">*</span> <span class="dv">5</span></span>
+<span id="cb886-3"><a href="model-diagnostics.html#cb886-3" tabindex="-1"></a>  y <span class="ot">=</span> <span class="dv">3</span> <span class="sc">+</span> <span class="dv">5</span> <span class="sc">*</span> x <span class="sc">+</span> <span class="fu">rnorm</span>(<span class="at">n =</span> sample_size, <span class="at">mean =</span> <span class="dv">0</span>, <span class="at">sd =</span> <span class="dv">1</span>)</span>
+<span id="cb886-4"><a href="model-diagnostics.html#cb886-4" tabindex="-1"></a>  <span class="fu">data.frame</span>(x, y)</span>
+<span id="cb886-5"><a href="model-diagnostics.html#cb886-5" tabindex="-1"></a>}</span>
+<span id="cb886-6"><a href="model-diagnostics.html#cb886-6" tabindex="-1"></a></span>
+<span id="cb886-7"><a href="model-diagnostics.html#cb886-7" tabindex="-1"></a>sim_2 <span class="ot">=</span> <span class="cf">function</span>(<span class="at">sample_size =</span> <span class="dv">500</span>) {</span>
+<span id="cb886-8"><a href="model-diagnostics.html#cb886-8" tabindex="-1"></a>  x <span class="ot">=</span> <span class="fu">runif</span>(<span class="at">n =</span> sample_size) <span class="sc">*</span> <span class="dv">5</span></span>
+<span id="cb886-9"><a href="model-diagnostics.html#cb886-9" tabindex="-1"></a>  y <span class="ot">=</span> <span class="dv">3</span> <span class="sc">+</span> <span class="dv">5</span> <span class="sc">*</span> x <span class="sc">+</span> <span class="fu">rnorm</span>(<span class="at">n =</span> sample_size, <span class="at">mean =</span> <span class="dv">0</span>, <span class="at">sd =</span> x)</span>
+<span id="cb886-10"><a href="model-diagnostics.html#cb886-10" tabindex="-1"></a>  <span class="fu">data.frame</span>(x, y)</span>
+<span id="cb886-11"><a href="model-diagnostics.html#cb886-11" tabindex="-1"></a>}</span>
+<span id="cb886-12"><a href="model-diagnostics.html#cb886-12" tabindex="-1"></a></span>
+<span id="cb886-13"><a href="model-diagnostics.html#cb886-13" tabindex="-1"></a>sim_3 <span class="ot">=</span> <span class="cf">function</span>(<span class="at">sample_size =</span> <span class="dv">500</span>) {</span>
+<span id="cb886-14"><a href="model-diagnostics.html#cb886-14" tabindex="-1"></a>  x <span class="ot">=</span> <span class="fu">runif</span>(<span class="at">n =</span> sample_size) <span class="sc">*</span> <span class="dv">5</span></span>
+<span id="cb886-15"><a href="model-diagnostics.html#cb886-15" tabindex="-1"></a>  y <span class="ot">=</span> <span class="dv">3</span> <span class="sc">+</span> <span class="dv">5</span> <span class="sc">*</span> x <span class="sc">^</span> <span class="dv">2</span> <span class="sc">+</span> <span class="fu">rnorm</span>(<span class="at">n =</span> sample_size, <span class="at">mean =</span> <span class="dv">0</span>, <span class="at">sd =</span> <span class="dv">5</span>)</span>
+<span id="cb886-16"><a href="model-diagnostics.html#cb886-16" tabindex="-1"></a>  <span class="fu">data.frame</span>(x, y)</span>
+<span id="cb886-17"><a href="model-diagnostics.html#cb886-17" tabindex="-1"></a>}</span></code></pre></div>
 <div id="fitted-versus-residuals-plot" class="section level3 hasAnchor" number="13.2.1">
 <h3><span class="header-section-number">13.2.1</span> Fitted versus Residuals Plot<a href="model-diagnostics.html#fitted-versus-residuals-plot" class="anchor-section" aria-label="Anchor link to header"></a></h3>
 <p>Probably our most useful tool will be a <strong>Fitted versus Residuals Plot</strong>. It will be useful for checking both the <strong>linearity</strong> and <strong>constant variance</strong> assumptions.</p>
 <p>Data generated from Model 1 above should not show any signs of violating assumptions, so we’ll use this to see what a good fitted versus residuals plot should look like. First, we’ll simulate observations from this model.</p>
-<div class="sourceCode" id="cb887"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb887-1"><a href="model-diagnostics.html#cb887-1" aria-hidden="true" tabindex="-1"></a><span class="fu">set.seed</span>(<span class="dv">42</span>)</span>
-<span id="cb887-2"><a href="model-diagnostics.html#cb887-2" aria-hidden="true" tabindex="-1"></a>sim_data_1 <span class="ot">=</span> <span class="fu">sim_1</span>()</span>
-<span id="cb887-3"><a href="model-diagnostics.html#cb887-3" aria-hidden="true" tabindex="-1"></a><span class="fu">head</span>(sim_data_1)</span></code></pre></div>
+<div class="sourceCode" id="cb887"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb887-1"><a href="model-diagnostics.html#cb887-1" tabindex="-1"></a><span class="fu">set.seed</span>(<span class="dv">42</span>)</span>
+<span id="cb887-2"><a href="model-diagnostics.html#cb887-2" tabindex="-1"></a>sim_data_1 <span class="ot">=</span> <span class="fu">sim_1</span>()</span>
+<span id="cb887-3"><a href="model-diagnostics.html#cb887-3" tabindex="-1"></a><span class="fu">head</span>(sim_data_1)</span></code></pre></div>
 <pre><code>##          x         y
 ## 1 4.574030 24.773995
 ## 2 4.685377 26.475936
@@ -590,15 +590,15 @@ <h3><span class="header-section-number">13.2.1</span> Fitted versus Residuals Pl
 ## 5 3.208728 20.341344
 ## 6 2.595480 14.943525</code></pre>
 <p>We then fit the model and add the fitted line to a scatterplot.</p>
-<div class="sourceCode" id="cb889"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb889-1"><a href="model-diagnostics.html#cb889-1" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(y <span class="sc">~</span> x, <span class="at">data =</span> sim_data_1, <span class="at">col =</span> <span class="st">&quot;grey&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>,</span>
-<span id="cb889-2"><a href="model-diagnostics.html#cb889-2" aria-hidden="true" tabindex="-1"></a>     <span class="at">main =</span> <span class="st">&quot;Data from Model 1&quot;</span>)</span>
-<span id="cb889-3"><a href="model-diagnostics.html#cb889-3" aria-hidden="true" tabindex="-1"></a>fit_1 <span class="ot">=</span> <span class="fu">lm</span>(y <span class="sc">~</span> x, <span class="at">data =</span> sim_data_1)</span>
-<span id="cb889-4"><a href="model-diagnostics.html#cb889-4" aria-hidden="true" tabindex="-1"></a><span class="fu">abline</span>(fit_1, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">3</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb889"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb889-1"><a href="model-diagnostics.html#cb889-1" tabindex="-1"></a><span class="fu">plot</span>(y <span class="sc">~</span> x, <span class="at">data =</span> sim_data_1, <span class="at">col =</span> <span class="st">&quot;grey&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>,</span>
+<span id="cb889-2"><a href="model-diagnostics.html#cb889-2" tabindex="-1"></a>     <span class="at">main =</span> <span class="st">&quot;Data from Model 1&quot;</span>)</span>
+<span id="cb889-3"><a href="model-diagnostics.html#cb889-3" tabindex="-1"></a>fit_1 <span class="ot">=</span> <span class="fu">lm</span>(y <span class="sc">~</span> x, <span class="at">data =</span> sim_data_1)</span>
+<span id="cb889-4"><a href="model-diagnostics.html#cb889-4" tabindex="-1"></a><span class="fu">abline</span>(fit_1, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">3</span>)</span></code></pre></div>
 <p><img src="diagnostics_files/figure-html/unnamed-chunk-4-1.png" width="672" style="display: block; margin: auto;" /></p>
 <p>We now plot a fitted versus residuals plot. Note, this is residuals on the <span class="math inline">\(y\)</span>-axis despite the ordering in the name. Sometimes you will see this called a residuals versus fitted, or residuals versus predicted plot.</p>
-<div class="sourceCode" id="cb890"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb890-1"><a href="model-diagnostics.html#cb890-1" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(<span class="fu">fitted</span>(fit_1), <span class="fu">resid</span>(fit_1), <span class="at">col =</span> <span class="st">&quot;grey&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>,</span>
-<span id="cb890-2"><a href="model-diagnostics.html#cb890-2" aria-hidden="true" tabindex="-1"></a>     <span class="at">xlab =</span> <span class="st">&quot;Fitted&quot;</span>, <span class="at">ylab =</span> <span class="st">&quot;Residuals&quot;</span>, <span class="at">main =</span> <span class="st">&quot;Data from Model 1&quot;</span>)</span>
-<span id="cb890-3"><a href="model-diagnostics.html#cb890-3" aria-hidden="true" tabindex="-1"></a><span class="fu">abline</span>(<span class="at">h =</span> <span class="dv">0</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb890"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb890-1"><a href="model-diagnostics.html#cb890-1" tabindex="-1"></a><span class="fu">plot</span>(<span class="fu">fitted</span>(fit_1), <span class="fu">resid</span>(fit_1), <span class="at">col =</span> <span class="st">&quot;grey&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>,</span>
+<span id="cb890-2"><a href="model-diagnostics.html#cb890-2" tabindex="-1"></a>     <span class="at">xlab =</span> <span class="st">&quot;Fitted&quot;</span>, <span class="at">ylab =</span> <span class="st">&quot;Residuals&quot;</span>, <span class="at">main =</span> <span class="st">&quot;Data from Model 1&quot;</span>)</span>
+<span id="cb890-3"><a href="model-diagnostics.html#cb890-3" tabindex="-1"></a><span class="fu">abline</span>(<span class="at">h =</span> <span class="dv">0</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
 <p><img src="diagnostics_files/figure-html/unnamed-chunk-5-1.png" width="672" style="display: block; margin: auto;" /></p>
 <p>We should look for two things in this plot.</p>
 <ul>
@@ -608,31 +608,31 @@ <h3><span class="header-section-number">13.2.1</span> Fitted versus Residuals Pl
 <p>Here we see this is the case for both.</p>
 <p>To get a better idea of how a fitted versus residuals plot can be useful, we will simulate from models with violated assumptions.</p>
 <p>Model 2 is an example of non-constant variance. In this case, the variance is larger for larger values of the predictor variable <span class="math inline">\(x\)</span>.</p>
-<div class="sourceCode" id="cb891"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb891-1"><a href="model-diagnostics.html#cb891-1" aria-hidden="true" tabindex="-1"></a><span class="fu">set.seed</span>(<span class="dv">42</span>)</span>
-<span id="cb891-2"><a href="model-diagnostics.html#cb891-2" aria-hidden="true" tabindex="-1"></a>sim_data_2 <span class="ot">=</span> <span class="fu">sim_2</span>()</span>
-<span id="cb891-3"><a href="model-diagnostics.html#cb891-3" aria-hidden="true" tabindex="-1"></a>fit_2 <span class="ot">=</span> <span class="fu">lm</span>(y <span class="sc">~</span> x, <span class="at">data =</span> sim_data_2)</span>
-<span id="cb891-4"><a href="model-diagnostics.html#cb891-4" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(y <span class="sc">~</span> x, <span class="at">data =</span> sim_data_2, <span class="at">col =</span> <span class="st">&quot;grey&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>,</span>
-<span id="cb891-5"><a href="model-diagnostics.html#cb891-5" aria-hidden="true" tabindex="-1"></a>     <span class="at">main =</span> <span class="st">&quot;Data from Model 2&quot;</span>)</span>
-<span id="cb891-6"><a href="model-diagnostics.html#cb891-6" aria-hidden="true" tabindex="-1"></a><span class="fu">abline</span>(fit_2, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">3</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb891"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb891-1"><a href="model-diagnostics.html#cb891-1" tabindex="-1"></a><span class="fu">set.seed</span>(<span class="dv">42</span>)</span>
+<span id="cb891-2"><a href="model-diagnostics.html#cb891-2" tabindex="-1"></a>sim_data_2 <span class="ot">=</span> <span class="fu">sim_2</span>()</span>
+<span id="cb891-3"><a href="model-diagnostics.html#cb891-3" tabindex="-1"></a>fit_2 <span class="ot">=</span> <span class="fu">lm</span>(y <span class="sc">~</span> x, <span class="at">data =</span> sim_data_2)</span>
+<span id="cb891-4"><a href="model-diagnostics.html#cb891-4" tabindex="-1"></a><span class="fu">plot</span>(y <span class="sc">~</span> x, <span class="at">data =</span> sim_data_2, <span class="at">col =</span> <span class="st">&quot;grey&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>,</span>
+<span id="cb891-5"><a href="model-diagnostics.html#cb891-5" tabindex="-1"></a>     <span class="at">main =</span> <span class="st">&quot;Data from Model 2&quot;</span>)</span>
+<span id="cb891-6"><a href="model-diagnostics.html#cb891-6" tabindex="-1"></a><span class="fu">abline</span>(fit_2, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">3</span>)</span></code></pre></div>
 <p><img src="diagnostics_files/figure-html/unnamed-chunk-6-1.png" width="672" style="display: block; margin: auto;" /></p>
 <p>This actually is rather easy to see here by adding the fitted line to a scatterplot. This is because we are only performing simple linear regression. With multiple regression, a fitted versus residuals plot is a necessity, since adding a fitted regression to a scatterplot isn’t exactly possible.</p>
-<div class="sourceCode" id="cb892"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb892-1"><a href="model-diagnostics.html#cb892-1" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(<span class="fu">fitted</span>(fit_2), <span class="fu">resid</span>(fit_2), <span class="at">col =</span> <span class="st">&quot;grey&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>,</span>
-<span id="cb892-2"><a href="model-diagnostics.html#cb892-2" aria-hidden="true" tabindex="-1"></a>     <span class="at">xlab =</span> <span class="st">&quot;Fitted&quot;</span>, <span class="at">ylab =</span> <span class="st">&quot;Residuals&quot;</span>, <span class="at">main =</span> <span class="st">&quot;Data from Model 2&quot;</span>)</span>
-<span id="cb892-3"><a href="model-diagnostics.html#cb892-3" aria-hidden="true" tabindex="-1"></a><span class="fu">abline</span>(<span class="at">h =</span> <span class="dv">0</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb892"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb892-1"><a href="model-diagnostics.html#cb892-1" tabindex="-1"></a><span class="fu">plot</span>(<span class="fu">fitted</span>(fit_2), <span class="fu">resid</span>(fit_2), <span class="at">col =</span> <span class="st">&quot;grey&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>,</span>
+<span id="cb892-2"><a href="model-diagnostics.html#cb892-2" tabindex="-1"></a>     <span class="at">xlab =</span> <span class="st">&quot;Fitted&quot;</span>, <span class="at">ylab =</span> <span class="st">&quot;Residuals&quot;</span>, <span class="at">main =</span> <span class="st">&quot;Data from Model 2&quot;</span>)</span>
+<span id="cb892-3"><a href="model-diagnostics.html#cb892-3" tabindex="-1"></a><span class="fu">abline</span>(<span class="at">h =</span> <span class="dv">0</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
 <p><img src="diagnostics_files/figure-html/unnamed-chunk-7-1.png" width="672" style="display: block; margin: auto;" /></p>
 <p>On the fitted versus residuals plot, we see two things very clearly. For any fitted value, the residuals seem roughly centered at 0. This is good! The linearity assumption is not violated. However, we also see very clearly, that for larger fitted values, the spread of the residuals is larger. This is bad! The constant variance assumption is violated here.</p>
 <p>Now we will demonstrate a model which does not meet the linearity assumption. Model 3 is an example of a model where <span class="math inline">\(Y\)</span> is not a linear combination of the predictors. In this case the predictor is <span class="math inline">\(x\)</span>, but the model uses <span class="math inline">\(x^2\)</span>. (We’ll see later that this is something that a “linear” model can deal with. The fix is simple, just make <span class="math inline">\(x^2\)</span> a predictor!)</p>
-<div class="sourceCode" id="cb893"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb893-1"><a href="model-diagnostics.html#cb893-1" aria-hidden="true" tabindex="-1"></a><span class="fu">set.seed</span>(<span class="dv">42</span>)</span>
-<span id="cb893-2"><a href="model-diagnostics.html#cb893-2" aria-hidden="true" tabindex="-1"></a>sim_data_3 <span class="ot">=</span> <span class="fu">sim_3</span>()</span>
-<span id="cb893-3"><a href="model-diagnostics.html#cb893-3" aria-hidden="true" tabindex="-1"></a>fit_3 <span class="ot">=</span> <span class="fu">lm</span>(y <span class="sc">~</span> x, <span class="at">data =</span> sim_data_3)</span>
-<span id="cb893-4"><a href="model-diagnostics.html#cb893-4" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(y <span class="sc">~</span> x, <span class="at">data =</span> sim_data_3, <span class="at">col =</span> <span class="st">&quot;grey&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>,</span>
-<span id="cb893-5"><a href="model-diagnostics.html#cb893-5" aria-hidden="true" tabindex="-1"></a>     <span class="at">main =</span> <span class="st">&quot;Data from Model 3&quot;</span>)</span>
-<span id="cb893-6"><a href="model-diagnostics.html#cb893-6" aria-hidden="true" tabindex="-1"></a><span class="fu">abline</span>(fit_3, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">3</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb893"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb893-1"><a href="model-diagnostics.html#cb893-1" tabindex="-1"></a><span class="fu">set.seed</span>(<span class="dv">42</span>)</span>
+<span id="cb893-2"><a href="model-diagnostics.html#cb893-2" tabindex="-1"></a>sim_data_3 <span class="ot">=</span> <span class="fu">sim_3</span>()</span>
+<span id="cb893-3"><a href="model-diagnostics.html#cb893-3" tabindex="-1"></a>fit_3 <span class="ot">=</span> <span class="fu">lm</span>(y <span class="sc">~</span> x, <span class="at">data =</span> sim_data_3)</span>
+<span id="cb893-4"><a href="model-diagnostics.html#cb893-4" tabindex="-1"></a><span class="fu">plot</span>(y <span class="sc">~</span> x, <span class="at">data =</span> sim_data_3, <span class="at">col =</span> <span class="st">&quot;grey&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>,</span>
+<span id="cb893-5"><a href="model-diagnostics.html#cb893-5" tabindex="-1"></a>     <span class="at">main =</span> <span class="st">&quot;Data from Model 3&quot;</span>)</span>
+<span id="cb893-6"><a href="model-diagnostics.html#cb893-6" tabindex="-1"></a><span class="fu">abline</span>(fit_3, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">3</span>)</span></code></pre></div>
 <p><img src="diagnostics_files/figure-html/unnamed-chunk-8-1.png" width="672" style="display: block; margin: auto;" /></p>
 <p>Again, this is rather clear on the scatterplot, but again, we wouldn’t be able to check this plot for multiple regression.</p>
-<div class="sourceCode" id="cb894"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb894-1"><a href="model-diagnostics.html#cb894-1" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(<span class="fu">fitted</span>(fit_3), <span class="fu">resid</span>(fit_3), <span class="at">col =</span> <span class="st">&quot;grey&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>,</span>
-<span id="cb894-2"><a href="model-diagnostics.html#cb894-2" aria-hidden="true" tabindex="-1"></a>     <span class="at">xlab =</span> <span class="st">&quot;Fitted&quot;</span>, <span class="at">ylab =</span> <span class="st">&quot;Residuals&quot;</span>, <span class="at">main =</span> <span class="st">&quot;Data from Model 3&quot;</span>)</span>
-<span id="cb894-3"><a href="model-diagnostics.html#cb894-3" aria-hidden="true" tabindex="-1"></a><span class="fu">abline</span>(<span class="at">h =</span> <span class="dv">0</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb894"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb894-1"><a href="model-diagnostics.html#cb894-1" tabindex="-1"></a><span class="fu">plot</span>(<span class="fu">fitted</span>(fit_3), <span class="fu">resid</span>(fit_3), <span class="at">col =</span> <span class="st">&quot;grey&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>,</span>
+<span id="cb894-2"><a href="model-diagnostics.html#cb894-2" tabindex="-1"></a>     <span class="at">xlab =</span> <span class="st">&quot;Fitted&quot;</span>, <span class="at">ylab =</span> <span class="st">&quot;Residuals&quot;</span>, <span class="at">main =</span> <span class="st">&quot;Data from Model 3&quot;</span>)</span>
+<span id="cb894-3"><a href="model-diagnostics.html#cb894-3" tabindex="-1"></a><span class="fu">abline</span>(<span class="at">h =</span> <span class="dv">0</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
 <p><img src="diagnostics_files/figure-html/unnamed-chunk-9-1.png" width="672" style="display: block; margin: auto;" /></p>
 <p>This time on the fitted versus residuals plot, for any fitted value, the spread of the residuals is about the same. However, they are not even close to centered at zero! At small and large fitted values the model is underestimating, while at medium fitted values, the model is overestimating. These are systematic errors, not random noise. So the constant variance assumption is met, but the linearity assumption is violated. The form of our model is simply wrong. We’re trying to fit a line to a curve!</p>
 </div>
@@ -646,29 +646,29 @@ <h3><span class="header-section-number">13.2.2</span> Breusch-Pagan Test<a href=
 </ul>
 <p>Isn’t that convenient? A test that will specifically test the <strong>constant variance</strong> assumption.</p>
 <p>The Breusch-Pagan Test can not be performed by default in <code>R</code>, however the function <code>bptest</code> in the <code>lmtest</code> package implements the test.</p>
-<div class="sourceCode" id="cb895"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb895-1"><a href="model-diagnostics.html#cb895-1" aria-hidden="true" tabindex="-1"></a><span class="co">#install.packages(&quot;lmtest&quot;)</span></span>
-<span id="cb895-2"><a href="model-diagnostics.html#cb895-2" aria-hidden="true" tabindex="-1"></a><span class="fu">library</span>(lmtest)</span></code></pre></div>
+<div class="sourceCode" id="cb895"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb895-1"><a href="model-diagnostics.html#cb895-1" tabindex="-1"></a><span class="co">#install.packages(&quot;lmtest&quot;)</span></span>
+<span id="cb895-2"><a href="model-diagnostics.html#cb895-2" tabindex="-1"></a><span class="fu">library</span>(lmtest)</span></code></pre></div>
 <p>Let’s try it on the three models we fit above. Recall,</p>
 <ul>
 <li><code>fit_1</code> had no violation of assumptions,</li>
 <li><code>fit_2</code> violated the constant variance assumption, but not linearity,</li>
 <li><code>fit_3</code> violated linearity, but not constant variance.</li>
 </ul>
-<div class="sourceCode" id="cb896"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb896-1"><a href="model-diagnostics.html#cb896-1" aria-hidden="true" tabindex="-1"></a><span class="fu">bptest</span>(fit_1)</span></code></pre></div>
+<div class="sourceCode" id="cb896"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb896-1"><a href="model-diagnostics.html#cb896-1" tabindex="-1"></a><span class="fu">bptest</span>(fit_1)</span></code></pre></div>
 <pre><code>## 
 ##  studentized Breusch-Pagan test
 ## 
 ## data:  fit_1
 ## BP = 1.0234, df = 1, p-value = 0.3117</code></pre>
 <p>For <code>fit_1</code> we see a large p-value, so we do not reject the null of homoscedasticity, which is what we would expect.</p>
-<div class="sourceCode" id="cb898"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb898-1"><a href="model-diagnostics.html#cb898-1" aria-hidden="true" tabindex="-1"></a><span class="fu">bptest</span>(fit_2)</span></code></pre></div>
+<div class="sourceCode" id="cb898"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb898-1"><a href="model-diagnostics.html#cb898-1" tabindex="-1"></a><span class="fu">bptest</span>(fit_2)</span></code></pre></div>
 <pre><code>## 
 ##  studentized Breusch-Pagan test
 ## 
 ## data:  fit_2
 ## BP = 76.693, df = 1, p-value &lt; 2.2e-16</code></pre>
 <p>For <code>fit_2</code> we see a small p-value, so we reject the null of homoscedasticity. The constant variance assumption is violated. This matches our findings with a fitted versus residuals plot.</p>
-<div class="sourceCode" id="cb900"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb900-1"><a href="model-diagnostics.html#cb900-1" aria-hidden="true" tabindex="-1"></a><span class="fu">bptest</span>(fit_3)</span></code></pre></div>
+<div class="sourceCode" id="cb900"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb900-1"><a href="model-diagnostics.html#cb900-1" tabindex="-1"></a><span class="fu">bptest</span>(fit_3)</span></code></pre></div>
 <pre><code>## 
 ##  studentized Breusch-Pagan test
 ## 
@@ -679,25 +679,25 @@ <h3><span class="header-section-number">13.2.2</span> Breusch-Pagan Test<a href=
 <div id="histograms-1" class="section level3 hasAnchor" number="13.2.3">
 <h3><span class="header-section-number">13.2.3</span> Histograms<a href="model-diagnostics.html#histograms-1" class="anchor-section" aria-label="Anchor link to header"></a></h3>
 <p>We have a number of tools for assessing the normality assumption. The most obvious would be to make a histogram of the residuals. If it appears roughly normal, then we’ll believe the errors could truly be normal.</p>
-<div class="sourceCode" id="cb902"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb902-1"><a href="model-diagnostics.html#cb902-1" aria-hidden="true" tabindex="-1"></a><span class="fu">par</span>(<span class="at">mfrow =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">3</span>))</span>
-<span id="cb902-2"><a href="model-diagnostics.html#cb902-2" aria-hidden="true" tabindex="-1"></a><span class="fu">hist</span>(<span class="fu">resid</span>(fit_1),</span>
-<span id="cb902-3"><a href="model-diagnostics.html#cb902-3" aria-hidden="true" tabindex="-1"></a>     <span class="at">xlab   =</span> <span class="st">&quot;Residuals&quot;</span>,</span>
-<span id="cb902-4"><a href="model-diagnostics.html#cb902-4" aria-hidden="true" tabindex="-1"></a>     <span class="at">main   =</span> <span class="st">&quot;Histogram of Residuals, fit_1&quot;</span>,</span>
-<span id="cb902-5"><a href="model-diagnostics.html#cb902-5" aria-hidden="true" tabindex="-1"></a>     <span class="at">col    =</span> <span class="st">&quot;darkorange&quot;</span>,</span>
-<span id="cb902-6"><a href="model-diagnostics.html#cb902-6" aria-hidden="true" tabindex="-1"></a>     <span class="at">border =</span> <span class="st">&quot;dodgerblue&quot;</span>,</span>
-<span id="cb902-7"><a href="model-diagnostics.html#cb902-7" aria-hidden="true" tabindex="-1"></a>     <span class="at">breaks =</span> <span class="dv">20</span>)</span>
-<span id="cb902-8"><a href="model-diagnostics.html#cb902-8" aria-hidden="true" tabindex="-1"></a><span class="fu">hist</span>(<span class="fu">resid</span>(fit_2),</span>
-<span id="cb902-9"><a href="model-diagnostics.html#cb902-9" aria-hidden="true" tabindex="-1"></a>     <span class="at">xlab   =</span> <span class="st">&quot;Residuals&quot;</span>,</span>
-<span id="cb902-10"><a href="model-diagnostics.html#cb902-10" aria-hidden="true" tabindex="-1"></a>     <span class="at">main   =</span> <span class="st">&quot;Histogram of Residuals, fit_2&quot;</span>,</span>
-<span id="cb902-11"><a href="model-diagnostics.html#cb902-11" aria-hidden="true" tabindex="-1"></a>     <span class="at">col    =</span> <span class="st">&quot;darkorange&quot;</span>,</span>
-<span id="cb902-12"><a href="model-diagnostics.html#cb902-12" aria-hidden="true" tabindex="-1"></a>     <span class="at">border =</span> <span class="st">&quot;dodgerblue&quot;</span>,</span>
-<span id="cb902-13"><a href="model-diagnostics.html#cb902-13" aria-hidden="true" tabindex="-1"></a>     <span class="at">breaks =</span> <span class="dv">20</span>)</span>
-<span id="cb902-14"><a href="model-diagnostics.html#cb902-14" aria-hidden="true" tabindex="-1"></a><span class="fu">hist</span>(<span class="fu">resid</span>(fit_3),</span>
-<span id="cb902-15"><a href="model-diagnostics.html#cb902-15" aria-hidden="true" tabindex="-1"></a>     <span class="at">xlab   =</span> <span class="st">&quot;Residuals&quot;</span>,</span>
-<span id="cb902-16"><a href="model-diagnostics.html#cb902-16" aria-hidden="true" tabindex="-1"></a>     <span class="at">main   =</span> <span class="st">&quot;Histogram of Residuals, fit_3&quot;</span>,</span>
-<span id="cb902-17"><a href="model-diagnostics.html#cb902-17" aria-hidden="true" tabindex="-1"></a>     <span class="at">col    =</span> <span class="st">&quot;darkorange&quot;</span>,</span>
-<span id="cb902-18"><a href="model-diagnostics.html#cb902-18" aria-hidden="true" tabindex="-1"></a>     <span class="at">border =</span> <span class="st">&quot;dodgerblue&quot;</span>,</span>
-<span id="cb902-19"><a href="model-diagnostics.html#cb902-19" aria-hidden="true" tabindex="-1"></a>     <span class="at">breaks =</span> <span class="dv">20</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb902"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb902-1"><a href="model-diagnostics.html#cb902-1" tabindex="-1"></a><span class="fu">par</span>(<span class="at">mfrow =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">3</span>))</span>
+<span id="cb902-2"><a href="model-diagnostics.html#cb902-2" tabindex="-1"></a><span class="fu">hist</span>(<span class="fu">resid</span>(fit_1),</span>
+<span id="cb902-3"><a href="model-diagnostics.html#cb902-3" tabindex="-1"></a>     <span class="at">xlab   =</span> <span class="st">&quot;Residuals&quot;</span>,</span>
+<span id="cb902-4"><a href="model-diagnostics.html#cb902-4" tabindex="-1"></a>     <span class="at">main   =</span> <span class="st">&quot;Histogram of Residuals, fit_1&quot;</span>,</span>
+<span id="cb902-5"><a href="model-diagnostics.html#cb902-5" tabindex="-1"></a>     <span class="at">col    =</span> <span class="st">&quot;darkorange&quot;</span>,</span>
+<span id="cb902-6"><a href="model-diagnostics.html#cb902-6" tabindex="-1"></a>     <span class="at">border =</span> <span class="st">&quot;dodgerblue&quot;</span>,</span>
+<span id="cb902-7"><a href="model-diagnostics.html#cb902-7" tabindex="-1"></a>     <span class="at">breaks =</span> <span class="dv">20</span>)</span>
+<span id="cb902-8"><a href="model-diagnostics.html#cb902-8" tabindex="-1"></a><span class="fu">hist</span>(<span class="fu">resid</span>(fit_2),</span>
+<span id="cb902-9"><a href="model-diagnostics.html#cb902-9" tabindex="-1"></a>     <span class="at">xlab   =</span> <span class="st">&quot;Residuals&quot;</span>,</span>
+<span id="cb902-10"><a href="model-diagnostics.html#cb902-10" tabindex="-1"></a>     <span class="at">main   =</span> <span class="st">&quot;Histogram of Residuals, fit_2&quot;</span>,</span>
+<span id="cb902-11"><a href="model-diagnostics.html#cb902-11" tabindex="-1"></a>     <span class="at">col    =</span> <span class="st">&quot;darkorange&quot;</span>,</span>
+<span id="cb902-12"><a href="model-diagnostics.html#cb902-12" tabindex="-1"></a>     <span class="at">border =</span> <span class="st">&quot;dodgerblue&quot;</span>,</span>
+<span id="cb902-13"><a href="model-diagnostics.html#cb902-13" tabindex="-1"></a>     <span class="at">breaks =</span> <span class="dv">20</span>)</span>
+<span id="cb902-14"><a href="model-diagnostics.html#cb902-14" tabindex="-1"></a><span class="fu">hist</span>(<span class="fu">resid</span>(fit_3),</span>
+<span id="cb902-15"><a href="model-diagnostics.html#cb902-15" tabindex="-1"></a>     <span class="at">xlab   =</span> <span class="st">&quot;Residuals&quot;</span>,</span>
+<span id="cb902-16"><a href="model-diagnostics.html#cb902-16" tabindex="-1"></a>     <span class="at">main   =</span> <span class="st">&quot;Histogram of Residuals, fit_3&quot;</span>,</span>
+<span id="cb902-17"><a href="model-diagnostics.html#cb902-17" tabindex="-1"></a>     <span class="at">col    =</span> <span class="st">&quot;darkorange&quot;</span>,</span>
+<span id="cb902-18"><a href="model-diagnostics.html#cb902-18" tabindex="-1"></a>     <span class="at">border =</span> <span class="st">&quot;dodgerblue&quot;</span>,</span>
+<span id="cb902-19"><a href="model-diagnostics.html#cb902-19" tabindex="-1"></a>     <span class="at">breaks =</span> <span class="dv">20</span>)</span></code></pre></div>
 <p><img src="diagnostics_files/figure-html/unnamed-chunk-14-1.png" width="1440" style="display: block; margin: auto;" /></p>
 <p>Above are histograms for each of the three regression models we have been considering. Notice that the first, for <code>fit_1</code> appears very normal. The third, for <code>fit_3</code>, appears to be very non-normal. However <code>fit_2</code> is not as clear. It does have a rough bell shape, however, it also has a very sharp peak. For this reason we will usually use more powerful tools such as <strong>Q-Q plots</strong> and the <strong>Shapiro-Wilk test</strong> for assessing the normality of errors.</p>
 </div>
@@ -705,64 +705,64 @@ <h3><span class="header-section-number">13.2.3</span> Histograms<a href="model-d
 <h3><span class="header-section-number">13.2.4</span> Q-Q Plots<a href="model-diagnostics.html#q-q-plots" class="anchor-section" aria-label="Anchor link to header"></a></h3>
 <p>Another visual method for assessing the normality of errors, which is more powerful than a histogram, is a normal quantile-quantile plot, or <strong>Q-Q plot</strong> for short.</p>
 <p>In <code>R</code> these are very easy to make. The <code>qqnorm()</code> function plots the points, and the <code>qqline()</code> function adds the necessary line. We create a Q-Q plot for the residuals of <code>fit_1</code> to check if the errors could truly be normally distributed.</p>
-<div class="sourceCode" id="cb903"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb903-1"><a href="model-diagnostics.html#cb903-1" aria-hidden="true" tabindex="-1"></a><span class="fu">qqnorm</span>(<span class="fu">resid</span>(fit_1), <span class="at">main =</span> <span class="st">&quot;Normal Q-Q Plot, fit_1&quot;</span>, <span class="at">col =</span> <span class="st">&quot;darkgrey&quot;</span>)</span>
-<span id="cb903-2"><a href="model-diagnostics.html#cb903-2" aria-hidden="true" tabindex="-1"></a><span class="fu">qqline</span>(<span class="fu">resid</span>(fit_1), <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb903"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb903-1"><a href="model-diagnostics.html#cb903-1" tabindex="-1"></a><span class="fu">qqnorm</span>(<span class="fu">resid</span>(fit_1), <span class="at">main =</span> <span class="st">&quot;Normal Q-Q Plot, fit_1&quot;</span>, <span class="at">col =</span> <span class="st">&quot;darkgrey&quot;</span>)</span>
+<span id="cb903-2"><a href="model-diagnostics.html#cb903-2" tabindex="-1"></a><span class="fu">qqline</span>(<span class="fu">resid</span>(fit_1), <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
 <p><img src="diagnostics_files/figure-html/unnamed-chunk-15-1.png" width="672" style="display: block; margin: auto;" /></p>
 <p>In short, if the points of the plot do not closely follow a straight line, this would suggest that the data do not come from a normal distribution.</p>
 <p>The calculations required to create the plot vary depending on the implementation, but essentially the <span class="math inline">\(y\)</span>-axis is the sorted data (observed, or sample quantiles), and the <span class="math inline">\(x\)</span>-axis is the values we would expect if the data did come from a normal distribution (theoretical quantiles).</p>
 <p>The <a href="http://en.wikipedia.org/wiki/Normal_probability_plot" target="_blank">Wikipedia page for Normal probability plots</a> gives details on how this is implemented in <code>R</code> if you are interested.</p>
 <p>Also, to get a better idea of how Q-Q plots work, here is a quick function which creates a Q-Q plot:</p>
-<div class="sourceCode" id="cb904"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb904-1"><a href="model-diagnostics.html#cb904-1" aria-hidden="true" tabindex="-1"></a>qq_plot <span class="ot">=</span> <span class="cf">function</span>(e) {</span>
-<span id="cb904-2"><a href="model-diagnostics.html#cb904-2" aria-hidden="true" tabindex="-1"></a></span>
-<span id="cb904-3"><a href="model-diagnostics.html#cb904-3" aria-hidden="true" tabindex="-1"></a>  n <span class="ot">=</span> <span class="fu">length</span>(e)</span>
-<span id="cb904-4"><a href="model-diagnostics.html#cb904-4" aria-hidden="true" tabindex="-1"></a>  normal_quantiles <span class="ot">=</span> <span class="fu">qnorm</span>(((<span class="dv">1</span><span class="sc">:</span>n <span class="sc">-</span> <span class="fl">0.5</span>) <span class="sc">/</span> n))</span>
-<span id="cb904-5"><a href="model-diagnostics.html#cb904-5" aria-hidden="true" tabindex="-1"></a>  <span class="co"># normal_quantiles = qnorm(((1:n) / (n + 1)))</span></span>
-<span id="cb904-6"><a href="model-diagnostics.html#cb904-6" aria-hidden="true" tabindex="-1"></a></span>
-<span id="cb904-7"><a href="model-diagnostics.html#cb904-7" aria-hidden="true" tabindex="-1"></a>  <span class="co"># plot theoretical verus observed quantiles</span></span>
-<span id="cb904-8"><a href="model-diagnostics.html#cb904-8" aria-hidden="true" tabindex="-1"></a>  <span class="fu">plot</span>(normal_quantiles, <span class="fu">sort</span>(e),</span>
-<span id="cb904-9"><a href="model-diagnostics.html#cb904-9" aria-hidden="true" tabindex="-1"></a>       <span class="at">xlab =</span> <span class="fu">c</span>(<span class="st">&quot;Theoretical Quantiles&quot;</span>),</span>
-<span id="cb904-10"><a href="model-diagnostics.html#cb904-10" aria-hidden="true" tabindex="-1"></a>       <span class="at">ylab =</span> <span class="fu">c</span>(<span class="st">&quot;Sample Quantiles&quot;</span>),</span>
-<span id="cb904-11"><a href="model-diagnostics.html#cb904-11" aria-hidden="true" tabindex="-1"></a>       <span class="at">col =</span> <span class="st">&quot;darkgrey&quot;</span>)</span>
-<span id="cb904-12"><a href="model-diagnostics.html#cb904-12" aria-hidden="true" tabindex="-1"></a>  <span class="fu">title</span>(<span class="st">&quot;Normal Q-Q Plot&quot;</span>)</span>
-<span id="cb904-13"><a href="model-diagnostics.html#cb904-13" aria-hidden="true" tabindex="-1"></a></span>
-<span id="cb904-14"><a href="model-diagnostics.html#cb904-14" aria-hidden="true" tabindex="-1"></a>  <span class="co"># calculate line through the first and third quartiles</span></span>
-<span id="cb904-15"><a href="model-diagnostics.html#cb904-15" aria-hidden="true" tabindex="-1"></a>  slope     <span class="ot">=</span> (<span class="fu">quantile</span>(e, <span class="fl">0.75</span>) <span class="sc">-</span> <span class="fu">quantile</span>(e, <span class="fl">0.25</span>)) <span class="sc">/</span> (<span class="fu">qnorm</span>(<span class="fl">0.75</span>) <span class="sc">-</span> <span class="fu">qnorm</span>(<span class="fl">0.25</span>))</span>
-<span id="cb904-16"><a href="model-diagnostics.html#cb904-16" aria-hidden="true" tabindex="-1"></a>  intercept <span class="ot">=</span> <span class="fu">quantile</span>(e, <span class="fl">0.25</span>) <span class="sc">-</span> slope <span class="sc">*</span> <span class="fu">qnorm</span>(<span class="fl">0.25</span>)</span>
-<span id="cb904-17"><a href="model-diagnostics.html#cb904-17" aria-hidden="true" tabindex="-1"></a></span>
-<span id="cb904-18"><a href="model-diagnostics.html#cb904-18" aria-hidden="true" tabindex="-1"></a>  <span class="co"># add to existing plot</span></span>
-<span id="cb904-19"><a href="model-diagnostics.html#cb904-19" aria-hidden="true" tabindex="-1"></a>  <span class="fu">abline</span>(intercept, slope, <span class="at">lty =</span> <span class="dv">2</span>, <span class="at">lwd =</span> <span class="dv">2</span>, <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>)</span>
-<span id="cb904-20"><a href="model-diagnostics.html#cb904-20" aria-hidden="true" tabindex="-1"></a>}</span></code></pre></div>
+<div class="sourceCode" id="cb904"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb904-1"><a href="model-diagnostics.html#cb904-1" tabindex="-1"></a>qq_plot <span class="ot">=</span> <span class="cf">function</span>(e) {</span>
+<span id="cb904-2"><a href="model-diagnostics.html#cb904-2" tabindex="-1"></a></span>
+<span id="cb904-3"><a href="model-diagnostics.html#cb904-3" tabindex="-1"></a>  n <span class="ot">=</span> <span class="fu">length</span>(e)</span>
+<span id="cb904-4"><a href="model-diagnostics.html#cb904-4" tabindex="-1"></a>  normal_quantiles <span class="ot">=</span> <span class="fu">qnorm</span>(((<span class="dv">1</span><span class="sc">:</span>n <span class="sc">-</span> <span class="fl">0.5</span>) <span class="sc">/</span> n))</span>
+<span id="cb904-5"><a href="model-diagnostics.html#cb904-5" tabindex="-1"></a>  <span class="co"># normal_quantiles = qnorm(((1:n) / (n + 1)))</span></span>
+<span id="cb904-6"><a href="model-diagnostics.html#cb904-6" tabindex="-1"></a></span>
+<span id="cb904-7"><a href="model-diagnostics.html#cb904-7" tabindex="-1"></a>  <span class="co"># plot theoretical verus observed quantiles</span></span>
+<span id="cb904-8"><a href="model-diagnostics.html#cb904-8" tabindex="-1"></a>  <span class="fu">plot</span>(normal_quantiles, <span class="fu">sort</span>(e),</span>
+<span id="cb904-9"><a href="model-diagnostics.html#cb904-9" tabindex="-1"></a>       <span class="at">xlab =</span> <span class="fu">c</span>(<span class="st">&quot;Theoretical Quantiles&quot;</span>),</span>
+<span id="cb904-10"><a href="model-diagnostics.html#cb904-10" tabindex="-1"></a>       <span class="at">ylab =</span> <span class="fu">c</span>(<span class="st">&quot;Sample Quantiles&quot;</span>),</span>
+<span id="cb904-11"><a href="model-diagnostics.html#cb904-11" tabindex="-1"></a>       <span class="at">col =</span> <span class="st">&quot;darkgrey&quot;</span>)</span>
+<span id="cb904-12"><a href="model-diagnostics.html#cb904-12" tabindex="-1"></a>  <span class="fu">title</span>(<span class="st">&quot;Normal Q-Q Plot&quot;</span>)</span>
+<span id="cb904-13"><a href="model-diagnostics.html#cb904-13" tabindex="-1"></a></span>
+<span id="cb904-14"><a href="model-diagnostics.html#cb904-14" tabindex="-1"></a>  <span class="co"># calculate line through the first and third quartiles</span></span>
+<span id="cb904-15"><a href="model-diagnostics.html#cb904-15" tabindex="-1"></a>  slope     <span class="ot">=</span> (<span class="fu">quantile</span>(e, <span class="fl">0.75</span>) <span class="sc">-</span> <span class="fu">quantile</span>(e, <span class="fl">0.25</span>)) <span class="sc">/</span> (<span class="fu">qnorm</span>(<span class="fl">0.75</span>) <span class="sc">-</span> <span class="fu">qnorm</span>(<span class="fl">0.25</span>))</span>
+<span id="cb904-16"><a href="model-diagnostics.html#cb904-16" tabindex="-1"></a>  intercept <span class="ot">=</span> <span class="fu">quantile</span>(e, <span class="fl">0.25</span>) <span class="sc">-</span> slope <span class="sc">*</span> <span class="fu">qnorm</span>(<span class="fl">0.25</span>)</span>
+<span id="cb904-17"><a href="model-diagnostics.html#cb904-17" tabindex="-1"></a></span>
+<span id="cb904-18"><a href="model-diagnostics.html#cb904-18" tabindex="-1"></a>  <span class="co"># add to existing plot</span></span>
+<span id="cb904-19"><a href="model-diagnostics.html#cb904-19" tabindex="-1"></a>  <span class="fu">abline</span>(intercept, slope, <span class="at">lty =</span> <span class="dv">2</span>, <span class="at">lwd =</span> <span class="dv">2</span>, <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>)</span>
+<span id="cb904-20"><a href="model-diagnostics.html#cb904-20" tabindex="-1"></a>}</span></code></pre></div>
 <p>We can then verify that it is essentially equivalent to using <code>qqnorm()</code> and <code>qqline()</code> in <code>R</code>.</p>
-<div class="sourceCode" id="cb905"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb905-1"><a href="model-diagnostics.html#cb905-1" aria-hidden="true" tabindex="-1"></a><span class="fu">set.seed</span>(<span class="dv">420</span>)</span>
-<span id="cb905-2"><a href="model-diagnostics.html#cb905-2" aria-hidden="true" tabindex="-1"></a>x <span class="ot">=</span> <span class="fu">rnorm</span>(<span class="dv">100</span>, <span class="at">mean =</span> <span class="dv">0</span> , <span class="at">sd =</span> <span class="dv">1</span>)</span>
-<span id="cb905-3"><a href="model-diagnostics.html#cb905-3" aria-hidden="true" tabindex="-1"></a><span class="fu">par</span>(<span class="at">mfrow =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>))</span>
-<span id="cb905-4"><a href="model-diagnostics.html#cb905-4" aria-hidden="true" tabindex="-1"></a><span class="fu">qqnorm</span>(x, <span class="at">col =</span> <span class="st">&quot;darkgrey&quot;</span>)</span>
-<span id="cb905-5"><a href="model-diagnostics.html#cb905-5" aria-hidden="true" tabindex="-1"></a><span class="fu">qqline</span>(x, <span class="at">lty =</span> <span class="dv">2</span>, <span class="at">lwd =</span> <span class="dv">2</span>, <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>)</span>
-<span id="cb905-6"><a href="model-diagnostics.html#cb905-6" aria-hidden="true" tabindex="-1"></a><span class="fu">qq_plot</span>(x)</span></code></pre></div>
+<div class="sourceCode" id="cb905"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb905-1"><a href="model-diagnostics.html#cb905-1" tabindex="-1"></a><span class="fu">set.seed</span>(<span class="dv">420</span>)</span>
+<span id="cb905-2"><a href="model-diagnostics.html#cb905-2" tabindex="-1"></a>x <span class="ot">=</span> <span class="fu">rnorm</span>(<span class="dv">100</span>, <span class="at">mean =</span> <span class="dv">0</span> , <span class="at">sd =</span> <span class="dv">1</span>)</span>
+<span id="cb905-3"><a href="model-diagnostics.html#cb905-3" tabindex="-1"></a><span class="fu">par</span>(<span class="at">mfrow =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>))</span>
+<span id="cb905-4"><a href="model-diagnostics.html#cb905-4" tabindex="-1"></a><span class="fu">qqnorm</span>(x, <span class="at">col =</span> <span class="st">&quot;darkgrey&quot;</span>)</span>
+<span id="cb905-5"><a href="model-diagnostics.html#cb905-5" tabindex="-1"></a><span class="fu">qqline</span>(x, <span class="at">lty =</span> <span class="dv">2</span>, <span class="at">lwd =</span> <span class="dv">2</span>, <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>)</span>
+<span id="cb905-6"><a href="model-diagnostics.html#cb905-6" tabindex="-1"></a><span class="fu">qq_plot</span>(x)</span></code></pre></div>
 <p><img src="diagnostics_files/figure-html/unnamed-chunk-17-1.png" width="960" style="display: block; margin: auto;" /></p>
 <p>To get a better idea of what “close to the line” means, we perform a number of simulations, and create Q-Q plots.</p>
 <p>First we simulate data from a normal distribution with different sample sizes, and each time create a Q-Q plot.</p>
-<div class="sourceCode" id="cb906"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb906-1"><a href="model-diagnostics.html#cb906-1" aria-hidden="true" tabindex="-1"></a><span class="fu">par</span>(<span class="at">mfrow =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">3</span>))</span>
-<span id="cb906-2"><a href="model-diagnostics.html#cb906-2" aria-hidden="true" tabindex="-1"></a><span class="fu">set.seed</span>(<span class="dv">420</span>)</span>
-<span id="cb906-3"><a href="model-diagnostics.html#cb906-3" aria-hidden="true" tabindex="-1"></a><span class="fu">qq_plot</span>(<span class="fu">rnorm</span>(<span class="dv">10</span>))</span>
-<span id="cb906-4"><a href="model-diagnostics.html#cb906-4" aria-hidden="true" tabindex="-1"></a><span class="fu">qq_plot</span>(<span class="fu">rnorm</span>(<span class="dv">25</span>))</span>
-<span id="cb906-5"><a href="model-diagnostics.html#cb906-5" aria-hidden="true" tabindex="-1"></a><span class="fu">qq_plot</span>(<span class="fu">rnorm</span>(<span class="dv">100</span>))</span></code></pre></div>
+<div class="sourceCode" id="cb906"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb906-1"><a href="model-diagnostics.html#cb906-1" tabindex="-1"></a><span class="fu">par</span>(<span class="at">mfrow =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">3</span>))</span>
+<span id="cb906-2"><a href="model-diagnostics.html#cb906-2" tabindex="-1"></a><span class="fu">set.seed</span>(<span class="dv">420</span>)</span>
+<span id="cb906-3"><a href="model-diagnostics.html#cb906-3" tabindex="-1"></a><span class="fu">qq_plot</span>(<span class="fu">rnorm</span>(<span class="dv">10</span>))</span>
+<span id="cb906-4"><a href="model-diagnostics.html#cb906-4" tabindex="-1"></a><span class="fu">qq_plot</span>(<span class="fu">rnorm</span>(<span class="dv">25</span>))</span>
+<span id="cb906-5"><a href="model-diagnostics.html#cb906-5" tabindex="-1"></a><span class="fu">qq_plot</span>(<span class="fu">rnorm</span>(<span class="dv">100</span>))</span></code></pre></div>
 <p><img src="diagnostics_files/figure-html/unnamed-chunk-18-1.png" width="1152" style="display: block; margin: auto;" /></p>
 <p>Since this data <strong>is</strong> sampled from a normal distribution, these are all, by definition, good Q-Q plots. The points are “close to the line” and we would conclude that this data could have been sampled from a normal distribution. Notice in the first plot, one point is <em>somewhat</em> far from the line, but just one point, in combination with the small sample size, is not enough to make us worried. We see with the large sample size, all of the points are rather close to the line.</p>
 <p>Next, we simulate data from a <span class="math inline">\(t\)</span> distribution with a small degrees of freedom, for different sample sizes.</p>
-<div class="sourceCode" id="cb907"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb907-1"><a href="model-diagnostics.html#cb907-1" aria-hidden="true" tabindex="-1"></a><span class="fu">par</span>(<span class="at">mfrow =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">3</span>))</span>
-<span id="cb907-2"><a href="model-diagnostics.html#cb907-2" aria-hidden="true" tabindex="-1"></a><span class="fu">set.seed</span>(<span class="dv">420</span>)</span>
-<span id="cb907-3"><a href="model-diagnostics.html#cb907-3" aria-hidden="true" tabindex="-1"></a><span class="fu">qq_plot</span>(<span class="fu">rt</span>(<span class="dv">10</span>, <span class="at">df =</span> <span class="dv">4</span>))</span>
-<span id="cb907-4"><a href="model-diagnostics.html#cb907-4" aria-hidden="true" tabindex="-1"></a><span class="fu">qq_plot</span>(<span class="fu">rt</span>(<span class="dv">25</span>, <span class="at">df =</span> <span class="dv">4</span>))</span>
-<span id="cb907-5"><a href="model-diagnostics.html#cb907-5" aria-hidden="true" tabindex="-1"></a><span class="fu">qq_plot</span>(<span class="fu">rt</span>(<span class="dv">100</span>, <span class="at">df =</span> <span class="dv">4</span>))</span></code></pre></div>
+<div class="sourceCode" id="cb907"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb907-1"><a href="model-diagnostics.html#cb907-1" tabindex="-1"></a><span class="fu">par</span>(<span class="at">mfrow =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">3</span>))</span>
+<span id="cb907-2"><a href="model-diagnostics.html#cb907-2" tabindex="-1"></a><span class="fu">set.seed</span>(<span class="dv">420</span>)</span>
+<span id="cb907-3"><a href="model-diagnostics.html#cb907-3" tabindex="-1"></a><span class="fu">qq_plot</span>(<span class="fu">rt</span>(<span class="dv">10</span>, <span class="at">df =</span> <span class="dv">4</span>))</span>
+<span id="cb907-4"><a href="model-diagnostics.html#cb907-4" tabindex="-1"></a><span class="fu">qq_plot</span>(<span class="fu">rt</span>(<span class="dv">25</span>, <span class="at">df =</span> <span class="dv">4</span>))</span>
+<span id="cb907-5"><a href="model-diagnostics.html#cb907-5" tabindex="-1"></a><span class="fu">qq_plot</span>(<span class="fu">rt</span>(<span class="dv">100</span>, <span class="at">df =</span> <span class="dv">4</span>))</span></code></pre></div>
 <p><img src="diagnostics_files/figure-html/unnamed-chunk-19-1.png" width="1152" style="display: block; margin: auto;" /></p>
 <p>Recall that as the degrees of freedom for a <span class="math inline">\(t\)</span> distribution become larger, the distribution becomes more and more similar to a normal. Here, using 4 degrees of freedom, we have a distribution that is somewhat normal, it is symmetrical and roughly bell-shaped, however it has “fat tails.” This presents itself clearly in the third panel. While many of the points are close to the line, at the edges, there are large discrepancies. This indicates that the values are too small (negative) or too large (positive) compared to what we would expect for a normal distribution. So for the sample size of <code>100</code>, we would conclude that the normality assumption is violated. (If these were residuals of a model.) For sample sizes of <code>10</code> and <code>25</code> we may be suspicious, but not entirely confident. Reading Q-Q plots, is a bit of an art, not completely a science.</p>
 <p>Next, we simulate data from an exponential distribution.</p>
-<div class="sourceCode" id="cb908"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb908-1"><a href="model-diagnostics.html#cb908-1" aria-hidden="true" tabindex="-1"></a><span class="fu">par</span>(<span class="at">mfrow =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">3</span>))</span>
-<span id="cb908-2"><a href="model-diagnostics.html#cb908-2" aria-hidden="true" tabindex="-1"></a><span class="fu">set.seed</span>(<span class="dv">420</span>)</span>
-<span id="cb908-3"><a href="model-diagnostics.html#cb908-3" aria-hidden="true" tabindex="-1"></a><span class="fu">qq_plot</span>(<span class="fu">rexp</span>(<span class="dv">10</span>))</span>
-<span id="cb908-4"><a href="model-diagnostics.html#cb908-4" aria-hidden="true" tabindex="-1"></a><span class="fu">qq_plot</span>(<span class="fu">rexp</span>(<span class="dv">25</span>))</span>
-<span id="cb908-5"><a href="model-diagnostics.html#cb908-5" aria-hidden="true" tabindex="-1"></a><span class="fu">qq_plot</span>(<span class="fu">rexp</span>(<span class="dv">100</span>))</span></code></pre></div>
+<div class="sourceCode" id="cb908"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb908-1"><a href="model-diagnostics.html#cb908-1" tabindex="-1"></a><span class="fu">par</span>(<span class="at">mfrow =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">3</span>))</span>
+<span id="cb908-2"><a href="model-diagnostics.html#cb908-2" tabindex="-1"></a><span class="fu">set.seed</span>(<span class="dv">420</span>)</span>
+<span id="cb908-3"><a href="model-diagnostics.html#cb908-3" tabindex="-1"></a><span class="fu">qq_plot</span>(<span class="fu">rexp</span>(<span class="dv">10</span>))</span>
+<span id="cb908-4"><a href="model-diagnostics.html#cb908-4" tabindex="-1"></a><span class="fu">qq_plot</span>(<span class="fu">rexp</span>(<span class="dv">25</span>))</span>
+<span id="cb908-5"><a href="model-diagnostics.html#cb908-5" tabindex="-1"></a><span class="fu">qq_plot</span>(<span class="fu">rexp</span>(<span class="dv">100</span>))</span></code></pre></div>
 <p><img src="diagnostics_files/figure-html/unnamed-chunk-20-1.png" width="1152" style="display: block; margin: auto;" /></p>
 <p>This is a distribution that is not very similar to a normal, so in all three cases, we see points that are far from the lines, so we would think that the normality assumption is violated.</p>
 <p>For a better understanding of which Q-Q plots are “good,” repeat the simulations above a number of times (without setting the seed) and pay attention to the differences between those that are simulated from normal, and those that are not. Also consider different sample sizes and distribution parameters.</p>
@@ -773,30 +773,30 @@ <h3><span class="header-section-number">13.2.4</span> Q-Q Plots<a href="model-di
 <li><code>fit_3</code> violated linearity, but not constant variance.</li>
 </ul>
 <p>We’ll now create a Q-Q plot for each to assess normality of errors.</p>
-<div class="sourceCode" id="cb909"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb909-1"><a href="model-diagnostics.html#cb909-1" aria-hidden="true" tabindex="-1"></a><span class="fu">qqnorm</span>(<span class="fu">resid</span>(fit_1), <span class="at">main =</span> <span class="st">&quot;Normal Q-Q Plot, fit_1&quot;</span>, <span class="at">col =</span> <span class="st">&quot;darkgrey&quot;</span>)</span>
-<span id="cb909-2"><a href="model-diagnostics.html#cb909-2" aria-hidden="true" tabindex="-1"></a><span class="fu">qqline</span>(<span class="fu">resid</span>(fit_1), <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb909"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb909-1"><a href="model-diagnostics.html#cb909-1" tabindex="-1"></a><span class="fu">qqnorm</span>(<span class="fu">resid</span>(fit_1), <span class="at">main =</span> <span class="st">&quot;Normal Q-Q Plot, fit_1&quot;</span>, <span class="at">col =</span> <span class="st">&quot;darkgrey&quot;</span>)</span>
+<span id="cb909-2"><a href="model-diagnostics.html#cb909-2" tabindex="-1"></a><span class="fu">qqline</span>(<span class="fu">resid</span>(fit_1), <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
 <p><img src="diagnostics_files/figure-html/unnamed-chunk-21-1.png" width="672" style="display: block; margin: auto;" /></p>
 <p>For <code>fit_1</code>, we have a near perfect Q-Q plot. We would believe the errors follow a normal distribution.</p>
-<div class="sourceCode" id="cb910"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb910-1"><a href="model-diagnostics.html#cb910-1" aria-hidden="true" tabindex="-1"></a><span class="fu">qqnorm</span>(<span class="fu">resid</span>(fit_2), <span class="at">main =</span> <span class="st">&quot;Normal Q-Q Plot, fit_2&quot;</span>, <span class="at">col =</span> <span class="st">&quot;darkgrey&quot;</span>)</span>
-<span id="cb910-2"><a href="model-diagnostics.html#cb910-2" aria-hidden="true" tabindex="-1"></a><span class="fu">qqline</span>(<span class="fu">resid</span>(fit_2), <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb910"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb910-1"><a href="model-diagnostics.html#cb910-1" tabindex="-1"></a><span class="fu">qqnorm</span>(<span class="fu">resid</span>(fit_2), <span class="at">main =</span> <span class="st">&quot;Normal Q-Q Plot, fit_2&quot;</span>, <span class="at">col =</span> <span class="st">&quot;darkgrey&quot;</span>)</span>
+<span id="cb910-2"><a href="model-diagnostics.html#cb910-2" tabindex="-1"></a><span class="fu">qqline</span>(<span class="fu">resid</span>(fit_2), <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
 <p><img src="diagnostics_files/figure-html/unnamed-chunk-22-1.png" width="672" style="display: block; margin: auto;" /></p>
 <p>For <code>fit_2</code>, we have a suspect Q-Q plot. We would probably <strong>not</strong> believe the errors follow a normal distribution.</p>
-<div class="sourceCode" id="cb911"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb911-1"><a href="model-diagnostics.html#cb911-1" aria-hidden="true" tabindex="-1"></a><span class="fu">qqnorm</span>(<span class="fu">resid</span>(fit_3), <span class="at">main =</span> <span class="st">&quot;Normal Q-Q Plot, fit_3&quot;</span>, <span class="at">col =</span> <span class="st">&quot;darkgrey&quot;</span>)</span>
-<span id="cb911-2"><a href="model-diagnostics.html#cb911-2" aria-hidden="true" tabindex="-1"></a><span class="fu">qqline</span>(<span class="fu">resid</span>(fit_3), <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb911"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb911-1"><a href="model-diagnostics.html#cb911-1" tabindex="-1"></a><span class="fu">qqnorm</span>(<span class="fu">resid</span>(fit_3), <span class="at">main =</span> <span class="st">&quot;Normal Q-Q Plot, fit_3&quot;</span>, <span class="at">col =</span> <span class="st">&quot;darkgrey&quot;</span>)</span>
+<span id="cb911-2"><a href="model-diagnostics.html#cb911-2" tabindex="-1"></a><span class="fu">qqline</span>(<span class="fu">resid</span>(fit_3), <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
 <p><img src="diagnostics_files/figure-html/unnamed-chunk-23-1.png" width="672" style="display: block; margin: auto;" /></p>
 <p>Lastly, for <code>fit_3</code>, we again have a suspect Q-Q plot. We would probably <strong>not</strong> believe the errors follow a normal distribution.</p>
 </div>
 <div id="shapiro-wilk-test" class="section level3 hasAnchor" number="13.2.5">
 <h3><span class="header-section-number">13.2.5</span> Shapiro-Wilk Test<a href="model-diagnostics.html#shapiro-wilk-test" class="anchor-section" aria-label="Anchor link to header"></a></h3>
 <p>Histograms and Q-Q Plots give a nice visual representation of the residuals distribution, however if we are interested in formal testing, there are a number of options available. A commonly used test is the <strong>Shapiro–Wilk test</strong>, which is implemented in <code>R</code>.</p>
-<div class="sourceCode" id="cb912"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb912-1"><a href="model-diagnostics.html#cb912-1" aria-hidden="true" tabindex="-1"></a><span class="fu">set.seed</span>(<span class="dv">42</span>)</span>
-<span id="cb912-2"><a href="model-diagnostics.html#cb912-2" aria-hidden="true" tabindex="-1"></a><span class="fu">shapiro.test</span>(<span class="fu">rnorm</span>(<span class="dv">25</span>))</span></code></pre></div>
+<div class="sourceCode" id="cb912"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb912-1"><a href="model-diagnostics.html#cb912-1" tabindex="-1"></a><span class="fu">set.seed</span>(<span class="dv">42</span>)</span>
+<span id="cb912-2"><a href="model-diagnostics.html#cb912-2" tabindex="-1"></a><span class="fu">shapiro.test</span>(<span class="fu">rnorm</span>(<span class="dv">25</span>))</span></code></pre></div>
 <pre><code>## 
 ##  Shapiro-Wilk normality test
 ## 
 ## data:  rnorm(25)
 ## W = 0.9499, p-value = 0.2495</code></pre>
-<div class="sourceCode" id="cb914"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb914-1"><a href="model-diagnostics.html#cb914-1" aria-hidden="true" tabindex="-1"></a><span class="fu">shapiro.test</span>(<span class="fu">rexp</span>(<span class="dv">25</span>))</span></code></pre></div>
+<div class="sourceCode" id="cb914"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb914-1"><a href="model-diagnostics.html#cb914-1" tabindex="-1"></a><span class="fu">shapiro.test</span>(<span class="fu">rexp</span>(<span class="dv">25</span>))</span></code></pre></div>
 <pre><code>## 
 ##  Shapiro-Wilk normality test
 ## 
@@ -806,19 +806,19 @@ <h3><span class="header-section-number">13.2.5</span> Shapiro-Wilk Test<a href="
 <p>For details, see: <a href="https://en.wikipedia.org/wiki/Shapiro-Wilk_test" target="_blank">Wikipedia: Shapiro–Wilk test.</a></p>
 <p>In the above examples, we see we fail to reject for the data sampled from normal, and reject on the non-normal data, for any reasonable <span class="math inline">\(\alpha\)</span>.</p>
 <p>Returning again to <code>fit_1</code>, <code>fit_2</code>, and <code>fit_3</code>, we see the result of running <code>shapiro.test()</code> on the residuals of each, returns a result for each that matches decisions based on the Q-Q plots.</p>
-<div class="sourceCode" id="cb916"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb916-1"><a href="model-diagnostics.html#cb916-1" aria-hidden="true" tabindex="-1"></a><span class="fu">shapiro.test</span>(<span class="fu">resid</span>(fit_1))</span></code></pre></div>
+<div class="sourceCode" id="cb916"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb916-1"><a href="model-diagnostics.html#cb916-1" tabindex="-1"></a><span class="fu">shapiro.test</span>(<span class="fu">resid</span>(fit_1))</span></code></pre></div>
 <pre><code>## 
 ##  Shapiro-Wilk normality test
 ## 
 ## data:  resid(fit_1)
 ## W = 0.99858, p-value = 0.9622</code></pre>
-<div class="sourceCode" id="cb918"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb918-1"><a href="model-diagnostics.html#cb918-1" aria-hidden="true" tabindex="-1"></a><span class="fu">shapiro.test</span>(<span class="fu">resid</span>(fit_2))</span></code></pre></div>
+<div class="sourceCode" id="cb918"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb918-1"><a href="model-diagnostics.html#cb918-1" tabindex="-1"></a><span class="fu">shapiro.test</span>(<span class="fu">resid</span>(fit_2))</span></code></pre></div>
 <pre><code>## 
 ##  Shapiro-Wilk normality test
 ## 
 ## data:  resid(fit_2)
 ## W = 0.93697, p-value = 1.056e-13</code></pre>
-<div class="sourceCode" id="cb920"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb920-1"><a href="model-diagnostics.html#cb920-1" aria-hidden="true" tabindex="-1"></a><span class="fu">shapiro.test</span>(<span class="fu">resid</span>(fit_3))</span></code></pre></div>
+<div class="sourceCode" id="cb920"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb920-1"><a href="model-diagnostics.html#cb920-1" tabindex="-1"></a><span class="fu">shapiro.test</span>(<span class="fu">resid</span>(fit_3))</span></code></pre></div>
 <pre><code>## 
 ##  Shapiro-Wilk normality test
 ## 
@@ -830,65 +830,65 @@ <h3><span class="header-section-number">13.2.5</span> Shapiro-Wilk Test<a href="
 <h2><span class="header-section-number">13.3</span> Unusual Observations<a href="model-diagnostics.html#unusual-observations" class="anchor-section" aria-label="Anchor link to header"></a></h2>
 <p>In addition to checking the assumptions of regression, we also look for any “unusual observations” in the data. Often a small number of data points can have an extremely large influence on a regression, sometimes so much so that the regression assumptions are violated as a result of these points.</p>
 <p>The following three plots are inspired by an example from <a href="http://www.maths.bath.ac.uk/~jjf23/LMR/" target="_blank">Linear Models with R</a>.</p>
-<div class="sourceCode" id="cb922"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb922-1"><a href="model-diagnostics.html#cb922-1" aria-hidden="true" tabindex="-1"></a><span class="fu">par</span>(<span class="at">mfrow =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">3</span>))</span>
-<span id="cb922-2"><a href="model-diagnostics.html#cb922-2" aria-hidden="true" tabindex="-1"></a><span class="fu">set.seed</span>(<span class="dv">42</span>)</span>
-<span id="cb922-3"><a href="model-diagnostics.html#cb922-3" aria-hidden="true" tabindex="-1"></a>ex_data  <span class="ot">=</span> <span class="fu">data.frame</span>(<span class="at">x =</span> <span class="dv">1</span><span class="sc">:</span><span class="dv">10</span>,</span>
-<span id="cb922-4"><a href="model-diagnostics.html#cb922-4" aria-hidden="true" tabindex="-1"></a>                      <span class="at">y =</span> <span class="dv">10</span><span class="sc">:</span><span class="dv">1</span> <span class="sc">+</span> <span class="fu">rnorm</span>(<span class="at">n =</span> <span class="dv">10</span>))</span>
-<span id="cb922-5"><a href="model-diagnostics.html#cb922-5" aria-hidden="true" tabindex="-1"></a>ex_model <span class="ot">=</span> <span class="fu">lm</span>(y <span class="sc">~</span> x, <span class="at">data =</span> ex_data)</span>
-<span id="cb922-6"><a href="model-diagnostics.html#cb922-6" aria-hidden="true" tabindex="-1"></a></span>
-<span id="cb922-7"><a href="model-diagnostics.html#cb922-7" aria-hidden="true" tabindex="-1"></a><span class="co"># low leverage, large residual, small influence</span></span>
-<span id="cb922-8"><a href="model-diagnostics.html#cb922-8" aria-hidden="true" tabindex="-1"></a>point_1 <span class="ot">=</span> <span class="fu">c</span>(<span class="fl">5.4</span>, <span class="dv">11</span>)</span>
-<span id="cb922-9"><a href="model-diagnostics.html#cb922-9" aria-hidden="true" tabindex="-1"></a>ex_data_1 <span class="ot">=</span> <span class="fu">rbind</span>(ex_data, point_1)</span>
-<span id="cb922-10"><a href="model-diagnostics.html#cb922-10" aria-hidden="true" tabindex="-1"></a>model_1 <span class="ot">=</span> <span class="fu">lm</span>(y <span class="sc">~</span> x, <span class="at">data =</span> ex_data_1)</span>
-<span id="cb922-11"><a href="model-diagnostics.html#cb922-11" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(y <span class="sc">~</span> x, <span class="at">data =</span> ex_data_1, <span class="at">cex =</span> <span class="dv">2</span>, <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">col =</span> <span class="st">&quot;grey&quot;</span>,</span>
-<span id="cb922-12"><a href="model-diagnostics.html#cb922-12" aria-hidden="true" tabindex="-1"></a>     <span class="at">main =</span> <span class="st">&quot;Low Leverage, Large Residual, Small Influence&quot;</span>)</span>
-<span id="cb922-13"><a href="model-diagnostics.html#cb922-13" aria-hidden="true" tabindex="-1"></a><span class="fu">points</span>(<span class="at">x =</span> point_1[<span class="dv">1</span>], <span class="at">y =</span> point_1[<span class="dv">2</span>], <span class="at">pch =</span> <span class="dv">1</span>, <span class="at">cex =</span> <span class="dv">4</span>, <span class="at">col =</span> <span class="st">&quot;black&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span>
-<span id="cb922-14"><a href="model-diagnostics.html#cb922-14" aria-hidden="true" tabindex="-1"></a><span class="fu">abline</span>(ex_model, <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span>
-<span id="cb922-15"><a href="model-diagnostics.html#cb922-15" aria-hidden="true" tabindex="-1"></a><span class="fu">abline</span>(model_1, <span class="at">lty =</span> <span class="dv">2</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span>
-<span id="cb922-16"><a href="model-diagnostics.html#cb922-16" aria-hidden="true" tabindex="-1"></a><span class="fu">legend</span>(<span class="st">&quot;bottomleft&quot;</span>, <span class="fu">c</span>(<span class="st">&quot;Original Data&quot;</span>, <span class="st">&quot;Added Point&quot;</span>),</span>
-<span id="cb922-17"><a href="model-diagnostics.html#cb922-17" aria-hidden="true" tabindex="-1"></a>       <span class="at">lty =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>), <span class="at">col =</span> <span class="fu">c</span>(<span class="st">&quot;dodgerblue&quot;</span>, <span class="st">&quot;darkorange&quot;</span>))</span>
-<span id="cb922-18"><a href="model-diagnostics.html#cb922-18" aria-hidden="true" tabindex="-1"></a></span>
-<span id="cb922-19"><a href="model-diagnostics.html#cb922-19" aria-hidden="true" tabindex="-1"></a><span class="co"># high leverage, small residual, small influence</span></span>
-<span id="cb922-20"><a href="model-diagnostics.html#cb922-20" aria-hidden="true" tabindex="-1"></a>point_2 <span class="ot">=</span> <span class="fu">c</span>(<span class="dv">18</span>, <span class="sc">-</span><span class="fl">5.7</span>)</span>
-<span id="cb922-21"><a href="model-diagnostics.html#cb922-21" aria-hidden="true" tabindex="-1"></a>ex_data_2 <span class="ot">=</span> <span class="fu">rbind</span>(ex_data, point_2)</span>
-<span id="cb922-22"><a href="model-diagnostics.html#cb922-22" aria-hidden="true" tabindex="-1"></a>model_2 <span class="ot">=</span> <span class="fu">lm</span>(y <span class="sc">~</span> x, <span class="at">data =</span> ex_data_2)</span>
-<span id="cb922-23"><a href="model-diagnostics.html#cb922-23" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(y <span class="sc">~</span> x, <span class="at">data =</span> ex_data_2, <span class="at">cex =</span> <span class="dv">2</span>, <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">col =</span> <span class="st">&quot;grey&quot;</span>,</span>
-<span id="cb922-24"><a href="model-diagnostics.html#cb922-24" aria-hidden="true" tabindex="-1"></a>     <span class="at">main =</span> <span class="st">&quot;High Leverage, Small Residual, Small Influence&quot;</span>)</span>
-<span id="cb922-25"><a href="model-diagnostics.html#cb922-25" aria-hidden="true" tabindex="-1"></a><span class="fu">points</span>(<span class="at">x =</span> point_2[<span class="dv">1</span>], <span class="at">y =</span> point_2[<span class="dv">2</span>], <span class="at">pch =</span> <span class="dv">1</span>, <span class="at">cex =</span> <span class="dv">4</span>, <span class="at">col =</span> <span class="st">&quot;black&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span>
-<span id="cb922-26"><a href="model-diagnostics.html#cb922-26" aria-hidden="true" tabindex="-1"></a><span class="fu">abline</span>(ex_model, <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span>
-<span id="cb922-27"><a href="model-diagnostics.html#cb922-27" aria-hidden="true" tabindex="-1"></a><span class="fu">abline</span>(model_2, <span class="at">lty =</span> <span class="dv">2</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span>
-<span id="cb922-28"><a href="model-diagnostics.html#cb922-28" aria-hidden="true" tabindex="-1"></a><span class="fu">legend</span>(<span class="st">&quot;bottomleft&quot;</span>, <span class="fu">c</span>(<span class="st">&quot;Original Data&quot;</span>, <span class="st">&quot;Added Point&quot;</span>),</span>
-<span id="cb922-29"><a href="model-diagnostics.html#cb922-29" aria-hidden="true" tabindex="-1"></a>       <span class="at">lty =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>), <span class="at">col =</span> <span class="fu">c</span>(<span class="st">&quot;dodgerblue&quot;</span>, <span class="st">&quot;darkorange&quot;</span>))</span>
-<span id="cb922-30"><a href="model-diagnostics.html#cb922-30" aria-hidden="true" tabindex="-1"></a></span>
-<span id="cb922-31"><a href="model-diagnostics.html#cb922-31" aria-hidden="true" tabindex="-1"></a><span class="co"># high leverage, large residual, large influence</span></span>
-<span id="cb922-32"><a href="model-diagnostics.html#cb922-32" aria-hidden="true" tabindex="-1"></a>point_3 <span class="ot">=</span> <span class="fu">c</span>(<span class="dv">14</span>, <span class="fl">5.1</span>)</span>
-<span id="cb922-33"><a href="model-diagnostics.html#cb922-33" aria-hidden="true" tabindex="-1"></a>ex_data_3 <span class="ot">=</span> <span class="fu">rbind</span>(ex_data, point_3)</span>
-<span id="cb922-34"><a href="model-diagnostics.html#cb922-34" aria-hidden="true" tabindex="-1"></a>model_3 <span class="ot">=</span> <span class="fu">lm</span>(y <span class="sc">~</span> x, <span class="at">data =</span> ex_data_3)</span>
-<span id="cb922-35"><a href="model-diagnostics.html#cb922-35" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(y <span class="sc">~</span> x, <span class="at">data =</span> ex_data_3, <span class="at">cex =</span> <span class="dv">2</span>, <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">col =</span> <span class="st">&quot;grey&quot;</span>, <span class="at">ylim =</span> <span class="fu">c</span>(<span class="sc">-</span><span class="dv">3</span>, <span class="dv">12</span>),</span>
-<span id="cb922-36"><a href="model-diagnostics.html#cb922-36" aria-hidden="true" tabindex="-1"></a>     <span class="at">main =</span> <span class="st">&quot;High Leverage, Large Residual, Large Influence&quot;</span>)</span>
-<span id="cb922-37"><a href="model-diagnostics.html#cb922-37" aria-hidden="true" tabindex="-1"></a><span class="fu">points</span>(<span class="at">x =</span> point_3[<span class="dv">1</span>], <span class="at">y =</span> point_3[<span class="dv">2</span>], <span class="at">pch =</span> <span class="dv">1</span>, <span class="at">cex =</span> <span class="dv">4</span>, <span class="at">col =</span> <span class="st">&quot;black&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span>
-<span id="cb922-38"><a href="model-diagnostics.html#cb922-38" aria-hidden="true" tabindex="-1"></a><span class="fu">abline</span>(ex_model, <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span>
-<span id="cb922-39"><a href="model-diagnostics.html#cb922-39" aria-hidden="true" tabindex="-1"></a><span class="fu">abline</span>(model_3, <span class="at">lty =</span> <span class="dv">2</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span>
-<span id="cb922-40"><a href="model-diagnostics.html#cb922-40" aria-hidden="true" tabindex="-1"></a><span class="fu">legend</span>(<span class="st">&quot;bottomleft&quot;</span>, <span class="fu">c</span>(<span class="st">&quot;Original Data&quot;</span>, <span class="st">&quot;Added Point&quot;</span>),</span>
-<span id="cb922-41"><a href="model-diagnostics.html#cb922-41" aria-hidden="true" tabindex="-1"></a>       <span class="at">lty =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>), <span class="at">col =</span> <span class="fu">c</span>(<span class="st">&quot;dodgerblue&quot;</span>, <span class="st">&quot;darkorange&quot;</span>))</span></code></pre></div>
+<div class="sourceCode" id="cb922"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb922-1"><a href="model-diagnostics.html#cb922-1" tabindex="-1"></a><span class="fu">par</span>(<span class="at">mfrow =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">3</span>))</span>
+<span id="cb922-2"><a href="model-diagnostics.html#cb922-2" tabindex="-1"></a><span class="fu">set.seed</span>(<span class="dv">42</span>)</span>
+<span id="cb922-3"><a href="model-diagnostics.html#cb922-3" tabindex="-1"></a>ex_data  <span class="ot">=</span> <span class="fu">data.frame</span>(<span class="at">x =</span> <span class="dv">1</span><span class="sc">:</span><span class="dv">10</span>,</span>
+<span id="cb922-4"><a href="model-diagnostics.html#cb922-4" tabindex="-1"></a>                      <span class="at">y =</span> <span class="dv">10</span><span class="sc">:</span><span class="dv">1</span> <span class="sc">+</span> <span class="fu">rnorm</span>(<span class="at">n =</span> <span class="dv">10</span>))</span>
+<span id="cb922-5"><a href="model-diagnostics.html#cb922-5" tabindex="-1"></a>ex_model <span class="ot">=</span> <span class="fu">lm</span>(y <span class="sc">~</span> x, <span class="at">data =</span> ex_data)</span>
+<span id="cb922-6"><a href="model-diagnostics.html#cb922-6" tabindex="-1"></a></span>
+<span id="cb922-7"><a href="model-diagnostics.html#cb922-7" tabindex="-1"></a><span class="co"># low leverage, large residual, small influence</span></span>
+<span id="cb922-8"><a href="model-diagnostics.html#cb922-8" tabindex="-1"></a>point_1 <span class="ot">=</span> <span class="fu">c</span>(<span class="fl">5.4</span>, <span class="dv">11</span>)</span>
+<span id="cb922-9"><a href="model-diagnostics.html#cb922-9" tabindex="-1"></a>ex_data_1 <span class="ot">=</span> <span class="fu">rbind</span>(ex_data, point_1)</span>
+<span id="cb922-10"><a href="model-diagnostics.html#cb922-10" tabindex="-1"></a>model_1 <span class="ot">=</span> <span class="fu">lm</span>(y <span class="sc">~</span> x, <span class="at">data =</span> ex_data_1)</span>
+<span id="cb922-11"><a href="model-diagnostics.html#cb922-11" tabindex="-1"></a><span class="fu">plot</span>(y <span class="sc">~</span> x, <span class="at">data =</span> ex_data_1, <span class="at">cex =</span> <span class="dv">2</span>, <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">col =</span> <span class="st">&quot;grey&quot;</span>,</span>
+<span id="cb922-12"><a href="model-diagnostics.html#cb922-12" tabindex="-1"></a>     <span class="at">main =</span> <span class="st">&quot;Low Leverage, Large Residual, Small Influence&quot;</span>)</span>
+<span id="cb922-13"><a href="model-diagnostics.html#cb922-13" tabindex="-1"></a><span class="fu">points</span>(<span class="at">x =</span> point_1[<span class="dv">1</span>], <span class="at">y =</span> point_1[<span class="dv">2</span>], <span class="at">pch =</span> <span class="dv">1</span>, <span class="at">cex =</span> <span class="dv">4</span>, <span class="at">col =</span> <span class="st">&quot;black&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span>
+<span id="cb922-14"><a href="model-diagnostics.html#cb922-14" tabindex="-1"></a><span class="fu">abline</span>(ex_model, <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span>
+<span id="cb922-15"><a href="model-diagnostics.html#cb922-15" tabindex="-1"></a><span class="fu">abline</span>(model_1, <span class="at">lty =</span> <span class="dv">2</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span>
+<span id="cb922-16"><a href="model-diagnostics.html#cb922-16" tabindex="-1"></a><span class="fu">legend</span>(<span class="st">&quot;bottomleft&quot;</span>, <span class="fu">c</span>(<span class="st">&quot;Original Data&quot;</span>, <span class="st">&quot;Added Point&quot;</span>),</span>
+<span id="cb922-17"><a href="model-diagnostics.html#cb922-17" tabindex="-1"></a>       <span class="at">lty =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>), <span class="at">col =</span> <span class="fu">c</span>(<span class="st">&quot;dodgerblue&quot;</span>, <span class="st">&quot;darkorange&quot;</span>))</span>
+<span id="cb922-18"><a href="model-diagnostics.html#cb922-18" tabindex="-1"></a></span>
+<span id="cb922-19"><a href="model-diagnostics.html#cb922-19" tabindex="-1"></a><span class="co"># high leverage, small residual, small influence</span></span>
+<span id="cb922-20"><a href="model-diagnostics.html#cb922-20" tabindex="-1"></a>point_2 <span class="ot">=</span> <span class="fu">c</span>(<span class="dv">18</span>, <span class="sc">-</span><span class="fl">5.7</span>)</span>
+<span id="cb922-21"><a href="model-diagnostics.html#cb922-21" tabindex="-1"></a>ex_data_2 <span class="ot">=</span> <span class="fu">rbind</span>(ex_data, point_2)</span>
+<span id="cb922-22"><a href="model-diagnostics.html#cb922-22" tabindex="-1"></a>model_2 <span class="ot">=</span> <span class="fu">lm</span>(y <span class="sc">~</span> x, <span class="at">data =</span> ex_data_2)</span>
+<span id="cb922-23"><a href="model-diagnostics.html#cb922-23" tabindex="-1"></a><span class="fu">plot</span>(y <span class="sc">~</span> x, <span class="at">data =</span> ex_data_2, <span class="at">cex =</span> <span class="dv">2</span>, <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">col =</span> <span class="st">&quot;grey&quot;</span>,</span>
+<span id="cb922-24"><a href="model-diagnostics.html#cb922-24" tabindex="-1"></a>     <span class="at">main =</span> <span class="st">&quot;High Leverage, Small Residual, Small Influence&quot;</span>)</span>
+<span id="cb922-25"><a href="model-diagnostics.html#cb922-25" tabindex="-1"></a><span class="fu">points</span>(<span class="at">x =</span> point_2[<span class="dv">1</span>], <span class="at">y =</span> point_2[<span class="dv">2</span>], <span class="at">pch =</span> <span class="dv">1</span>, <span class="at">cex =</span> <span class="dv">4</span>, <span class="at">col =</span> <span class="st">&quot;black&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span>
+<span id="cb922-26"><a href="model-diagnostics.html#cb922-26" tabindex="-1"></a><span class="fu">abline</span>(ex_model, <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span>
+<span id="cb922-27"><a href="model-diagnostics.html#cb922-27" tabindex="-1"></a><span class="fu">abline</span>(model_2, <span class="at">lty =</span> <span class="dv">2</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span>
+<span id="cb922-28"><a href="model-diagnostics.html#cb922-28" tabindex="-1"></a><span class="fu">legend</span>(<span class="st">&quot;bottomleft&quot;</span>, <span class="fu">c</span>(<span class="st">&quot;Original Data&quot;</span>, <span class="st">&quot;Added Point&quot;</span>),</span>
+<span id="cb922-29"><a href="model-diagnostics.html#cb922-29" tabindex="-1"></a>       <span class="at">lty =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>), <span class="at">col =</span> <span class="fu">c</span>(<span class="st">&quot;dodgerblue&quot;</span>, <span class="st">&quot;darkorange&quot;</span>))</span>
+<span id="cb922-30"><a href="model-diagnostics.html#cb922-30" tabindex="-1"></a></span>
+<span id="cb922-31"><a href="model-diagnostics.html#cb922-31" tabindex="-1"></a><span class="co"># high leverage, large residual, large influence</span></span>
+<span id="cb922-32"><a href="model-diagnostics.html#cb922-32" tabindex="-1"></a>point_3 <span class="ot">=</span> <span class="fu">c</span>(<span class="dv">14</span>, <span class="fl">5.1</span>)</span>
+<span id="cb922-33"><a href="model-diagnostics.html#cb922-33" tabindex="-1"></a>ex_data_3 <span class="ot">=</span> <span class="fu">rbind</span>(ex_data, point_3)</span>
+<span id="cb922-34"><a href="model-diagnostics.html#cb922-34" tabindex="-1"></a>model_3 <span class="ot">=</span> <span class="fu">lm</span>(y <span class="sc">~</span> x, <span class="at">data =</span> ex_data_3)</span>
+<span id="cb922-35"><a href="model-diagnostics.html#cb922-35" tabindex="-1"></a><span class="fu">plot</span>(y <span class="sc">~</span> x, <span class="at">data =</span> ex_data_3, <span class="at">cex =</span> <span class="dv">2</span>, <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">col =</span> <span class="st">&quot;grey&quot;</span>, <span class="at">ylim =</span> <span class="fu">c</span>(<span class="sc">-</span><span class="dv">3</span>, <span class="dv">12</span>),</span>
+<span id="cb922-36"><a href="model-diagnostics.html#cb922-36" tabindex="-1"></a>     <span class="at">main =</span> <span class="st">&quot;High Leverage, Large Residual, Large Influence&quot;</span>)</span>
+<span id="cb922-37"><a href="model-diagnostics.html#cb922-37" tabindex="-1"></a><span class="fu">points</span>(<span class="at">x =</span> point_3[<span class="dv">1</span>], <span class="at">y =</span> point_3[<span class="dv">2</span>], <span class="at">pch =</span> <span class="dv">1</span>, <span class="at">cex =</span> <span class="dv">4</span>, <span class="at">col =</span> <span class="st">&quot;black&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span>
+<span id="cb922-38"><a href="model-diagnostics.html#cb922-38" tabindex="-1"></a><span class="fu">abline</span>(ex_model, <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span>
+<span id="cb922-39"><a href="model-diagnostics.html#cb922-39" tabindex="-1"></a><span class="fu">abline</span>(model_3, <span class="at">lty =</span> <span class="dv">2</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span>
+<span id="cb922-40"><a href="model-diagnostics.html#cb922-40" tabindex="-1"></a><span class="fu">legend</span>(<span class="st">&quot;bottomleft&quot;</span>, <span class="fu">c</span>(<span class="st">&quot;Original Data&quot;</span>, <span class="st">&quot;Added Point&quot;</span>),</span>
+<span id="cb922-41"><a href="model-diagnostics.html#cb922-41" tabindex="-1"></a>       <span class="at">lty =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>), <span class="at">col =</span> <span class="fu">c</span>(<span class="st">&quot;dodgerblue&quot;</span>, <span class="st">&quot;darkorange&quot;</span>))</span></code></pre></div>
 <p><img src="diagnostics_files/figure-html/unusual_obs_plot-1.png" width="1440" style="display: block; margin: auto;" /></p>
 <p>The blue solid line in each plot is a regression fit to the 10 original data points stored in <code>ex_data</code>. The dashed orange line in each plot is the result of adding a single point to the original data in <code>ex_data</code>. This additional point is indicated by the circled point.</p>
 <p>The slope of the regression for the original ten points, the solid blue line, is given by:</p>
-<div class="sourceCode" id="cb923"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb923-1"><a href="model-diagnostics.html#cb923-1" aria-hidden="true" tabindex="-1"></a><span class="fu">coef</span>(ex_model)[<span class="dv">2</span>]</span></code></pre></div>
+<div class="sourceCode" id="cb923"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb923-1"><a href="model-diagnostics.html#cb923-1" tabindex="-1"></a><span class="fu">coef</span>(ex_model)[<span class="dv">2</span>]</span></code></pre></div>
 <pre><code>##          x 
 ## -0.9696033</code></pre>
 <p>The added point in the first plot has a <em>small</em> effect on the slope, which becomes:</p>
-<div class="sourceCode" id="cb925"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb925-1"><a href="model-diagnostics.html#cb925-1" aria-hidden="true" tabindex="-1"></a><span class="fu">coef</span>(model_1)[<span class="dv">2</span>]</span></code></pre></div>
+<div class="sourceCode" id="cb925"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb925-1"><a href="model-diagnostics.html#cb925-1" tabindex="-1"></a><span class="fu">coef</span>(model_1)[<span class="dv">2</span>]</span></code></pre></div>
 <pre><code>##          x 
 ## -0.9749534</code></pre>
 <p>We will say that this point has low leverage, is an outlier due to its large residual, but has small influence.</p>
 <p>The added point in the second plot also has a <em>small</em> effect on the slope, which is:</p>
-<div class="sourceCode" id="cb927"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb927-1"><a href="model-diagnostics.html#cb927-1" aria-hidden="true" tabindex="-1"></a><span class="fu">coef</span>(model_2)[<span class="dv">2</span>]</span></code></pre></div>
+<div class="sourceCode" id="cb927"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb927-1"><a href="model-diagnostics.html#cb927-1" tabindex="-1"></a><span class="fu">coef</span>(model_2)[<span class="dv">2</span>]</span></code></pre></div>
 <pre><code>##          x 
 ## -0.9507397</code></pre>
 <p>We will say that this point has high leverage, is not an outlier due to its small residual, and has a very small influence.</p>
 <p>Lastly, the added point in the third plot has a <em>large</em> effect on the slope, which is now:</p>
-<div class="sourceCode" id="cb929"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb929-1"><a href="model-diagnostics.html#cb929-1" aria-hidden="true" tabindex="-1"></a><span class="fu">coef</span>(model_3)[<span class="dv">2</span>]</span></code></pre></div>
+<div class="sourceCode" id="cb929"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb929-1"><a href="model-diagnostics.html#cb929-1" tabindex="-1"></a><span class="fu">coef</span>(model_3)[<span class="dv">2</span>]</span></code></pre></div>
 <pre><code>##          x 
 ## -0.5892241</code></pre>
 <p>This added point is influential. It both has high leverage, and is an outlier due to its large residual.</p>
@@ -937,43 +937,43 @@ <h3><span class="header-section-number">13.3.1</span> Leverage<a href="model-dia
 \]</span></p>
 <p>This expression should be familiar. (Think back to inference for SLR.) It suggests that the large leverages occur when <span class="math inline">\(x\)</span> values are far from their mean. Recall that the regression goes through the point <span class="math inline">\((\bar{x},\bar{y})\)</span>.</p>
 <p>There are multiple ways to find leverages in <code>R</code>.</p>
-<div class="sourceCode" id="cb931"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb931-1"><a href="model-diagnostics.html#cb931-1" aria-hidden="true" tabindex="-1"></a>lev_ex <span class="ot">=</span> <span class="fu">data.frame</span>(</span>
-<span id="cb931-2"><a href="model-diagnostics.html#cb931-2" aria-hidden="true" tabindex="-1"></a>  <span class="at">x1 =</span> <span class="fu">c</span>(<span class="dv">0</span>, <span class="dv">11</span>, <span class="dv">11</span>, <span class="dv">7</span>, <span class="dv">4</span>, <span class="dv">10</span>, <span class="dv">5</span>, <span class="dv">8</span>),</span>
-<span id="cb931-3"><a href="model-diagnostics.html#cb931-3" aria-hidden="true" tabindex="-1"></a>  <span class="at">x2 =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">5</span>, <span class="dv">4</span>, <span class="dv">3</span>, <span class="dv">1</span>, <span class="dv">4</span>, <span class="dv">4</span>, <span class="dv">2</span>),</span>
-<span id="cb931-4"><a href="model-diagnostics.html#cb931-4" aria-hidden="true" tabindex="-1"></a>  <span class="at">y  =</span> <span class="fu">c</span>(<span class="dv">11</span>, <span class="dv">15</span>, <span class="dv">13</span>, <span class="dv">14</span>, <span class="dv">0</span>, <span class="dv">19</span>, <span class="dv">16</span>, <span class="dv">8</span>))</span>
-<span id="cb931-5"><a href="model-diagnostics.html#cb931-5" aria-hidden="true" tabindex="-1"></a></span>
-<span id="cb931-6"><a href="model-diagnostics.html#cb931-6" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(x2 <span class="sc">~</span> x1, <span class="at">data =</span> lev_ex, <span class="at">cex =</span> <span class="dv">2</span>)</span>
-<span id="cb931-7"><a href="model-diagnostics.html#cb931-7" aria-hidden="true" tabindex="-1"></a><span class="fu">points</span>(<span class="dv">7</span>, <span class="dv">3</span>, <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">col =</span> <span class="st">&quot;red&quot;</span>, <span class="at">cex =</span> <span class="dv">2</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb931"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb931-1"><a href="model-diagnostics.html#cb931-1" tabindex="-1"></a>lev_ex <span class="ot">=</span> <span class="fu">data.frame</span>(</span>
+<span id="cb931-2"><a href="model-diagnostics.html#cb931-2" tabindex="-1"></a>  <span class="at">x1 =</span> <span class="fu">c</span>(<span class="dv">0</span>, <span class="dv">11</span>, <span class="dv">11</span>, <span class="dv">7</span>, <span class="dv">4</span>, <span class="dv">10</span>, <span class="dv">5</span>, <span class="dv">8</span>),</span>
+<span id="cb931-3"><a href="model-diagnostics.html#cb931-3" tabindex="-1"></a>  <span class="at">x2 =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">5</span>, <span class="dv">4</span>, <span class="dv">3</span>, <span class="dv">1</span>, <span class="dv">4</span>, <span class="dv">4</span>, <span class="dv">2</span>),</span>
+<span id="cb931-4"><a href="model-diagnostics.html#cb931-4" tabindex="-1"></a>  <span class="at">y  =</span> <span class="fu">c</span>(<span class="dv">11</span>, <span class="dv">15</span>, <span class="dv">13</span>, <span class="dv">14</span>, <span class="dv">0</span>, <span class="dv">19</span>, <span class="dv">16</span>, <span class="dv">8</span>))</span>
+<span id="cb931-5"><a href="model-diagnostics.html#cb931-5" tabindex="-1"></a></span>
+<span id="cb931-6"><a href="model-diagnostics.html#cb931-6" tabindex="-1"></a><span class="fu">plot</span>(x2 <span class="sc">~</span> x1, <span class="at">data =</span> lev_ex, <span class="at">cex =</span> <span class="dv">2</span>)</span>
+<span id="cb931-7"><a href="model-diagnostics.html#cb931-7" tabindex="-1"></a><span class="fu">points</span>(<span class="dv">7</span>, <span class="dv">3</span>, <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">col =</span> <span class="st">&quot;red&quot;</span>, <span class="at">cex =</span> <span class="dv">2</span>)</span></code></pre></div>
 <p><img src="diagnostics_files/figure-html/unnamed-chunk-32-1.png" width="672" style="display: block; margin: auto;" /></p>
 <p>Here we’ve created some multivariate data. Notice that we have plotted the <span class="math inline">\(x\)</span> values, not the <span class="math inline">\(y\)</span> values. The red point is <span class="math inline">\((7, 3)\)</span> which is the mean of <code>x1</code> and the mean of <code>x2</code> respectively.</p>
 <p>We could calculate the leverages using the expressions defined above. We first create the <span class="math inline">\(X\)</span> matrix, then calculate <span class="math inline">\(H\)</span> as defined, and extract the diagonal elements.</p>
-<div class="sourceCode" id="cb932"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb932-1"><a href="model-diagnostics.html#cb932-1" aria-hidden="true" tabindex="-1"></a>X <span class="ot">=</span> <span class="fu">cbind</span>(<span class="fu">rep</span>(<span class="dv">1</span>, <span class="dv">8</span>), lev_ex<span class="sc">$</span>x1, lev_ex<span class="sc">$</span>x2)</span>
-<span id="cb932-2"><a href="model-diagnostics.html#cb932-2" aria-hidden="true" tabindex="-1"></a>H <span class="ot">=</span> X <span class="sc">%*%</span> <span class="fu">solve</span>(<span class="fu">t</span>(X) <span class="sc">%*%</span> X) <span class="sc">%*%</span> <span class="fu">t</span>(X)</span>
-<span id="cb932-3"><a href="model-diagnostics.html#cb932-3" aria-hidden="true" tabindex="-1"></a><span class="fu">diag</span>(H)</span></code></pre></div>
+<div class="sourceCode" id="cb932"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb932-1"><a href="model-diagnostics.html#cb932-1" tabindex="-1"></a>X <span class="ot">=</span> <span class="fu">cbind</span>(<span class="fu">rep</span>(<span class="dv">1</span>, <span class="dv">8</span>), lev_ex<span class="sc">$</span>x1, lev_ex<span class="sc">$</span>x2)</span>
+<span id="cb932-2"><a href="model-diagnostics.html#cb932-2" tabindex="-1"></a>H <span class="ot">=</span> X <span class="sc">%*%</span> <span class="fu">solve</span>(<span class="fu">t</span>(X) <span class="sc">%*%</span> X) <span class="sc">%*%</span> <span class="fu">t</span>(X)</span>
+<span id="cb932-3"><a href="model-diagnostics.html#cb932-3" tabindex="-1"></a><span class="fu">diag</span>(H)</span></code></pre></div>
 <pre><code>## [1] 0.6000 0.3750 0.2875 0.1250 0.4000 0.2125 0.5875 0.4125</code></pre>
 <p>Notice here, we have two predictors, so the regression would have 3 <span class="math inline">\(\beta\)</span> parameters, so the sum of the diagonal elements is 3.</p>
-<div class="sourceCode" id="cb934"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb934-1"><a href="model-diagnostics.html#cb934-1" aria-hidden="true" tabindex="-1"></a><span class="fu">sum</span>(<span class="fu">diag</span>(H))</span></code></pre></div>
+<div class="sourceCode" id="cb934"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb934-1"><a href="model-diagnostics.html#cb934-1" tabindex="-1"></a><span class="fu">sum</span>(<span class="fu">diag</span>(H))</span></code></pre></div>
 <pre><code>## [1] 3</code></pre>
 <p>Alternatively, the method we will use more often, is to simply fit a regression, then use the <code>hatvalues()</code> function, which returns the leverages.</p>
-<div class="sourceCode" id="cb936"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb936-1"><a href="model-diagnostics.html#cb936-1" aria-hidden="true" tabindex="-1"></a>lev_fit <span class="ot">=</span> <span class="fu">lm</span>(y <span class="sc">~</span> ., <span class="at">data =</span> lev_ex)</span>
-<span id="cb936-2"><a href="model-diagnostics.html#cb936-2" aria-hidden="true" tabindex="-1"></a><span class="fu">hatvalues</span>(lev_fit)</span></code></pre></div>
+<div class="sourceCode" id="cb936"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb936-1"><a href="model-diagnostics.html#cb936-1" tabindex="-1"></a>lev_fit <span class="ot">=</span> <span class="fu">lm</span>(y <span class="sc">~</span> ., <span class="at">data =</span> lev_ex)</span>
+<span id="cb936-2"><a href="model-diagnostics.html#cb936-2" tabindex="-1"></a><span class="fu">hatvalues</span>(lev_fit)</span></code></pre></div>
 <pre><code>##      1      2      3      4      5      6      7      8 
 ## 0.6000 0.3750 0.2875 0.1250 0.4000 0.2125 0.5875 0.4125</code></pre>
 <p>Again, note that here we have “used” the <span class="math inline">\(y\)</span> values to fit the regression, but <code>R</code> still ignores them when calculating the leverages, as leverages only depend on the <span class="math inline">\(x\)</span> values.</p>
-<div class="sourceCode" id="cb938"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb938-1"><a href="model-diagnostics.html#cb938-1" aria-hidden="true" tabindex="-1"></a><span class="fu">coef</span>(lev_fit)</span></code></pre></div>
+<div class="sourceCode" id="cb938"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb938-1"><a href="model-diagnostics.html#cb938-1" tabindex="-1"></a><span class="fu">coef</span>(lev_fit)</span></code></pre></div>
 <pre><code>## (Intercept)          x1          x2 
 ##         3.7        -0.7         4.4</code></pre>
 <p>Let’s see what happens to these coefficients when we modify the <code>y</code> value of the point with the highest leverage.</p>
-<div class="sourceCode" id="cb940"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb940-1"><a href="model-diagnostics.html#cb940-1" aria-hidden="true" tabindex="-1"></a><span class="fu">which.max</span>(<span class="fu">hatvalues</span>(lev_fit))</span></code></pre></div>
+<div class="sourceCode" id="cb940"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb940-1"><a href="model-diagnostics.html#cb940-1" tabindex="-1"></a><span class="fu">which.max</span>(<span class="fu">hatvalues</span>(lev_fit))</span></code></pre></div>
 <pre><code>## 1 
 ## 1</code></pre>
-<div class="sourceCode" id="cb942"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb942-1"><a href="model-diagnostics.html#cb942-1" aria-hidden="true" tabindex="-1"></a>lev_ex[<span class="fu">which.max</span>(<span class="fu">hatvalues</span>(lev_fit)),]</span></code></pre></div>
+<div class="sourceCode" id="cb942"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb942-1"><a href="model-diagnostics.html#cb942-1" tabindex="-1"></a>lev_ex[<span class="fu">which.max</span>(<span class="fu">hatvalues</span>(lev_fit)),]</span></code></pre></div>
 <pre><code>##   x1 x2  y
 ## 1  0  1 11</code></pre>
 <p>We see that the original <code>y</code> value is 11. We’ll create a copy of the data, and modify this point to have a <code>y</code> value of <code>20</code>.</p>
-<div class="sourceCode" id="cb944"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb944-1"><a href="model-diagnostics.html#cb944-1" aria-hidden="true" tabindex="-1"></a>lev_ex_1 <span class="ot">=</span> lev_ex</span>
-<span id="cb944-2"><a href="model-diagnostics.html#cb944-2" aria-hidden="true" tabindex="-1"></a>lev_ex_1<span class="sc">$</span>y[<span class="dv">1</span>] <span class="ot">=</span> <span class="dv">20</span></span>
-<span id="cb944-3"><a href="model-diagnostics.html#cb944-3" aria-hidden="true" tabindex="-1"></a><span class="fu">lm</span>(y <span class="sc">~</span> ., <span class="at">data =</span> lev_ex_1)</span></code></pre></div>
+<div class="sourceCode" id="cb944"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb944-1"><a href="model-diagnostics.html#cb944-1" tabindex="-1"></a>lev_ex_1 <span class="ot">=</span> lev_ex</span>
+<span id="cb944-2"><a href="model-diagnostics.html#cb944-2" tabindex="-1"></a>lev_ex_1<span class="sc">$</span>y[<span class="dv">1</span>] <span class="ot">=</span> <span class="dv">20</span></span>
+<span id="cb944-3"><a href="model-diagnostics.html#cb944-3" tabindex="-1"></a><span class="fu">lm</span>(y <span class="sc">~</span> ., <span class="at">data =</span> lev_ex_1)</span></code></pre></div>
 <pre><code>## 
 ## Call:
 ## lm(formula = y ~ ., data = lev_ex_1)
@@ -983,16 +983,16 @@ <h3><span class="header-section-number">13.3.1</span> Leverage<a href="model-dia
 ##       8.875       -1.375        4.625</code></pre>
 <p>Notice the <strong>large</strong> changes in the coefficients. Also notice that each of the coefficients has changed in some way. Note that the leverages of the points would not have changed, as we have not modified any of the <span class="math inline">\(x\)</span> values.</p>
 <p>Now let’s see what happens to these coefficients when we modify the <code>y</code> value of the point with the lowest leverage.</p>
-<div class="sourceCode" id="cb946"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb946-1"><a href="model-diagnostics.html#cb946-1" aria-hidden="true" tabindex="-1"></a><span class="fu">which.min</span>(<span class="fu">hatvalues</span>(lev_fit))</span></code></pre></div>
+<div class="sourceCode" id="cb946"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb946-1"><a href="model-diagnostics.html#cb946-1" tabindex="-1"></a><span class="fu">which.min</span>(<span class="fu">hatvalues</span>(lev_fit))</span></code></pre></div>
 <pre><code>## 4 
 ## 4</code></pre>
-<div class="sourceCode" id="cb948"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb948-1"><a href="model-diagnostics.html#cb948-1" aria-hidden="true" tabindex="-1"></a>lev_ex[<span class="fu">which.min</span>(<span class="fu">hatvalues</span>(lev_fit)),]</span></code></pre></div>
+<div class="sourceCode" id="cb948"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb948-1"><a href="model-diagnostics.html#cb948-1" tabindex="-1"></a>lev_ex[<span class="fu">which.min</span>(<span class="fu">hatvalues</span>(lev_fit)),]</span></code></pre></div>
 <pre><code>##   x1 x2  y
 ## 4  7  3 14</code></pre>
 <p>We see that the original <code>y</code> value is 14. We’ll again create a copy of the data, and modify this point to have a <code>y</code> value of <code>30</code>.</p>
-<div class="sourceCode" id="cb950"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb950-1"><a href="model-diagnostics.html#cb950-1" aria-hidden="true" tabindex="-1"></a>lev_ex_2 <span class="ot">=</span> lev_ex</span>
-<span id="cb950-2"><a href="model-diagnostics.html#cb950-2" aria-hidden="true" tabindex="-1"></a>lev_ex_2<span class="sc">$</span>y[<span class="dv">4</span>] <span class="ot">=</span> <span class="dv">30</span></span>
-<span id="cb950-3"><a href="model-diagnostics.html#cb950-3" aria-hidden="true" tabindex="-1"></a><span class="fu">lm</span>(y <span class="sc">~</span> ., <span class="at">data =</span> lev_ex_2)</span></code></pre></div>
+<div class="sourceCode" id="cb950"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb950-1"><a href="model-diagnostics.html#cb950-1" tabindex="-1"></a>lev_ex_2 <span class="ot">=</span> lev_ex</span>
+<span id="cb950-2"><a href="model-diagnostics.html#cb950-2" tabindex="-1"></a>lev_ex_2<span class="sc">$</span>y[<span class="dv">4</span>] <span class="ot">=</span> <span class="dv">30</span></span>
+<span id="cb950-3"><a href="model-diagnostics.html#cb950-3" tabindex="-1"></a><span class="fu">lm</span>(y <span class="sc">~</span> ., <span class="at">data =</span> lev_ex_2)</span></code></pre></div>
 <pre><code>## 
 ## Call:
 ## lm(formula = y ~ ., data = lev_ex_2)
@@ -1001,38 +1001,38 @@ <h3><span class="header-section-number">13.3.1</span> Leverage<a href="model-dia
 ## (Intercept)           x1           x2  
 ##         5.7         -0.7          4.4</code></pre>
 <p>This time despite a large change in the <code>y</code> value, there is only small change in the coefficients. Also, only the intercept has changed!</p>
-<div class="sourceCode" id="cb952"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb952-1"><a href="model-diagnostics.html#cb952-1" aria-hidden="true" tabindex="-1"></a><span class="fu">mean</span>(lev_ex<span class="sc">$</span>x1)</span></code></pre></div>
+<div class="sourceCode" id="cb952"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb952-1"><a href="model-diagnostics.html#cb952-1" tabindex="-1"></a><span class="fu">mean</span>(lev_ex<span class="sc">$</span>x1)</span></code></pre></div>
 <pre><code>## [1] 7</code></pre>
-<div class="sourceCode" id="cb954"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb954-1"><a href="model-diagnostics.html#cb954-1" aria-hidden="true" tabindex="-1"></a><span class="fu">mean</span>(lev_ex<span class="sc">$</span>x2)</span></code></pre></div>
+<div class="sourceCode" id="cb954"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb954-1"><a href="model-diagnostics.html#cb954-1" tabindex="-1"></a><span class="fu">mean</span>(lev_ex<span class="sc">$</span>x2)</span></code></pre></div>
 <pre><code>## [1] 3</code></pre>
-<div class="sourceCode" id="cb956"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb956-1"><a href="model-diagnostics.html#cb956-1" aria-hidden="true" tabindex="-1"></a>lev_ex[<span class="dv">4</span>,]</span></code></pre></div>
+<div class="sourceCode" id="cb956"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb956-1"><a href="model-diagnostics.html#cb956-1" tabindex="-1"></a>lev_ex[<span class="dv">4</span>,]</span></code></pre></div>
 <pre><code>##   x1 x2  y
 ## 4  7  3 14</code></pre>
 <p>Notice that this point was the mean of both of the predictors.</p>
 <p>Returning to our three plots, each with an added point, we can calculate the leverages for each. Note that the 11th data point each time is the added data point.</p>
-<div class="sourceCode" id="cb958"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb958-1"><a href="model-diagnostics.html#cb958-1" aria-hidden="true" tabindex="-1"></a><span class="fu">hatvalues</span>(model_1)</span></code></pre></div>
+<div class="sourceCode" id="cb958"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb958-1"><a href="model-diagnostics.html#cb958-1" tabindex="-1"></a><span class="fu">hatvalues</span>(model_1)</span></code></pre></div>
 <pre><code>##          1          2          3          4          5          6          7 
 ## 0.33534597 0.23860732 0.16610842 0.11784927 0.09382988 0.09405024 0.11851036 
 ##          8          9         10         11 
 ## 0.16721022 0.24014985 0.33732922 0.09100926</code></pre>
-<div class="sourceCode" id="cb960"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb960-1"><a href="model-diagnostics.html#cb960-1" aria-hidden="true" tabindex="-1"></a><span class="fu">hatvalues</span>(model_2)</span></code></pre></div>
+<div class="sourceCode" id="cb960"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb960-1"><a href="model-diagnostics.html#cb960-1" tabindex="-1"></a><span class="fu">hatvalues</span>(model_2)</span></code></pre></div>
 <pre><code>##          1          2          3          4          5          6          7 
 ## 0.23238866 0.18663968 0.14979757 0.12186235 0.10283401 0.09271255 0.09149798 
 ##          8          9         10         11 
 ## 0.09919028 0.11578947 0.14129555 0.66599190</code></pre>
-<div class="sourceCode" id="cb962"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb962-1"><a href="model-diagnostics.html#cb962-1" aria-hidden="true" tabindex="-1"></a><span class="fu">hatvalues</span>(model_3)</span></code></pre></div>
+<div class="sourceCode" id="cb962"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb962-1"><a href="model-diagnostics.html#cb962-1" tabindex="-1"></a><span class="fu">hatvalues</span>(model_3)</span></code></pre></div>
 <pre><code>##          1          2          3          4          5          6          7 
 ## 0.27852761 0.21411043 0.16319018 0.12576687 0.10184049 0.09141104 0.09447853 
 ##          8          9         10         11 
 ## 0.11104294 0.14110429 0.18466258 0.49386503</code></pre>
 <p>Are any of these large?</p>
-<div class="sourceCode" id="cb964"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb964-1"><a href="model-diagnostics.html#cb964-1" aria-hidden="true" tabindex="-1"></a><span class="fu">hatvalues</span>(model_1) <span class="sc">&gt;</span> <span class="dv">2</span> <span class="sc">*</span> <span class="fu">mean</span>(<span class="fu">hatvalues</span>(model_1))</span></code></pre></div>
+<div class="sourceCode" id="cb964"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb964-1"><a href="model-diagnostics.html#cb964-1" tabindex="-1"></a><span class="fu">hatvalues</span>(model_1) <span class="sc">&gt;</span> <span class="dv">2</span> <span class="sc">*</span> <span class="fu">mean</span>(<span class="fu">hatvalues</span>(model_1))</span></code></pre></div>
 <pre><code>##     1     2     3     4     5     6     7     8     9    10    11 
 ## FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE</code></pre>
-<div class="sourceCode" id="cb966"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb966-1"><a href="model-diagnostics.html#cb966-1" aria-hidden="true" tabindex="-1"></a><span class="fu">hatvalues</span>(model_2) <span class="sc">&gt;</span> <span class="dv">2</span> <span class="sc">*</span> <span class="fu">mean</span>(<span class="fu">hatvalues</span>(model_2))</span></code></pre></div>
+<div class="sourceCode" id="cb966"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb966-1"><a href="model-diagnostics.html#cb966-1" tabindex="-1"></a><span class="fu">hatvalues</span>(model_2) <span class="sc">&gt;</span> <span class="dv">2</span> <span class="sc">*</span> <span class="fu">mean</span>(<span class="fu">hatvalues</span>(model_2))</span></code></pre></div>
 <pre><code>##     1     2     3     4     5     6     7     8     9    10    11 
 ## FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE  TRUE</code></pre>
-<div class="sourceCode" id="cb968"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb968-1"><a href="model-diagnostics.html#cb968-1" aria-hidden="true" tabindex="-1"></a><span class="fu">hatvalues</span>(model_3) <span class="sc">&gt;</span> <span class="dv">2</span> <span class="sc">*</span> <span class="fu">mean</span>(<span class="fu">hatvalues</span>(model_3))</span></code></pre></div>
+<div class="sourceCode" id="cb968"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb968-1"><a href="model-diagnostics.html#cb968-1" tabindex="-1"></a><span class="fu">hatvalues</span>(model_3) <span class="sc">&gt;</span> <span class="dv">2</span> <span class="sc">*</span> <span class="fu">mean</span>(<span class="fu">hatvalues</span>(model_3))</span></code></pre></div>
 <pre><code>##     1     2     3     4     5     6     7     8     9    10    11 
 ## FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE  TRUE</code></pre>
 <p>We see that in the second and third plots, the added point is a point of high leverage. Recall that only in the third plot did that have an influence on the regression. To understand why, we’ll need to discuss outliers.</p>
@@ -1059,44 +1059,44 @@ <h3><span class="header-section-number">13.3.2</span> Outliers<a href="model-dia
 <p>when <span class="math inline">\(n\)</span> is large.</p>
 <p>We can use this fact to identify “large” residuals. For example, standardized residuals greater than 2 in magnitude should only happen approximately 5 percent of the time.</p>
 <p>Returning again to our three plots, each with an added point, we can calculate the residuals and standardized residuals for each. Standardized residuals can be obtained in <code>R</code> by using <code>rstandard()</code> where we would normally use <code>resid()</code>.</p>
-<div class="sourceCode" id="cb970"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb970-1"><a href="model-diagnostics.html#cb970-1" aria-hidden="true" tabindex="-1"></a><span class="fu">resid</span>(model_1)</span></code></pre></div>
+<div class="sourceCode" id="cb970"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb970-1"><a href="model-diagnostics.html#cb970-1" tabindex="-1"></a><span class="fu">resid</span>(model_1)</span></code></pre></div>
 <pre><code>##          1          2          3          4          5          6          7 
 ##  0.4949887 -1.4657145 -0.5629345 -0.3182468 -0.5718877 -1.1073271  0.4852728 
 ##          8          9         10         11 
 ## -1.1459548  0.9420814 -1.1641029  4.4138254</code></pre>
-<div class="sourceCode" id="cb972"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb972-1"><a href="model-diagnostics.html#cb972-1" aria-hidden="true" tabindex="-1"></a><span class="fu">rstandard</span>(model_1)</span></code></pre></div>
+<div class="sourceCode" id="cb972"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb972-1"><a href="model-diagnostics.html#cb972-1" tabindex="-1"></a><span class="fu">rstandard</span>(model_1)</span></code></pre></div>
 <pre><code>##          1          2          3          4          5          6          7 
 ##  0.3464701 -0.9585470 -0.3517802 -0.1933575 -0.3428264 -0.6638841  0.2949482 
 ##          8          9         10         11 
 ## -0.7165857  0.6167268 -0.8160389  2.6418234</code></pre>
-<div class="sourceCode" id="cb974"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb974-1"><a href="model-diagnostics.html#cb974-1" aria-hidden="true" tabindex="-1"></a><span class="fu">rstandard</span>(model_1)[<span class="fu">abs</span>(<span class="fu">rstandard</span>(model_1)) <span class="sc">&gt;</span> <span class="dv">2</span>]</span></code></pre></div>
+<div class="sourceCode" id="cb974"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb974-1"><a href="model-diagnostics.html#cb974-1" tabindex="-1"></a><span class="fu">rstandard</span>(model_1)[<span class="fu">abs</span>(<span class="fu">rstandard</span>(model_1)) <span class="sc">&gt;</span> <span class="dv">2</span>]</span></code></pre></div>
 <pre><code>##       11 
 ## 2.641823</code></pre>
 <p>In the first plot, we see that the 11th point, the added point, is a large standardized residual.</p>
-<div class="sourceCode" id="cb976"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb976-1"><a href="model-diagnostics.html#cb976-1" aria-hidden="true" tabindex="-1"></a><span class="fu">resid</span>(model_2)</span></code></pre></div>
+<div class="sourceCode" id="cb976"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb976-1"><a href="model-diagnostics.html#cb976-1" tabindex="-1"></a><span class="fu">resid</span>(model_2)</span></code></pre></div>
 <pre><code>##           1           2           3           4           5           6 
 ##  1.03288292 -0.95203397 -0.07346766  0.14700626 -0.13084829 -0.69050140 
 ##           7           8           9          10          11 
 ##  0.87788484 -0.77755647  1.28626601 -0.84413207  0.12449986</code></pre>
-<div class="sourceCode" id="cb978"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb978-1"><a href="model-diagnostics.html#cb978-1" aria-hidden="true" tabindex="-1"></a><span class="fu">rstandard</span>(model_2)</span></code></pre></div>
+<div class="sourceCode" id="cb978"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb978-1"><a href="model-diagnostics.html#cb978-1" tabindex="-1"></a><span class="fu">rstandard</span>(model_2)</span></code></pre></div>
 <pre><code>##           1           2           3           4           5           6 
 ##  1.41447023 -1.26655590 -0.09559792  0.18822094 -0.16574677 -0.86977220 
 ##           7           8           9          10          11 
 ##  1.10506546 -0.98294409  1.64121833 -1.09295417  0.25846620</code></pre>
-<div class="sourceCode" id="cb980"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb980-1"><a href="model-diagnostics.html#cb980-1" aria-hidden="true" tabindex="-1"></a><span class="fu">rstandard</span>(model_2)[<span class="fu">abs</span>(<span class="fu">rstandard</span>(model_2)) <span class="sc">&gt;</span> <span class="dv">2</span>]</span></code></pre></div>
+<div class="sourceCode" id="cb980"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb980-1"><a href="model-diagnostics.html#cb980-1" tabindex="-1"></a><span class="fu">rstandard</span>(model_2)[<span class="fu">abs</span>(<span class="fu">rstandard</span>(model_2)) <span class="sc">&gt;</span> <span class="dv">2</span>]</span></code></pre></div>
 <pre><code>## named numeric(0)</code></pre>
 <p>In the second plot, we see that there are no points with large standardized residuals.</p>
-<div class="sourceCode" id="cb982"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb982-1"><a href="model-diagnostics.html#cb982-1" aria-hidden="true" tabindex="-1"></a><span class="fu">resid</span>(model_3)</span></code></pre></div>
+<div class="sourceCode" id="cb982"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb982-1"><a href="model-diagnostics.html#cb982-1" tabindex="-1"></a><span class="fu">resid</span>(model_3)</span></code></pre></div>
 <pre><code>##           1           2           3           4           5           6 
 ##  2.30296166 -0.04347087  0.47357980  0.33253808 -0.30683212 -1.22800087 
 ##           7           8           9          10          11 
 ## -0.02113027 -2.03808722 -0.33578039 -2.82769411  3.69191633</code></pre>
-<div class="sourceCode" id="cb984"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb984-1"><a href="model-diagnostics.html#cb984-1" aria-hidden="true" tabindex="-1"></a><span class="fu">rstandard</span>(model_3)</span></code></pre></div>
+<div class="sourceCode" id="cb984"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb984-1"><a href="model-diagnostics.html#cb984-1" tabindex="-1"></a><span class="fu">rstandard</span>(model_3)</span></code></pre></div>
 <pre><code>##           1           2           3           4           5           6 
 ##  1.41302755 -0.02555591  0.26980722  0.18535382 -0.16873216 -0.67141143 
 ##           7           8           9          10          11 
 ## -0.01157256 -1.12656475 -0.18882474 -1.63206526  2.70453408</code></pre>
-<div class="sourceCode" id="cb986"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb986-1"><a href="model-diagnostics.html#cb986-1" aria-hidden="true" tabindex="-1"></a><span class="fu">rstandard</span>(model_3)[<span class="fu">abs</span>(<span class="fu">rstandard</span>(model_3)) <span class="sc">&gt;</span> <span class="dv">2</span>]</span></code></pre></div>
+<div class="sourceCode" id="cb986"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb986-1"><a href="model-diagnostics.html#cb986-1" tabindex="-1"></a><span class="fu">rstandard</span>(model_3)[<span class="fu">abs</span>(<span class="fu">rstandard</span>(model_3)) <span class="sc">&gt;</span> <span class="dv">2</span>]</span></code></pre></div>
 <pre><code>##       11 
 ## 2.704534</code></pre>
 <p>In the last plot, we see that the 11th point, the added point, is a large standardized residual.</p>
@@ -1124,13 +1124,13 @@ <h3><span class="header-section-number">13.3.3</span> Influence<a href="model-di
 <li>Plot Three: high leverage, large residual.</li>
 </ul>
 <p>We’ll now directly check if each of these is influential.</p>
-<div class="sourceCode" id="cb988"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb988-1"><a href="model-diagnostics.html#cb988-1" aria-hidden="true" tabindex="-1"></a><span class="fu">cooks.distance</span>(model_1)[<span class="dv">11</span>] <span class="sc">&gt;</span> <span class="dv">4</span> <span class="sc">/</span> <span class="fu">length</span>(<span class="fu">cooks.distance</span>(model_1))</span></code></pre></div>
+<div class="sourceCode" id="cb988"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb988-1"><a href="model-diagnostics.html#cb988-1" tabindex="-1"></a><span class="fu">cooks.distance</span>(model_1)[<span class="dv">11</span>] <span class="sc">&gt;</span> <span class="dv">4</span> <span class="sc">/</span> <span class="fu">length</span>(<span class="fu">cooks.distance</span>(model_1))</span></code></pre></div>
 <pre><code>##    11 
 ## FALSE</code></pre>
-<div class="sourceCode" id="cb990"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb990-1"><a href="model-diagnostics.html#cb990-1" aria-hidden="true" tabindex="-1"></a><span class="fu">cooks.distance</span>(model_2)[<span class="dv">11</span>] <span class="sc">&gt;</span> <span class="dv">4</span> <span class="sc">/</span> <span class="fu">length</span>(<span class="fu">cooks.distance</span>(model_2))</span></code></pre></div>
+<div class="sourceCode" id="cb990"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb990-1"><a href="model-diagnostics.html#cb990-1" tabindex="-1"></a><span class="fu">cooks.distance</span>(model_2)[<span class="dv">11</span>] <span class="sc">&gt;</span> <span class="dv">4</span> <span class="sc">/</span> <span class="fu">length</span>(<span class="fu">cooks.distance</span>(model_2))</span></code></pre></div>
 <pre><code>##    11 
 ## FALSE</code></pre>
-<div class="sourceCode" id="cb992"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb992-1"><a href="model-diagnostics.html#cb992-1" aria-hidden="true" tabindex="-1"></a><span class="fu">cooks.distance</span>(model_3)[<span class="dv">11</span>] <span class="sc">&gt;</span> <span class="dv">4</span> <span class="sc">/</span> <span class="fu">length</span>(<span class="fu">cooks.distance</span>(model_3))</span></code></pre></div>
+<div class="sourceCode" id="cb992"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb992-1"><a href="model-diagnostics.html#cb992-1" tabindex="-1"></a><span class="fu">cooks.distance</span>(model_3)[<span class="dv">11</span>] <span class="sc">&gt;</span> <span class="dv">4</span> <span class="sc">/</span> <span class="fu">length</span>(<span class="fu">cooks.distance</span>(model_3))</span></code></pre></div>
 <pre><code>##   11 
 ## TRUE</code></pre>
 <p>And, as expected, the added point in the third plot, with high leverage and a large residual, is considered influential!</p>
@@ -1142,64 +1142,64 @@ <h2><span class="header-section-number">13.4</span> Data Analysis Examples<a hre
 <h3><span class="header-section-number">13.4.1</span> Good Diagnostics<a href="model-diagnostics.html#good-diagnostics" class="anchor-section" aria-label="Anchor link to header"></a></h3>
 <p>Last chapter we fit an additive regression to the <code>mtcars</code> data with <code>mpg</code> as the response and <code>hp</code> and <code>am</code> as predictors. Let’s perform some diagnostics on this model.</p>
 <p>First, fit the model as we did last chapter.</p>
-<div class="sourceCode" id="cb994"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb994-1"><a href="model-diagnostics.html#cb994-1" aria-hidden="true" tabindex="-1"></a>mpg_hp_add <span class="ot">=</span> <span class="fu">lm</span>(mpg <span class="sc">~</span> hp <span class="sc">+</span> am, <span class="at">data =</span> mtcars)</span></code></pre></div>
-<div class="sourceCode" id="cb995"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb995-1"><a href="model-diagnostics.html#cb995-1" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(<span class="fu">fitted</span>(mpg_hp_add), <span class="fu">resid</span>(mpg_hp_add), <span class="at">col =</span> <span class="st">&quot;grey&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>,</span>
-<span id="cb995-2"><a href="model-diagnostics.html#cb995-2" aria-hidden="true" tabindex="-1"></a>     <span class="at">xlab =</span> <span class="st">&quot;Fitted&quot;</span>, <span class="at">ylab =</span> <span class="st">&quot;Residual&quot;</span>,</span>
-<span id="cb995-3"><a href="model-diagnostics.html#cb995-3" aria-hidden="true" tabindex="-1"></a>     <span class="at">main =</span> <span class="st">&quot;mtcars: Fitted versus Residuals&quot;</span>)</span>
-<span id="cb995-4"><a href="model-diagnostics.html#cb995-4" aria-hidden="true" tabindex="-1"></a><span class="fu">abline</span>(<span class="at">h =</span> <span class="dv">0</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb994"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb994-1"><a href="model-diagnostics.html#cb994-1" tabindex="-1"></a>mpg_hp_add <span class="ot">=</span> <span class="fu">lm</span>(mpg <span class="sc">~</span> hp <span class="sc">+</span> am, <span class="at">data =</span> mtcars)</span></code></pre></div>
+<div class="sourceCode" id="cb995"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb995-1"><a href="model-diagnostics.html#cb995-1" tabindex="-1"></a><span class="fu">plot</span>(<span class="fu">fitted</span>(mpg_hp_add), <span class="fu">resid</span>(mpg_hp_add), <span class="at">col =</span> <span class="st">&quot;grey&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>,</span>
+<span id="cb995-2"><a href="model-diagnostics.html#cb995-2" tabindex="-1"></a>     <span class="at">xlab =</span> <span class="st">&quot;Fitted&quot;</span>, <span class="at">ylab =</span> <span class="st">&quot;Residual&quot;</span>,</span>
+<span id="cb995-3"><a href="model-diagnostics.html#cb995-3" tabindex="-1"></a>     <span class="at">main =</span> <span class="st">&quot;mtcars: Fitted versus Residuals&quot;</span>)</span>
+<span id="cb995-4"><a href="model-diagnostics.html#cb995-4" tabindex="-1"></a><span class="fu">abline</span>(<span class="at">h =</span> <span class="dv">0</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
 <p><img src="diagnostics_files/figure-html/unnamed-chunk-50-1.png" width="672" style="display: block; margin: auto;" /></p>
 <p>The fitted versus residuals plot looks good. We don’t see any obvious pattern, and the variance looks roughly constant. (Maybe a little larger for large fitted values, but not enough to worry about.)</p>
-<div class="sourceCode" id="cb996"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb996-1"><a href="model-diagnostics.html#cb996-1" aria-hidden="true" tabindex="-1"></a><span class="fu">bptest</span>(mpg_hp_add)</span></code></pre></div>
+<div class="sourceCode" id="cb996"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb996-1"><a href="model-diagnostics.html#cb996-1" tabindex="-1"></a><span class="fu">bptest</span>(mpg_hp_add)</span></code></pre></div>
 <pre><code>## 
 ##  studentized Breusch-Pagan test
 ## 
 ## data:  mpg_hp_add
 ## BP = 7.5858, df = 2, p-value = 0.02253</code></pre>
 <p>The Breusch-Pagan test verifies this, at least for a small <span class="math inline">\(\alpha\)</span> value.</p>
-<div class="sourceCode" id="cb998"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb998-1"><a href="model-diagnostics.html#cb998-1" aria-hidden="true" tabindex="-1"></a><span class="fu">qqnorm</span>(<span class="fu">resid</span>(mpg_hp_add), <span class="at">col =</span> <span class="st">&quot;darkgrey&quot;</span>)</span>
-<span id="cb998-2"><a href="model-diagnostics.html#cb998-2" aria-hidden="true" tabindex="-1"></a><span class="fu">qqline</span>(<span class="fu">resid</span>(mpg_hp_add), <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb998"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb998-1"><a href="model-diagnostics.html#cb998-1" tabindex="-1"></a><span class="fu">qqnorm</span>(<span class="fu">resid</span>(mpg_hp_add), <span class="at">col =</span> <span class="st">&quot;darkgrey&quot;</span>)</span>
+<span id="cb998-2"><a href="model-diagnostics.html#cb998-2" tabindex="-1"></a><span class="fu">qqline</span>(<span class="fu">resid</span>(mpg_hp_add), <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
 <p><img src="diagnostics_files/figure-html/unnamed-chunk-52-1.png" width="672" style="display: block; margin: auto;" /></p>
 <p>The Q-Q plot looks extremely good and the Shapiro-Wilk test agrees.</p>
-<div class="sourceCode" id="cb999"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb999-1"><a href="model-diagnostics.html#cb999-1" aria-hidden="true" tabindex="-1"></a><span class="fu">shapiro.test</span>(<span class="fu">resid</span>(mpg_hp_add))</span></code></pre></div>
+<div class="sourceCode" id="cb999"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb999-1"><a href="model-diagnostics.html#cb999-1" tabindex="-1"></a><span class="fu">shapiro.test</span>(<span class="fu">resid</span>(mpg_hp_add))</span></code></pre></div>
 <pre><code>## 
 ##  Shapiro-Wilk normality test
 ## 
 ## data:  resid(mpg_hp_add)
 ## W = 0.96485, p-value = 0.3706</code></pre>
-<div class="sourceCode" id="cb1001"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1001-1"><a href="model-diagnostics.html#cb1001-1" aria-hidden="true" tabindex="-1"></a><span class="fu">sum</span>(<span class="fu">hatvalues</span>(mpg_hp_add) <span class="sc">&gt;</span> <span class="dv">2</span> <span class="sc">*</span> <span class="fu">mean</span>(<span class="fu">hatvalues</span>(mpg_hp_add)))</span></code></pre></div>
+<div class="sourceCode" id="cb1001"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1001-1"><a href="model-diagnostics.html#cb1001-1" tabindex="-1"></a><span class="fu">sum</span>(<span class="fu">hatvalues</span>(mpg_hp_add) <span class="sc">&gt;</span> <span class="dv">2</span> <span class="sc">*</span> <span class="fu">mean</span>(<span class="fu">hatvalues</span>(mpg_hp_add)))</span></code></pre></div>
 <pre><code>## [1] 2</code></pre>
 <p>We see that there are two points of large leverage.</p>
-<div class="sourceCode" id="cb1003"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1003-1"><a href="model-diagnostics.html#cb1003-1" aria-hidden="true" tabindex="-1"></a><span class="fu">sum</span>(<span class="fu">abs</span>(<span class="fu">rstandard</span>(mpg_hp_add)) <span class="sc">&gt;</span> <span class="dv">2</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1003"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1003-1"><a href="model-diagnostics.html#cb1003-1" tabindex="-1"></a><span class="fu">sum</span>(<span class="fu">abs</span>(<span class="fu">rstandard</span>(mpg_hp_add)) <span class="sc">&gt;</span> <span class="dv">2</span>)</span></code></pre></div>
 <pre><code>## [1] 1</code></pre>
 <p>There is also one point with a large residual. Do these result in any points that are considered influential?</p>
-<div class="sourceCode" id="cb1005"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1005-1"><a href="model-diagnostics.html#cb1005-1" aria-hidden="true" tabindex="-1"></a>cd_mpg_hp_add <span class="ot">=</span> <span class="fu">cooks.distance</span>(mpg_hp_add)</span>
-<span id="cb1005-2"><a href="model-diagnostics.html#cb1005-2" aria-hidden="true" tabindex="-1"></a><span class="fu">sum</span>(cd_mpg_hp_add <span class="sc">&gt;</span> <span class="dv">4</span> <span class="sc">/</span> <span class="fu">length</span>(cd_mpg_hp_add))</span></code></pre></div>
+<div class="sourceCode" id="cb1005"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1005-1"><a href="model-diagnostics.html#cb1005-1" tabindex="-1"></a>cd_mpg_hp_add <span class="ot">=</span> <span class="fu">cooks.distance</span>(mpg_hp_add)</span>
+<span id="cb1005-2"><a href="model-diagnostics.html#cb1005-2" tabindex="-1"></a><span class="fu">sum</span>(cd_mpg_hp_add <span class="sc">&gt;</span> <span class="dv">4</span> <span class="sc">/</span> <span class="fu">length</span>(cd_mpg_hp_add))</span></code></pre></div>
 <pre><code>## [1] 2</code></pre>
-<div class="sourceCode" id="cb1007"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1007-1"><a href="model-diagnostics.html#cb1007-1" aria-hidden="true" tabindex="-1"></a>large_cd_mpg <span class="ot">=</span> cd_mpg_hp_add <span class="sc">&gt;</span> <span class="dv">4</span> <span class="sc">/</span> <span class="fu">length</span>(cd_mpg_hp_add)</span>
-<span id="cb1007-2"><a href="model-diagnostics.html#cb1007-2" aria-hidden="true" tabindex="-1"></a>cd_mpg_hp_add[large_cd_mpg]</span></code></pre></div>
+<div class="sourceCode" id="cb1007"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1007-1"><a href="model-diagnostics.html#cb1007-1" tabindex="-1"></a>large_cd_mpg <span class="ot">=</span> cd_mpg_hp_add <span class="sc">&gt;</span> <span class="dv">4</span> <span class="sc">/</span> <span class="fu">length</span>(cd_mpg_hp_add)</span>
+<span id="cb1007-2"><a href="model-diagnostics.html#cb1007-2" tabindex="-1"></a>cd_mpg_hp_add[large_cd_mpg]</span></code></pre></div>
 <pre><code>## Toyota Corolla  Maserati Bora 
 ##      0.1772555      0.3447994</code></pre>
 <p>We find two influential points. Interestingly, they are <strong>very</strong> different cars.</p>
-<div class="sourceCode" id="cb1009"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1009-1"><a href="model-diagnostics.html#cb1009-1" aria-hidden="true" tabindex="-1"></a><span class="fu">coef</span>(mpg_hp_add)</span></code></pre></div>
+<div class="sourceCode" id="cb1009"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1009-1"><a href="model-diagnostics.html#cb1009-1" tabindex="-1"></a><span class="fu">coef</span>(mpg_hp_add)</span></code></pre></div>
 <pre><code>## (Intercept)          hp          am 
 ##  26.5849137  -0.0588878   5.2770853</code></pre>
 <p>Since the diagnostics looked good, there isn’t much need to worry about these two points, but let’s see how much the coefficients change if we remove them.</p>
-<div class="sourceCode" id="cb1011"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1011-1"><a href="model-diagnostics.html#cb1011-1" aria-hidden="true" tabindex="-1"></a>mpg_hp_add_fix <span class="ot">=</span> <span class="fu">lm</span>(mpg <span class="sc">~</span> hp <span class="sc">+</span> am,</span>
-<span id="cb1011-2"><a href="model-diagnostics.html#cb1011-2" aria-hidden="true" tabindex="-1"></a>                    <span class="at">data =</span> mtcars,</span>
-<span id="cb1011-3"><a href="model-diagnostics.html#cb1011-3" aria-hidden="true" tabindex="-1"></a>                    <span class="at">subset =</span> cd_mpg_hp_add <span class="sc">&lt;=</span> <span class="dv">4</span> <span class="sc">/</span> <span class="fu">length</span>(cd_mpg_hp_add))</span>
-<span id="cb1011-4"><a href="model-diagnostics.html#cb1011-4" aria-hidden="true" tabindex="-1"></a><span class="fu">coef</span>(mpg_hp_add_fix)</span></code></pre></div>
+<div class="sourceCode" id="cb1011"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1011-1"><a href="model-diagnostics.html#cb1011-1" tabindex="-1"></a>mpg_hp_add_fix <span class="ot">=</span> <span class="fu">lm</span>(mpg <span class="sc">~</span> hp <span class="sc">+</span> am,</span>
+<span id="cb1011-2"><a href="model-diagnostics.html#cb1011-2" tabindex="-1"></a>                    <span class="at">data =</span> mtcars,</span>
+<span id="cb1011-3"><a href="model-diagnostics.html#cb1011-3" tabindex="-1"></a>                    <span class="at">subset =</span> cd_mpg_hp_add <span class="sc">&lt;=</span> <span class="dv">4</span> <span class="sc">/</span> <span class="fu">length</span>(cd_mpg_hp_add))</span>
+<span id="cb1011-4"><a href="model-diagnostics.html#cb1011-4" tabindex="-1"></a><span class="fu">coef</span>(mpg_hp_add_fix)</span></code></pre></div>
 <pre><code>## (Intercept)          hp          am 
 ## 27.22190933 -0.06286249  4.29765867</code></pre>
 <p>It seems there isn’t much of a change in the coefficients as a result of removing the supposed influential points. Notice we did not create a new dataset to accomplish this. We instead used the <code>subset</code> argument to <code>lm()</code>. Think about what the code <code>cd_mpg_hp_add &lt;= 4 / length(cd_mpg_hp_add)</code> does here.</p>
-<div class="sourceCode" id="cb1013"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1013-1"><a href="model-diagnostics.html#cb1013-1" aria-hidden="true" tabindex="-1"></a><span class="fu">par</span>(<span class="at">mfrow =</span> <span class="fu">c</span>(<span class="dv">2</span>, <span class="dv">2</span>))</span>
-<span id="cb1013-2"><a href="model-diagnostics.html#cb1013-2" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(mpg_hp_add)</span></code></pre></div>
+<div class="sourceCode" id="cb1013"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1013-1"><a href="model-diagnostics.html#cb1013-1" tabindex="-1"></a><span class="fu">par</span>(<span class="at">mfrow =</span> <span class="fu">c</span>(<span class="dv">2</span>, <span class="dv">2</span>))</span>
+<span id="cb1013-2"><a href="model-diagnostics.html#cb1013-2" tabindex="-1"></a><span class="fu">plot</span>(mpg_hp_add)</span></code></pre></div>
 <p><img src="diagnostics_files/figure-html/unnamed-chunk-59-1.png" width="768" style="display: block; margin: auto;" /></p>
 <p>Notice that calling <code>plot()</code> on a variable which stores an object created by <code>lm()</code> outputs four diagnostic plots by default. Use <code>?plot.lm</code> to learn more. The first two should already be familiar.</p>
 </div>
 <div id="suspect-diagnostics" class="section level3 hasAnchor" number="13.4.2">
 <h3><span class="header-section-number">13.4.2</span> Suspect Diagnostics<a href="model-diagnostics.html#suspect-diagnostics" class="anchor-section" aria-label="Anchor link to header"></a></h3>
 <p>Let’s consider the model <code>big_model</code> from last chapter which was fit to the <code>autompg</code> dataset. It used <code>mpg</code> as the response, and considered many interaction terms between the predictors <code>disp</code>, <code>hp</code>, and <code>domestic</code>.</p>
-<div class="sourceCode" id="cb1014"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1014-1"><a href="model-diagnostics.html#cb1014-1" aria-hidden="true" tabindex="-1"></a><span class="fu">str</span>(autompg)</span></code></pre></div>
+<div class="sourceCode" id="cb1014"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1014-1"><a href="model-diagnostics.html#cb1014-1" tabindex="-1"></a><span class="fu">str</span>(autompg)</span></code></pre></div>
 <pre><code>## &#39;data.frame&#39;:    383 obs. of  9 variables:
 ##  $ mpg     : num  18 15 18 16 17 15 14 14 14 15 ...
 ##  $ cyl     : Factor w/ 3 levels &quot;4&quot;,&quot;6&quot;,&quot;8&quot;: 3 3 3 3 3 3 3 3 3 3 ...
@@ -1210,28 +1210,28 @@ <h3><span class="header-section-number">13.4.2</span> Suspect Diagnostics<a href
 ##  $ year    : int  70 70 70 70 70 70 70 70 70 70 ...
 ##  $ origin  : int  1 1 1 1 1 1 1 1 1 1 ...
 ##  $ domestic: num  1 1 1 1 1 1 1 1 1 1 ...</code></pre>
-<div class="sourceCode" id="cb1016"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1016-1"><a href="model-diagnostics.html#cb1016-1" aria-hidden="true" tabindex="-1"></a>big_model <span class="ot">=</span> <span class="fu">lm</span>(mpg <span class="sc">~</span> disp <span class="sc">*</span> hp <span class="sc">*</span> domestic, <span class="at">data =</span> autompg)</span></code></pre></div>
-<div class="sourceCode" id="cb1017"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1017-1"><a href="model-diagnostics.html#cb1017-1" aria-hidden="true" tabindex="-1"></a><span class="fu">qqnorm</span>(<span class="fu">resid</span>(big_model), <span class="at">col =</span> <span class="st">&quot;darkgrey&quot;</span>)</span>
-<span id="cb1017-2"><a href="model-diagnostics.html#cb1017-2" aria-hidden="true" tabindex="-1"></a><span class="fu">qqline</span>(<span class="fu">resid</span>(big_model), <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1016"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1016-1"><a href="model-diagnostics.html#cb1016-1" tabindex="-1"></a>big_model <span class="ot">=</span> <span class="fu">lm</span>(mpg <span class="sc">~</span> disp <span class="sc">*</span> hp <span class="sc">*</span> domestic, <span class="at">data =</span> autompg)</span></code></pre></div>
+<div class="sourceCode" id="cb1017"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1017-1"><a href="model-diagnostics.html#cb1017-1" tabindex="-1"></a><span class="fu">qqnorm</span>(<span class="fu">resid</span>(big_model), <span class="at">col =</span> <span class="st">&quot;darkgrey&quot;</span>)</span>
+<span id="cb1017-2"><a href="model-diagnostics.html#cb1017-2" tabindex="-1"></a><span class="fu">qqline</span>(<span class="fu">resid</span>(big_model), <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
 <p><img src="diagnostics_files/figure-html/unnamed-chunk-63-1.png" width="672" style="display: block; margin: auto;" /></p>
-<div class="sourceCode" id="cb1018"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1018-1"><a href="model-diagnostics.html#cb1018-1" aria-hidden="true" tabindex="-1"></a><span class="fu">shapiro.test</span>(<span class="fu">resid</span>(big_model))</span></code></pre></div>
+<div class="sourceCode" id="cb1018"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1018-1"><a href="model-diagnostics.html#cb1018-1" tabindex="-1"></a><span class="fu">shapiro.test</span>(<span class="fu">resid</span>(big_model))</span></code></pre></div>
 <pre><code>## 
 ##  Shapiro-Wilk normality test
 ## 
 ## data:  resid(big_model)
 ## W = 0.96161, p-value = 1.824e-08</code></pre>
 <p>Here both the Q-Q plot, and the Shapiro-Wilk test suggest that the normality assumption is violated.</p>
-<div class="sourceCode" id="cb1020"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1020-1"><a href="model-diagnostics.html#cb1020-1" aria-hidden="true" tabindex="-1"></a>big_mod_cd <span class="ot">=</span> <span class="fu">cooks.distance</span>(big_model)</span>
-<span id="cb1020-2"><a href="model-diagnostics.html#cb1020-2" aria-hidden="true" tabindex="-1"></a><span class="fu">sum</span>(big_mod_cd <span class="sc">&gt;</span> <span class="dv">4</span> <span class="sc">/</span> <span class="fu">length</span>(big_mod_cd))</span></code></pre></div>
+<div class="sourceCode" id="cb1020"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1020-1"><a href="model-diagnostics.html#cb1020-1" tabindex="-1"></a>big_mod_cd <span class="ot">=</span> <span class="fu">cooks.distance</span>(big_model)</span>
+<span id="cb1020-2"><a href="model-diagnostics.html#cb1020-2" tabindex="-1"></a><span class="fu">sum</span>(big_mod_cd <span class="sc">&gt;</span> <span class="dv">4</span> <span class="sc">/</span> <span class="fu">length</span>(big_mod_cd))</span></code></pre></div>
 <pre><code>## [1] 31</code></pre>
 <p>Here, we find 31, so perhaps removing them will help!</p>
-<div class="sourceCode" id="cb1022"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1022-1"><a href="model-diagnostics.html#cb1022-1" aria-hidden="true" tabindex="-1"></a>big_model_fix <span class="ot">=</span> <span class="fu">lm</span>(mpg <span class="sc">~</span> disp <span class="sc">*</span> hp <span class="sc">*</span> domestic,</span>
-<span id="cb1022-2"><a href="model-diagnostics.html#cb1022-2" aria-hidden="true" tabindex="-1"></a>                   <span class="at">data =</span> autompg,</span>
-<span id="cb1022-3"><a href="model-diagnostics.html#cb1022-3" aria-hidden="true" tabindex="-1"></a>                   <span class="at">subset =</span> big_mod_cd <span class="sc">&lt;</span> <span class="dv">4</span> <span class="sc">/</span> <span class="fu">length</span>(big_mod_cd))</span>
-<span id="cb1022-4"><a href="model-diagnostics.html#cb1022-4" aria-hidden="true" tabindex="-1"></a><span class="fu">qqnorm</span>(<span class="fu">resid</span>(big_model_fix), <span class="at">col =</span> <span class="st">&quot;grey&quot;</span>)</span>
-<span id="cb1022-5"><a href="model-diagnostics.html#cb1022-5" aria-hidden="true" tabindex="-1"></a><span class="fu">qqline</span>(<span class="fu">resid</span>(big_model_fix), <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1022"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1022-1"><a href="model-diagnostics.html#cb1022-1" tabindex="-1"></a>big_model_fix <span class="ot">=</span> <span class="fu">lm</span>(mpg <span class="sc">~</span> disp <span class="sc">*</span> hp <span class="sc">*</span> domestic,</span>
+<span id="cb1022-2"><a href="model-diagnostics.html#cb1022-2" tabindex="-1"></a>                   <span class="at">data =</span> autompg,</span>
+<span id="cb1022-3"><a href="model-diagnostics.html#cb1022-3" tabindex="-1"></a>                   <span class="at">subset =</span> big_mod_cd <span class="sc">&lt;</span> <span class="dv">4</span> <span class="sc">/</span> <span class="fu">length</span>(big_mod_cd))</span>
+<span id="cb1022-4"><a href="model-diagnostics.html#cb1022-4" tabindex="-1"></a><span class="fu">qqnorm</span>(<span class="fu">resid</span>(big_model_fix), <span class="at">col =</span> <span class="st">&quot;grey&quot;</span>)</span>
+<span id="cb1022-5"><a href="model-diagnostics.html#cb1022-5" tabindex="-1"></a><span class="fu">qqline</span>(<span class="fu">resid</span>(big_model_fix), <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
 <p><img src="diagnostics_files/figure-html/unnamed-chunk-65-1.png" width="672" style="display: block; margin: auto;" /></p>
-<div class="sourceCode" id="cb1023"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1023-1"><a href="model-diagnostics.html#cb1023-1" aria-hidden="true" tabindex="-1"></a><span class="fu">shapiro.test</span>(<span class="fu">resid</span>(big_model_fix))</span></code></pre></div>
+<div class="sourceCode" id="cb1023"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1023-1"><a href="model-diagnostics.html#cb1023-1" tabindex="-1"></a><span class="fu">shapiro.test</span>(<span class="fu">resid</span>(big_model_fix))</span></code></pre></div>
 <pre><code>## 
 ##  Shapiro-Wilk normality test
 ## 
@@ -1247,7 +1247,7 @@ <h2><span class="header-section-number">13.5</span> <code>R</code> Markdown<a hr
 <ul>
 <li><a href="diagnostics.Rmd" target="_blank"><code>diagnostics.Rmd</code></a></li>
 </ul>
-<p>The file was created using <code>R</code> version <code>4.3.2</code>.</p>
+<p>The file was created using <code>R</code> version <code>4.4.1</code>.</p>
 
 </div>
 </div>
diff --git a/multiple-linear-regression.html b/multiple-linear-regression.html
index 5eb3ec9..f9ee24b 100644
--- a/multiple-linear-regression.html
+++ b/multiple-linear-regression.html
@@ -6,7 +6,7 @@
   <meta http-equiv="X-UA-Compatible" content="IE=edge" />
   <title>Chapter 9 Multiple Linear Regression | Applied Statistics with R</title>
   <meta name="description" content="Chapter 9 Multiple Linear Regression | Applied Statistics with R" />
-  <meta name="generator" content="bookdown 0.37 and GitBook 2.6.7" />
+  <meta name="generator" content="bookdown 0.40 and GitBook 2.6.7" />
 
   <meta property="og:title" content="Chapter 9 Multiple Linear Regression | Applied Statistics with R" />
   <meta property="og:type" content="book" />
@@ -55,7 +55,7 @@
 
 <style type="text/css">
 pre > code.sourceCode { white-space: pre; position: relative; }
-pre > code.sourceCode > span { display: inline-block; line-height: 1.25; }
+pre > code.sourceCode > span { line-height: 1.25; }
 pre > code.sourceCode > span:empty { height: 1.2em; }
 .sourceCode { overflow: visible; }
 code.sourceCode > span { color: inherit; text-decoration: inherit; }
@@ -484,26 +484,26 @@ <h1><span class="header-section-number">Chapter 9</span> Multiple Linear Regress
 \]</span></p>
 <p>where <span class="math inline">\(\epsilon_i \sim N(0, \sigma^2)\)</span>.</p>
 <p>However, it is rarely the case that a dataset will have a single predictor variable. It is also rarely the case that a response variable will only depend on a single variable. So in this chapter, we will extend our current linear model to allow a response to depend on <em>multiple</em> predictors.</p>
-<div class="sourceCode" id="cb591"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb591-1"><a href="multiple-linear-regression.html#cb591-1" aria-hidden="true" tabindex="-1"></a><span class="co"># read the data from the web</span></span>
-<span id="cb591-2"><a href="multiple-linear-regression.html#cb591-2" aria-hidden="true" tabindex="-1"></a>autompg <span class="ot">=</span> <span class="fu">read.table</span>(</span>
-<span id="cb591-3"><a href="multiple-linear-regression.html#cb591-3" aria-hidden="true" tabindex="-1"></a>  <span class="st">&quot;http://archive.ics.uci.edu/ml/machine-learning-databases/auto-mpg/auto-mpg.data&quot;</span>,</span>
-<span id="cb591-4"><a href="multiple-linear-regression.html#cb591-4" aria-hidden="true" tabindex="-1"></a>  <span class="at">quote =</span> <span class="st">&quot;</span><span class="sc">\&quot;</span><span class="st">&quot;</span>,</span>
-<span id="cb591-5"><a href="multiple-linear-regression.html#cb591-5" aria-hidden="true" tabindex="-1"></a>  <span class="at">comment.char =</span> <span class="st">&quot;&quot;</span>,</span>
-<span id="cb591-6"><a href="multiple-linear-regression.html#cb591-6" aria-hidden="true" tabindex="-1"></a>  <span class="at">stringsAsFactors =</span> <span class="cn">FALSE</span>)</span>
-<span id="cb591-7"><a href="multiple-linear-regression.html#cb591-7" aria-hidden="true" tabindex="-1"></a><span class="co"># give the dataframe headers</span></span>
-<span id="cb591-8"><a href="multiple-linear-regression.html#cb591-8" aria-hidden="true" tabindex="-1"></a><span class="fu">colnames</span>(autompg) <span class="ot">=</span> <span class="fu">c</span>(<span class="st">&quot;mpg&quot;</span>, <span class="st">&quot;cyl&quot;</span>, <span class="st">&quot;disp&quot;</span>, <span class="st">&quot;hp&quot;</span>, <span class="st">&quot;wt&quot;</span>, <span class="st">&quot;acc&quot;</span>, <span class="st">&quot;year&quot;</span>, <span class="st">&quot;origin&quot;</span>, <span class="st">&quot;name&quot;</span>)</span>
-<span id="cb591-9"><a href="multiple-linear-regression.html#cb591-9" aria-hidden="true" tabindex="-1"></a><span class="co"># remove missing data, which is stored as &quot;?&quot;</span></span>
-<span id="cb591-10"><a href="multiple-linear-regression.html#cb591-10" aria-hidden="true" tabindex="-1"></a>autompg <span class="ot">=</span> <span class="fu">subset</span>(autompg, autompg<span class="sc">$</span>hp <span class="sc">!=</span> <span class="st">&quot;?&quot;</span>)</span>
-<span id="cb591-11"><a href="multiple-linear-regression.html#cb591-11" aria-hidden="true" tabindex="-1"></a><span class="co"># remove the plymouth reliant, as it causes some issues</span></span>
-<span id="cb591-12"><a href="multiple-linear-regression.html#cb591-12" aria-hidden="true" tabindex="-1"></a>autompg <span class="ot">=</span> <span class="fu">subset</span>(autompg, autompg<span class="sc">$</span>name <span class="sc">!=</span> <span class="st">&quot;plymouth reliant&quot;</span>)</span>
-<span id="cb591-13"><a href="multiple-linear-regression.html#cb591-13" aria-hidden="true" tabindex="-1"></a><span class="co"># give the dataset row names, based on the engine, year and name</span></span>
-<span id="cb591-14"><a href="multiple-linear-regression.html#cb591-14" aria-hidden="true" tabindex="-1"></a><span class="fu">rownames</span>(autompg) <span class="ot">=</span> <span class="fu">paste</span>(autompg<span class="sc">$</span>cyl, <span class="st">&quot;cylinder&quot;</span>, autompg<span class="sc">$</span>year, autompg<span class="sc">$</span>name)</span>
-<span id="cb591-15"><a href="multiple-linear-regression.html#cb591-15" aria-hidden="true" tabindex="-1"></a><span class="co"># remove the variable for name, as well as origin</span></span>
-<span id="cb591-16"><a href="multiple-linear-regression.html#cb591-16" aria-hidden="true" tabindex="-1"></a>autompg <span class="ot">=</span> <span class="fu">subset</span>(autompg, <span class="at">select =</span> <span class="fu">c</span>(<span class="st">&quot;mpg&quot;</span>, <span class="st">&quot;cyl&quot;</span>, <span class="st">&quot;disp&quot;</span>, <span class="st">&quot;hp&quot;</span>, <span class="st">&quot;wt&quot;</span>, <span class="st">&quot;acc&quot;</span>, <span class="st">&quot;year&quot;</span>))</span>
-<span id="cb591-17"><a href="multiple-linear-regression.html#cb591-17" aria-hidden="true" tabindex="-1"></a><span class="co"># change horsepower from character to numeric</span></span>
-<span id="cb591-18"><a href="multiple-linear-regression.html#cb591-18" aria-hidden="true" tabindex="-1"></a>autompg<span class="sc">$</span>hp <span class="ot">=</span> <span class="fu">as.numeric</span>(autompg<span class="sc">$</span>hp)</span>
-<span id="cb591-19"><a href="multiple-linear-regression.html#cb591-19" aria-hidden="true" tabindex="-1"></a><span class="co"># check final structure of data</span></span>
-<span id="cb591-20"><a href="multiple-linear-regression.html#cb591-20" aria-hidden="true" tabindex="-1"></a><span class="fu">str</span>(autompg)</span></code></pre></div>
+<div class="sourceCode" id="cb591"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb591-1"><a href="multiple-linear-regression.html#cb591-1" tabindex="-1"></a><span class="co"># read the data from the web</span></span>
+<span id="cb591-2"><a href="multiple-linear-regression.html#cb591-2" tabindex="-1"></a>autompg <span class="ot">=</span> <span class="fu">read.table</span>(</span>
+<span id="cb591-3"><a href="multiple-linear-regression.html#cb591-3" tabindex="-1"></a>  <span class="st">&quot;http://archive.ics.uci.edu/ml/machine-learning-databases/auto-mpg/auto-mpg.data&quot;</span>,</span>
+<span id="cb591-4"><a href="multiple-linear-regression.html#cb591-4" tabindex="-1"></a>  <span class="at">quote =</span> <span class="st">&quot;</span><span class="sc">\&quot;</span><span class="st">&quot;</span>,</span>
+<span id="cb591-5"><a href="multiple-linear-regression.html#cb591-5" tabindex="-1"></a>  <span class="at">comment.char =</span> <span class="st">&quot;&quot;</span>,</span>
+<span id="cb591-6"><a href="multiple-linear-regression.html#cb591-6" tabindex="-1"></a>  <span class="at">stringsAsFactors =</span> <span class="cn">FALSE</span>)</span>
+<span id="cb591-7"><a href="multiple-linear-regression.html#cb591-7" tabindex="-1"></a><span class="co"># give the dataframe headers</span></span>
+<span id="cb591-8"><a href="multiple-linear-regression.html#cb591-8" tabindex="-1"></a><span class="fu">colnames</span>(autompg) <span class="ot">=</span> <span class="fu">c</span>(<span class="st">&quot;mpg&quot;</span>, <span class="st">&quot;cyl&quot;</span>, <span class="st">&quot;disp&quot;</span>, <span class="st">&quot;hp&quot;</span>, <span class="st">&quot;wt&quot;</span>, <span class="st">&quot;acc&quot;</span>, <span class="st">&quot;year&quot;</span>, <span class="st">&quot;origin&quot;</span>, <span class="st">&quot;name&quot;</span>)</span>
+<span id="cb591-9"><a href="multiple-linear-regression.html#cb591-9" tabindex="-1"></a><span class="co"># remove missing data, which is stored as &quot;?&quot;</span></span>
+<span id="cb591-10"><a href="multiple-linear-regression.html#cb591-10" tabindex="-1"></a>autompg <span class="ot">=</span> <span class="fu">subset</span>(autompg, autompg<span class="sc">$</span>hp <span class="sc">!=</span> <span class="st">&quot;?&quot;</span>)</span>
+<span id="cb591-11"><a href="multiple-linear-regression.html#cb591-11" tabindex="-1"></a><span class="co"># remove the plymouth reliant, as it causes some issues</span></span>
+<span id="cb591-12"><a href="multiple-linear-regression.html#cb591-12" tabindex="-1"></a>autompg <span class="ot">=</span> <span class="fu">subset</span>(autompg, autompg<span class="sc">$</span>name <span class="sc">!=</span> <span class="st">&quot;plymouth reliant&quot;</span>)</span>
+<span id="cb591-13"><a href="multiple-linear-regression.html#cb591-13" tabindex="-1"></a><span class="co"># give the dataset row names, based on the engine, year and name</span></span>
+<span id="cb591-14"><a href="multiple-linear-regression.html#cb591-14" tabindex="-1"></a><span class="fu">rownames</span>(autompg) <span class="ot">=</span> <span class="fu">paste</span>(autompg<span class="sc">$</span>cyl, <span class="st">&quot;cylinder&quot;</span>, autompg<span class="sc">$</span>year, autompg<span class="sc">$</span>name)</span>
+<span id="cb591-15"><a href="multiple-linear-regression.html#cb591-15" tabindex="-1"></a><span class="co"># remove the variable for name, as well as origin</span></span>
+<span id="cb591-16"><a href="multiple-linear-regression.html#cb591-16" tabindex="-1"></a>autompg <span class="ot">=</span> <span class="fu">subset</span>(autompg, <span class="at">select =</span> <span class="fu">c</span>(<span class="st">&quot;mpg&quot;</span>, <span class="st">&quot;cyl&quot;</span>, <span class="st">&quot;disp&quot;</span>, <span class="st">&quot;hp&quot;</span>, <span class="st">&quot;wt&quot;</span>, <span class="st">&quot;acc&quot;</span>, <span class="st">&quot;year&quot;</span>))</span>
+<span id="cb591-17"><a href="multiple-linear-regression.html#cb591-17" tabindex="-1"></a><span class="co"># change horsepower from character to numeric</span></span>
+<span id="cb591-18"><a href="multiple-linear-regression.html#cb591-18" tabindex="-1"></a>autompg<span class="sc">$</span>hp <span class="ot">=</span> <span class="fu">as.numeric</span>(autompg<span class="sc">$</span>hp)</span>
+<span id="cb591-19"><a href="multiple-linear-regression.html#cb591-19" tabindex="-1"></a><span class="co"># check final structure of data</span></span>
+<span id="cb591-20"><a href="multiple-linear-regression.html#cb591-20" tabindex="-1"></a><span class="fu">str</span>(autompg)</span></code></pre></div>
 <pre><code>## &#39;data.frame&#39;:    390 obs. of  7 variables:
 ##  $ mpg : num  18 15 18 16 17 15 14 14 14 15 ...
 ##  $ cyl : int  8 8 8 8 8 8 8 8 8 8 ...
@@ -545,8 +545,8 @@ <h1><span class="header-section-number">Chapter 9</span> Multiple Linear Regress
 \end{aligned}
 \]</span></p>
 <p>We now have three equations and three variables, which we could solve, or we could simply let <code>R</code> solve for us.</p>
-<div class="sourceCode" id="cb593"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb593-1"><a href="multiple-linear-regression.html#cb593-1" aria-hidden="true" tabindex="-1"></a>mpg_model <span class="ot">=</span> <span class="fu">lm</span>(mpg <span class="sc">~</span> wt <span class="sc">+</span> year, <span class="at">data =</span> autompg)</span>
-<span id="cb593-2"><a href="multiple-linear-regression.html#cb593-2" aria-hidden="true" tabindex="-1"></a><span class="fu">coef</span>(mpg_model)</span></code></pre></div>
+<div class="sourceCode" id="cb593"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb593-1"><a href="multiple-linear-regression.html#cb593-1" tabindex="-1"></a>mpg_model <span class="ot">=</span> <span class="fu">lm</span>(mpg <span class="sc">~</span> wt <span class="sc">+</span> year, <span class="at">data =</span> autompg)</span>
+<span id="cb593-2"><a href="multiple-linear-regression.html#cb593-2" tabindex="-1"></a><span class="fu">coef</span>(mpg_model)</span></code></pre></div>
 <pre><code>##   (Intercept)            wt          year 
 ## -14.637641945  -0.006634876   0.761401955</code></pre>
 <p><span class="math display">\[
@@ -653,17 +653,17 @@ <h2><span class="header-section-number">9.1</span> Matrix Approach to Regression
 \hat{\beta} = \left(  X^\top X  \right)^{-1}X^\top y
 \]</span></p>
 <p>To verify that this is what <code>R</code> has done for us in the case of two predictors, we create an <span class="math inline">\(X\)</span> matrix. Note that the first column is all 1s, and the remaining columns contain the data.</p>
-<div class="sourceCode" id="cb595"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb595-1"><a href="multiple-linear-regression.html#cb595-1" aria-hidden="true" tabindex="-1"></a>n <span class="ot">=</span> <span class="fu">nrow</span>(autompg)</span>
-<span id="cb595-2"><a href="multiple-linear-regression.html#cb595-2" aria-hidden="true" tabindex="-1"></a>p <span class="ot">=</span> <span class="fu">length</span>(<span class="fu">coef</span>(mpg_model))</span>
-<span id="cb595-3"><a href="multiple-linear-regression.html#cb595-3" aria-hidden="true" tabindex="-1"></a>X <span class="ot">=</span> <span class="fu">cbind</span>(<span class="fu">rep</span>(<span class="dv">1</span>, n), autompg<span class="sc">$</span>wt, autompg<span class="sc">$</span>year)</span>
-<span id="cb595-4"><a href="multiple-linear-regression.html#cb595-4" aria-hidden="true" tabindex="-1"></a>y <span class="ot">=</span> autompg<span class="sc">$</span>mpg</span>
-<span id="cb595-5"><a href="multiple-linear-regression.html#cb595-5" aria-hidden="true" tabindex="-1"></a></span>
-<span id="cb595-6"><a href="multiple-linear-regression.html#cb595-6" aria-hidden="true" tabindex="-1"></a>(<span class="at">beta_hat =</span> <span class="fu">solve</span>(<span class="fu">t</span>(X) <span class="sc">%*%</span> X) <span class="sc">%*%</span> <span class="fu">t</span>(X) <span class="sc">%*%</span> y)</span></code></pre></div>
+<div class="sourceCode" id="cb595"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb595-1"><a href="multiple-linear-regression.html#cb595-1" tabindex="-1"></a>n <span class="ot">=</span> <span class="fu">nrow</span>(autompg)</span>
+<span id="cb595-2"><a href="multiple-linear-regression.html#cb595-2" tabindex="-1"></a>p <span class="ot">=</span> <span class="fu">length</span>(<span class="fu">coef</span>(mpg_model))</span>
+<span id="cb595-3"><a href="multiple-linear-regression.html#cb595-3" tabindex="-1"></a>X <span class="ot">=</span> <span class="fu">cbind</span>(<span class="fu">rep</span>(<span class="dv">1</span>, n), autompg<span class="sc">$</span>wt, autompg<span class="sc">$</span>year)</span>
+<span id="cb595-4"><a href="multiple-linear-regression.html#cb595-4" tabindex="-1"></a>y <span class="ot">=</span> autompg<span class="sc">$</span>mpg</span>
+<span id="cb595-5"><a href="multiple-linear-regression.html#cb595-5" tabindex="-1"></a></span>
+<span id="cb595-6"><a href="multiple-linear-regression.html#cb595-6" tabindex="-1"></a>(<span class="at">beta_hat =</span> <span class="fu">solve</span>(<span class="fu">t</span>(X) <span class="sc">%*%</span> X) <span class="sc">%*%</span> <span class="fu">t</span>(X) <span class="sc">%*%</span> y)</span></code></pre></div>
 <pre><code>##               [,1]
 ## [1,] -14.637641945
 ## [2,]  -0.006634876
 ## [3,]   0.761401955</code></pre>
-<div class="sourceCode" id="cb597"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb597-1"><a href="multiple-linear-regression.html#cb597-1" aria-hidden="true" tabindex="-1"></a><span class="fu">coef</span>(mpg_model)</span></code></pre></div>
+<div class="sourceCode" id="cb597"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb597-1"><a href="multiple-linear-regression.html#cb597-1" tabindex="-1"></a><span class="fu">coef</span>(mpg_model)</span></code></pre></div>
 <pre><code>##   (Intercept)            wt          year 
 ## -14.637641945  -0.006634876   0.761401955</code></pre>
 <p><span class="math display">\[
@@ -705,21 +705,21 @@ <h2><span class="header-section-number">9.1</span> Matrix Approach to Regression
 s_e = \sqrt{\frac{\sum_{i=1}^n (y_i - \hat{y}_i)^2}{n - p}}.
 \]</span></p>
 <p>In <code>R</code>, we could directly access <span class="math inline">\(s_e\)</span> for a fitted model, as we have seen before.</p>
-<div class="sourceCode" id="cb599"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb599-1"><a href="multiple-linear-regression.html#cb599-1" aria-hidden="true" tabindex="-1"></a><span class="fu">summary</span>(mpg_model)<span class="sc">$</span>sigma</span></code></pre></div>
+<div class="sourceCode" id="cb599"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb599-1"><a href="multiple-linear-regression.html#cb599-1" tabindex="-1"></a><span class="fu">summary</span>(mpg_model)<span class="sc">$</span>sigma</span></code></pre></div>
 <pre><code>## [1] 3.431367</code></pre>
 <p>And we can now verify that our math above is indeed calculating the same quantities.</p>
-<div class="sourceCode" id="cb601"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb601-1"><a href="multiple-linear-regression.html#cb601-1" aria-hidden="true" tabindex="-1"></a>y_hat <span class="ot">=</span> X <span class="sc">%*%</span> <span class="fu">solve</span>(<span class="fu">t</span>(X) <span class="sc">%*%</span> X) <span class="sc">%*%</span> <span class="fu">t</span>(X) <span class="sc">%*%</span> y</span>
-<span id="cb601-2"><a href="multiple-linear-regression.html#cb601-2" aria-hidden="true" tabindex="-1"></a>e     <span class="ot">=</span> y <span class="sc">-</span> y_hat</span>
-<span id="cb601-3"><a href="multiple-linear-regression.html#cb601-3" aria-hidden="true" tabindex="-1"></a><span class="fu">sqrt</span>(<span class="fu">t</span>(e) <span class="sc">%*%</span> e <span class="sc">/</span> (n <span class="sc">-</span> p))</span></code></pre></div>
+<div class="sourceCode" id="cb601"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb601-1"><a href="multiple-linear-regression.html#cb601-1" tabindex="-1"></a>y_hat <span class="ot">=</span> X <span class="sc">%*%</span> <span class="fu">solve</span>(<span class="fu">t</span>(X) <span class="sc">%*%</span> X) <span class="sc">%*%</span> <span class="fu">t</span>(X) <span class="sc">%*%</span> y</span>
+<span id="cb601-2"><a href="multiple-linear-regression.html#cb601-2" tabindex="-1"></a>e     <span class="ot">=</span> y <span class="sc">-</span> y_hat</span>
+<span id="cb601-3"><a href="multiple-linear-regression.html#cb601-3" tabindex="-1"></a><span class="fu">sqrt</span>(<span class="fu">t</span>(e) <span class="sc">%*%</span> e <span class="sc">/</span> (n <span class="sc">-</span> p))</span></code></pre></div>
 <pre><code>##          [,1]
 ## [1,] 3.431367</code></pre>
-<div class="sourceCode" id="cb603"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb603-1"><a href="multiple-linear-regression.html#cb603-1" aria-hidden="true" tabindex="-1"></a><span class="fu">sqrt</span>(<span class="fu">sum</span>((y <span class="sc">-</span> y_hat) <span class="sc">^</span> <span class="dv">2</span>) <span class="sc">/</span> (n <span class="sc">-</span> p))</span></code></pre></div>
+<div class="sourceCode" id="cb603"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb603-1"><a href="multiple-linear-regression.html#cb603-1" tabindex="-1"></a><span class="fu">sqrt</span>(<span class="fu">sum</span>((y <span class="sc">-</span> y_hat) <span class="sc">^</span> <span class="dv">2</span>) <span class="sc">/</span> (n <span class="sc">-</span> p))</span></code></pre></div>
 <pre><code>## [1] 3.431367</code></pre>
 </div>
 <div id="sampling-distribution" class="section level2 hasAnchor" number="9.2">
 <h2><span class="header-section-number">9.2</span> Sampling Distribution<a href="multiple-linear-regression.html#sampling-distribution" class="anchor-section" aria-label="Anchor link to header"></a></h2>
 <p>As we can see in the output below, the results of calling <code>summary()</code> are similar to SLR, but there are some differences, most obviously a new row for the added predictor variable.</p>
-<div class="sourceCode" id="cb605"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb605-1"><a href="multiple-linear-regression.html#cb605-1" aria-hidden="true" tabindex="-1"></a><span class="fu">summary</span>(mpg_model)</span></code></pre></div>
+<div class="sourceCode" id="cb605"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb605-1"><a href="multiple-linear-regression.html#cb605-1" tabindex="-1"></a><span class="fu">summary</span>(mpg_model)</span></code></pre></div>
 <pre><code>## 
 ## Call:
 ## lm(formula = mpg ~ wt + year, data = autompg)
@@ -835,7 +835,7 @@ <h3><span class="header-section-number">9.2.1</span> Single Parameter Tests<a hr
 H_0: \beta_1 = 0 \quad \text{vs} \quad H_1: \beta_1 \neq 0
 \]</span></p>
 <p>can be found in the <code>summary()</code> output, in particular:</p>
-<div class="sourceCode" id="cb607"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb607-1"><a href="multiple-linear-regression.html#cb607-1" aria-hidden="true" tabindex="-1"></a><span class="fu">summary</span>(mpg_model)<span class="sc">$</span>coef</span></code></pre></div>
+<div class="sourceCode" id="cb607"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb607-1"><a href="multiple-linear-regression.html#cb607-1" tabindex="-1"></a><span class="fu">summary</span>(mpg_model)<span class="sc">$</span>coef</span></code></pre></div>
 <pre><code>##                  Estimate   Std. Error    t value      Pr(&gt;|t|)
 ## (Intercept) -14.637641945 4.0233913563  -3.638135  3.118311e-04
 ## wt           -0.006634876 0.0002148504 -30.881372 1.850466e-106
@@ -863,7 +863,7 @@ <h3><span class="header-section-number">9.2.2</span> Confidence Intervals<a href
 \hat{\beta}_j \pm t_{\alpha/2, n - p} \cdot s_e\sqrt{C_{jj}}
 \]</span></p>
 <p>We can find these in <code>R</code> using the same method as before. Now there will simply be additional rows for the additional <span class="math inline">\(\beta\)</span>.</p>
-<div class="sourceCode" id="cb609"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb609-1"><a href="multiple-linear-regression.html#cb609-1" aria-hidden="true" tabindex="-1"></a><span class="fu">confint</span>(mpg_model, <span class="at">level =</span> <span class="fl">0.99</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb609"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb609-1"><a href="multiple-linear-regression.html#cb609-1" tabindex="-1"></a><span class="fu">confint</span>(mpg_model, <span class="at">level =</span> <span class="fl">0.99</span>)</span></code></pre></div>
 <pre><code>##                     0.5 %       99.5 %
 ## (Intercept) -25.052563681 -4.222720208
 ## wt           -0.007191036 -0.006078716
@@ -904,35 +904,35 @@ <h3><span class="header-section-number">9.2.3</span> Confidence Intervals for Me
 \hat{y}(x_0) \pm t_{\alpha/2, n - p} \cdot s_e \sqrt{x_{0}^\top\left(X^\top X\right)^{-1}x_{0}}
 \]</span></p>
 <p>The math has changed a bit, but the process in <code>R</code> remains almost identical. Here, we create a data frame for two additional cars. One car that weighs 3500 pounds produced in 1976, as well as a second car that weighs 5000 pounds which was produced in 1981.</p>
-<div class="sourceCode" id="cb611"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb611-1"><a href="multiple-linear-regression.html#cb611-1" aria-hidden="true" tabindex="-1"></a>new_cars <span class="ot">=</span> <span class="fu">data.frame</span>(<span class="at">wt =</span> <span class="fu">c</span>(<span class="dv">3500</span>, <span class="dv">5000</span>), <span class="at">year =</span> <span class="fu">c</span>(<span class="dv">76</span>, <span class="dv">81</span>))</span>
-<span id="cb611-2"><a href="multiple-linear-regression.html#cb611-2" aria-hidden="true" tabindex="-1"></a>new_cars</span></code></pre></div>
+<div class="sourceCode" id="cb611"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb611-1"><a href="multiple-linear-regression.html#cb611-1" tabindex="-1"></a>new_cars <span class="ot">=</span> <span class="fu">data.frame</span>(<span class="at">wt =</span> <span class="fu">c</span>(<span class="dv">3500</span>, <span class="dv">5000</span>), <span class="at">year =</span> <span class="fu">c</span>(<span class="dv">76</span>, <span class="dv">81</span>))</span>
+<span id="cb611-2"><a href="multiple-linear-regression.html#cb611-2" tabindex="-1"></a>new_cars</span></code></pre></div>
 <pre><code>##     wt year
 ## 1 3500   76
 ## 2 5000   81</code></pre>
 <p>We can then use the <code>predict()</code> function with <code>interval = "confidence"</code> to obtain intervals for the mean fuel efficiency for both new cars. Again, it is important to make the data passed to <code>newdata</code> a data frame, so that <code>R</code> knows which values are for which variables.</p>
-<div class="sourceCode" id="cb613"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb613-1"><a href="multiple-linear-regression.html#cb613-1" aria-hidden="true" tabindex="-1"></a><span class="fu">predict</span>(mpg_model, <span class="at">newdata =</span> new_cars, <span class="at">interval =</span> <span class="st">&quot;confidence&quot;</span>, <span class="at">level =</span> <span class="fl">0.99</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb613"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb613-1"><a href="multiple-linear-regression.html#cb613-1" tabindex="-1"></a><span class="fu">predict</span>(mpg_model, <span class="at">newdata =</span> new_cars, <span class="at">interval =</span> <span class="st">&quot;confidence&quot;</span>, <span class="at">level =</span> <span class="fl">0.99</span>)</span></code></pre></div>
 <pre><code>##        fit     lwr      upr
 ## 1 20.00684 19.4712 20.54248
 ## 2 13.86154 12.3341 15.38898</code></pre>
 <p><code>R</code> then reports the estimate <span class="math inline">\(\hat{y}(x_0)\)</span> (<code>fit</code>) for each, as well as the lower (<code>lwr</code>) and upper (<code>upr</code>) bounds for the interval at a desired level (99%).</p>
 <p>A word of caution here: one of these estimates is good while one is suspect.</p>
-<div class="sourceCode" id="cb615"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb615-1"><a href="multiple-linear-regression.html#cb615-1" aria-hidden="true" tabindex="-1"></a>new_cars<span class="sc">$</span>wt</span></code></pre></div>
+<div class="sourceCode" id="cb615"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb615-1"><a href="multiple-linear-regression.html#cb615-1" tabindex="-1"></a>new_cars<span class="sc">$</span>wt</span></code></pre></div>
 <pre><code>## [1] 3500 5000</code></pre>
-<div class="sourceCode" id="cb617"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb617-1"><a href="multiple-linear-regression.html#cb617-1" aria-hidden="true" tabindex="-1"></a><span class="fu">range</span>(autompg<span class="sc">$</span>wt)</span></code></pre></div>
+<div class="sourceCode" id="cb617"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb617-1"><a href="multiple-linear-regression.html#cb617-1" tabindex="-1"></a><span class="fu">range</span>(autompg<span class="sc">$</span>wt)</span></code></pre></div>
 <pre><code>## [1] 1613 5140</code></pre>
 <p>Note that both of the weights of the new cars are within the range of observed values.</p>
-<div class="sourceCode" id="cb619"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb619-1"><a href="multiple-linear-regression.html#cb619-1" aria-hidden="true" tabindex="-1"></a>new_cars<span class="sc">$</span>year</span></code></pre></div>
+<div class="sourceCode" id="cb619"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb619-1"><a href="multiple-linear-regression.html#cb619-1" tabindex="-1"></a>new_cars<span class="sc">$</span>year</span></code></pre></div>
 <pre><code>## [1] 76 81</code></pre>
-<div class="sourceCode" id="cb621"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb621-1"><a href="multiple-linear-regression.html#cb621-1" aria-hidden="true" tabindex="-1"></a><span class="fu">range</span>(autompg<span class="sc">$</span>year)</span></code></pre></div>
+<div class="sourceCode" id="cb621"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb621-1"><a href="multiple-linear-regression.html#cb621-1" tabindex="-1"></a><span class="fu">range</span>(autompg<span class="sc">$</span>year)</span></code></pre></div>
 <pre><code>## [1] 70 82</code></pre>
 <p>As are the years of each of the new cars.</p>
-<div class="sourceCode" id="cb623"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb623-1"><a href="multiple-linear-regression.html#cb623-1" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(year <span class="sc">~</span> wt, <span class="at">data =</span> autompg, <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">cex =</span> <span class="fl">1.5</span>)</span>
-<span id="cb623-2"><a href="multiple-linear-regression.html#cb623-2" aria-hidden="true" tabindex="-1"></a><span class="fu">points</span>(new_cars, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">cex =</span> <span class="dv">3</span>, <span class="at">pch =</span> <span class="st">&quot;X&quot;</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb623"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb623-1"><a href="multiple-linear-regression.html#cb623-1" tabindex="-1"></a><span class="fu">plot</span>(year <span class="sc">~</span> wt, <span class="at">data =</span> autompg, <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">cex =</span> <span class="fl">1.5</span>)</span>
+<span id="cb623-2"><a href="multiple-linear-regression.html#cb623-2" tabindex="-1"></a><span class="fu">points</span>(new_cars, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">cex =</span> <span class="dv">3</span>, <span class="at">pch =</span> <span class="st">&quot;X&quot;</span>)</span></code></pre></div>
 <p><img src="mlr_files/figure-html/unnamed-chunk-16-1.png" width="672" style="display: block; margin: auto;" /></p>
 <p>However, we have to consider weight and year together now. And based on the above plot, one of the new cars is within the “blob” of observed values, while the other, the car from 1981 weighing 5000 pounds, is noticeably outside of the observed values. This is a hidden extrapolation which you should be aware of when using multiple regression.</p>
 <p>Shifting gears back to the new data pair that can be reasonably estimated, we do a quick verification of some of the mathematics in <code>R</code>.</p>
-<div class="sourceCode" id="cb624"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb624-1"><a href="multiple-linear-regression.html#cb624-1" aria-hidden="true" tabindex="-1"></a>x0 <span class="ot">=</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">3500</span>, <span class="dv">76</span>)</span>
-<span id="cb624-2"><a href="multiple-linear-regression.html#cb624-2" aria-hidden="true" tabindex="-1"></a>x0 <span class="sc">%*%</span> beta_hat</span></code></pre></div>
+<div class="sourceCode" id="cb624"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb624-1"><a href="multiple-linear-regression.html#cb624-1" tabindex="-1"></a>x0 <span class="ot">=</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">3500</span>, <span class="dv">76</span>)</span>
+<span id="cb624-2"><a href="multiple-linear-regression.html#cb624-2" tabindex="-1"></a>x0 <span class="sc">%*%</span> beta_hat</span></code></pre></div>
 <pre><code>##          [,1]
 ## [1,] 20.00684</code></pre>
 <p><span class="math display">\[
@@ -963,13 +963,13 @@ <h3><span class="header-section-number">9.2.3</span> Confidence Intervals for Me
 \end{bmatrix}= 20.0068411
 \]</span></p>
 <p>Also note that, using a particular value for <span class="math inline">\(x_0\)</span>, we can essentially extract certain <span class="math inline">\(\hat{\beta}_j\)</span> values.</p>
-<div class="sourceCode" id="cb626"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb626-1"><a href="multiple-linear-regression.html#cb626-1" aria-hidden="true" tabindex="-1"></a>beta_hat</span></code></pre></div>
+<div class="sourceCode" id="cb626"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb626-1"><a href="multiple-linear-regression.html#cb626-1" tabindex="-1"></a>beta_hat</span></code></pre></div>
 <pre><code>##               [,1]
 ## [1,] -14.637641945
 ## [2,]  -0.006634876
 ## [3,]   0.761401955</code></pre>
-<div class="sourceCode" id="cb628"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb628-1"><a href="multiple-linear-regression.html#cb628-1" aria-hidden="true" tabindex="-1"></a>x0 <span class="ot">=</span> <span class="fu">c</span>(<span class="dv">0</span>, <span class="dv">0</span>, <span class="dv">1</span>)</span>
-<span id="cb628-2"><a href="multiple-linear-regression.html#cb628-2" aria-hidden="true" tabindex="-1"></a>x0 <span class="sc">%*%</span> beta_hat</span></code></pre></div>
+<div class="sourceCode" id="cb628"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb628-1"><a href="multiple-linear-regression.html#cb628-1" tabindex="-1"></a>x0 <span class="ot">=</span> <span class="fu">c</span>(<span class="dv">0</span>, <span class="dv">0</span>, <span class="dv">1</span>)</span>
+<span id="cb628-2"><a href="multiple-linear-regression.html#cb628-2" tabindex="-1"></a>x0 <span class="sc">%*%</span> beta_hat</span></code></pre></div>
 <pre><code>##          [,1]
 ## [1,] 0.761402</code></pre>
 <p>With this in mind, confidence intervals for the individual <span class="math inline">\(\hat{\beta}_j\)</span> are actually a special case of a confidence interval for mean response.</p>
@@ -998,11 +998,11 @@ <h3><span class="header-section-number">9.2.4</span> Prediction Intervals<a href
 <p><span class="math display">\[
 \hat{y}(x_0) \pm t_{\alpha/2, n - p} \cdot s_e \sqrt{1 + x_{0}^\top\left(X^\top X\right)^{-1}x_{0}}
 \]</span></p>
-<div class="sourceCode" id="cb630"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb630-1"><a href="multiple-linear-regression.html#cb630-1" aria-hidden="true" tabindex="-1"></a>new_cars</span></code></pre></div>
+<div class="sourceCode" id="cb630"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb630-1"><a href="multiple-linear-regression.html#cb630-1" tabindex="-1"></a>new_cars</span></code></pre></div>
 <pre><code>##     wt year
 ## 1 3500   76
 ## 2 5000   81</code></pre>
-<div class="sourceCode" id="cb632"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb632-1"><a href="multiple-linear-regression.html#cb632-1" aria-hidden="true" tabindex="-1"></a><span class="fu">predict</span>(mpg_model, <span class="at">newdata =</span> new_cars, <span class="at">interval =</span> <span class="st">&quot;prediction&quot;</span>, <span class="at">level =</span> <span class="fl">0.99</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb632"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb632-1"><a href="multiple-linear-regression.html#cb632-1" tabindex="-1"></a><span class="fu">predict</span>(mpg_model, <span class="at">newdata =</span> new_cars, <span class="at">interval =</span> <span class="st">&quot;prediction&quot;</span>, <span class="at">level =</span> <span class="fl">0.99</span>)</span></code></pre></div>
 <pre><code>##        fit       lwr      upr
 ## 1 20.00684 11.108294 28.90539
 ## 2 13.86154  4.848751 22.87432</code></pre>
@@ -1019,7 +1019,7 @@ <h2><span class="header-section-number">9.3</span> Significance of Regression<a
 \text{SST} = \text{SSE} + \text{SSReg}.
 \]</span></p>
 <p>This means that we can still calculate <span class="math inline">\(R^2\)</span> in the same manner as before, which <code>R</code> continues to do automatically.</p>
-<div class="sourceCode" id="cb634"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb634-1"><a href="multiple-linear-regression.html#cb634-1" aria-hidden="true" tabindex="-1"></a><span class="fu">summary</span>(mpg_model)<span class="sc">$</span>r.squared</span></code></pre></div>
+<div class="sourceCode" id="cb634"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb634-1"><a href="multiple-linear-regression.html#cb634-1" tabindex="-1"></a><span class="fu">summary</span>(mpg_model)<span class="sc">$</span>r.squared</span></code></pre></div>
 <pre><code>## [1] 0.8082355</code></pre>
 <p>The interpretation changes slightly as compared to SLR. In this MLR case, we say that <span class="math inline">\(80.82\%\)</span> of the observed variation in miles per gallon is explained by the linear relationship with the two predictor variables, weight and year.</p>
 <p>In multiple regression, the significance of regression test is</p>
@@ -1102,9 +1102,9 @@ <h2><span class="header-section-number">9.3</span> Significance of Regression<a
 <li><span class="math inline">\(H_1\)</span>: <span class="math inline">\(Y_i = \beta_0 + \beta_1 x_{i1} + \beta_2 x_{i2} + \epsilon_i\)</span></li>
 </ul>
 <p>That is, in the null model, we use neither of the predictors, whereas in the full (alternative) model, at least one of the predictors is useful.</p>
-<div class="sourceCode" id="cb636"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb636-1"><a href="multiple-linear-regression.html#cb636-1" aria-hidden="true" tabindex="-1"></a>null_mpg_model <span class="ot">=</span> <span class="fu">lm</span>(mpg <span class="sc">~</span> <span class="dv">1</span>, <span class="at">data =</span> autompg)</span>
-<span id="cb636-2"><a href="multiple-linear-regression.html#cb636-2" aria-hidden="true" tabindex="-1"></a>full_mpg_model <span class="ot">=</span> <span class="fu">lm</span>(mpg <span class="sc">~</span> wt <span class="sc">+</span> year, <span class="at">data =</span> autompg)</span>
-<span id="cb636-3"><a href="multiple-linear-regression.html#cb636-3" aria-hidden="true" tabindex="-1"></a><span class="fu">anova</span>(null_mpg_model, full_mpg_model)</span></code></pre></div>
+<div class="sourceCode" id="cb636"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb636-1"><a href="multiple-linear-regression.html#cb636-1" tabindex="-1"></a>null_mpg_model <span class="ot">=</span> <span class="fu">lm</span>(mpg <span class="sc">~</span> <span class="dv">1</span>, <span class="at">data =</span> autompg)</span>
+<span id="cb636-2"><a href="multiple-linear-regression.html#cb636-2" tabindex="-1"></a>full_mpg_model <span class="ot">=</span> <span class="fu">lm</span>(mpg <span class="sc">~</span> wt <span class="sc">+</span> year, <span class="at">data =</span> autompg)</span>
+<span id="cb636-3"><a href="multiple-linear-regression.html#cb636-3" tabindex="-1"></a><span class="fu">anova</span>(null_mpg_model, full_mpg_model)</span></code></pre></div>
 <pre><code>## Analysis of Variance Table
 ## 
 ## Model 1: mpg ~ 1
@@ -1115,7 +1115,7 @@ <h2><span class="header-section-number">9.3</span> Significance of Regression<a
 ## ---
 ## Signif. codes:  0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1</code></pre>
 <p>First, notice that <code>R</code> does not display the results in the same manner as the table above. More important than the layout of the table are its contents. We see that the value of the <span class="math inline">\(F\)</span> statistic is 815.55, and the p-value is extremely low, so we reject the null hypothesis at any reasonable <span class="math inline">\(\alpha\)</span> and say that the regression is significant. At least one of <code>wt</code> or <code>year</code> has a useful linear relationship with <code>mpg</code>.</p>
-<div class="sourceCode" id="cb638"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb638-1"><a href="multiple-linear-regression.html#cb638-1" aria-hidden="true" tabindex="-1"></a><span class="fu">summary</span>(mpg_model)</span></code></pre></div>
+<div class="sourceCode" id="cb638"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb638-1"><a href="multiple-linear-regression.html#cb638-1" tabindex="-1"></a><span class="fu">summary</span>(mpg_model)</span></code></pre></div>
 <pre><code>## 
 ## Call:
 ## lm(formula = mpg ~ wt + year, data = autompg)
@@ -1138,23 +1138,23 @@ <h2><span class="header-section-number">9.3</span> Significance of Regression<a
 <p>Notice that the value reported in the row for <code>F-statistic</code> is indeed the <span class="math inline">\(F\)</span> test statistic for the significance of regression test, and additionally it reports the two relevant degrees of freedom.</p>
 <p>Also, note that none of the individual <span class="math inline">\(t\)</span>-tests are equivalent to the <span class="math inline">\(F\)</span>-test as they were in SLR. This equivalence only holds for SLR because the individual test for <span class="math inline">\(\beta_1\)</span> is the same as testing for all non-intercept parameters, since there is only one.</p>
 <p>We can also verify the sums of squares and degrees of freedom directly in <code>R</code>. You should match these to the table from <code>R</code> and use this to match <code>R</code>’s output to the written table above.</p>
-<div class="sourceCode" id="cb640"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb640-1"><a href="multiple-linear-regression.html#cb640-1" aria-hidden="true" tabindex="-1"></a><span class="co"># SSReg</span></span>
-<span id="cb640-2"><a href="multiple-linear-regression.html#cb640-2" aria-hidden="true" tabindex="-1"></a><span class="fu">sum</span>((<span class="fu">fitted</span>(full_mpg_model) <span class="sc">-</span> <span class="fu">fitted</span>(null_mpg_model)) <span class="sc">^</span> <span class="dv">2</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb640"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb640-1"><a href="multiple-linear-regression.html#cb640-1" tabindex="-1"></a><span class="co"># SSReg</span></span>
+<span id="cb640-2"><a href="multiple-linear-regression.html#cb640-2" tabindex="-1"></a><span class="fu">sum</span>((<span class="fu">fitted</span>(full_mpg_model) <span class="sc">-</span> <span class="fu">fitted</span>(null_mpg_model)) <span class="sc">^</span> <span class="dv">2</span>)</span></code></pre></div>
 <pre><code>## [1] 19205.03</code></pre>
-<div class="sourceCode" id="cb642"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb642-1"><a href="multiple-linear-regression.html#cb642-1" aria-hidden="true" tabindex="-1"></a><span class="co"># SSE</span></span>
-<span id="cb642-2"><a href="multiple-linear-regression.html#cb642-2" aria-hidden="true" tabindex="-1"></a><span class="fu">sum</span>(<span class="fu">resid</span>(full_mpg_model) <span class="sc">^</span> <span class="dv">2</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb642"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb642-1"><a href="multiple-linear-regression.html#cb642-1" tabindex="-1"></a><span class="co"># SSE</span></span>
+<span id="cb642-2"><a href="multiple-linear-regression.html#cb642-2" tabindex="-1"></a><span class="fu">sum</span>(<span class="fu">resid</span>(full_mpg_model) <span class="sc">^</span> <span class="dv">2</span>)</span></code></pre></div>
 <pre><code>## [1] 4556.646</code></pre>
-<div class="sourceCode" id="cb644"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb644-1"><a href="multiple-linear-regression.html#cb644-1" aria-hidden="true" tabindex="-1"></a><span class="co"># SST</span></span>
-<span id="cb644-2"><a href="multiple-linear-regression.html#cb644-2" aria-hidden="true" tabindex="-1"></a><span class="fu">sum</span>(<span class="fu">resid</span>(null_mpg_model) <span class="sc">^</span> <span class="dv">2</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb644"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb644-1"><a href="multiple-linear-regression.html#cb644-1" tabindex="-1"></a><span class="co"># SST</span></span>
+<span id="cb644-2"><a href="multiple-linear-regression.html#cb644-2" tabindex="-1"></a><span class="fu">sum</span>(<span class="fu">resid</span>(null_mpg_model) <span class="sc">^</span> <span class="dv">2</span>)</span></code></pre></div>
 <pre><code>## [1] 23761.67</code></pre>
-<div class="sourceCode" id="cb646"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb646-1"><a href="multiple-linear-regression.html#cb646-1" aria-hidden="true" tabindex="-1"></a><span class="co"># Degrees of Freedom: Regression</span></span>
-<span id="cb646-2"><a href="multiple-linear-regression.html#cb646-2" aria-hidden="true" tabindex="-1"></a><span class="fu">length</span>(<span class="fu">coef</span>(full_mpg_model)) <span class="sc">-</span> <span class="fu">length</span>(<span class="fu">coef</span>(null_mpg_model))</span></code></pre></div>
+<div class="sourceCode" id="cb646"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb646-1"><a href="multiple-linear-regression.html#cb646-1" tabindex="-1"></a><span class="co"># Degrees of Freedom: Regression</span></span>
+<span id="cb646-2"><a href="multiple-linear-regression.html#cb646-2" tabindex="-1"></a><span class="fu">length</span>(<span class="fu">coef</span>(full_mpg_model)) <span class="sc">-</span> <span class="fu">length</span>(<span class="fu">coef</span>(null_mpg_model))</span></code></pre></div>
 <pre><code>## [1] 2</code></pre>
-<div class="sourceCode" id="cb648"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb648-1"><a href="multiple-linear-regression.html#cb648-1" aria-hidden="true" tabindex="-1"></a><span class="co"># Degrees of Freedom: Error</span></span>
-<span id="cb648-2"><a href="multiple-linear-regression.html#cb648-2" aria-hidden="true" tabindex="-1"></a><span class="fu">length</span>(<span class="fu">resid</span>(full_mpg_model)) <span class="sc">-</span> <span class="fu">length</span>(<span class="fu">coef</span>(full_mpg_model))</span></code></pre></div>
+<div class="sourceCode" id="cb648"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb648-1"><a href="multiple-linear-regression.html#cb648-1" tabindex="-1"></a><span class="co"># Degrees of Freedom: Error</span></span>
+<span id="cb648-2"><a href="multiple-linear-regression.html#cb648-2" tabindex="-1"></a><span class="fu">length</span>(<span class="fu">resid</span>(full_mpg_model)) <span class="sc">-</span> <span class="fu">length</span>(<span class="fu">coef</span>(full_mpg_model))</span></code></pre></div>
 <pre><code>## [1] 387</code></pre>
-<div class="sourceCode" id="cb650"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb650-1"><a href="multiple-linear-regression.html#cb650-1" aria-hidden="true" tabindex="-1"></a><span class="co"># Degrees of Freedom: Total</span></span>
-<span id="cb650-2"><a href="multiple-linear-regression.html#cb650-2" aria-hidden="true" tabindex="-1"></a><span class="fu">length</span>(<span class="fu">resid</span>(null_mpg_model)) <span class="sc">-</span> <span class="fu">length</span>(<span class="fu">coef</span>(null_mpg_model))</span></code></pre></div>
+<div class="sourceCode" id="cb650"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb650-1"><a href="multiple-linear-regression.html#cb650-1" tabindex="-1"></a><span class="co"># Degrees of Freedom: Total</span></span>
+<span id="cb650-2"><a href="multiple-linear-regression.html#cb650-2" tabindex="-1"></a><span class="fu">length</span>(<span class="fu">resid</span>(null_mpg_model)) <span class="sc">-</span> <span class="fu">length</span>(<span class="fu">coef</span>(null_mpg_model))</span></code></pre></div>
 <pre><code>## [1] 389</code></pre>
 </div>
 <div id="nested-models" class="section level2 hasAnchor" number="9.4">
@@ -1198,14 +1198,14 @@ <h2><span class="header-section-number">9.4</span> Nested Models<a href="multipl
 <td>Diff</td>
 <td><span class="math inline">\(\sum_{i=1}^{n}(\hat{y}_{1i} - \hat{y}_{0i})^2\)</span></td>
 <td><span class="math inline">\(p - q\)</span></td>
-<td><span class="math inline">\(\text{SSD} / (p - q)\)</span></td>
+<td><span class="math inline">\(\text{SSD}   / (p - q)\)</span></td>
 <td><span class="math inline">\(\text{MSD} / \text{MSE}\)</span></td>
 </tr>
 <tr class="even">
 <td>Full</td>
 <td><span class="math inline">\(\sum_{i=1}^{n}(y_i - \hat{y}_{1i})^2\)</span></td>
 <td><span class="math inline">\(n - p\)</span></td>
-<td><span class="math inline">\(\text{SSE} / (n - p)\)</span></td>
+<td><span class="math inline">\(\text{SSE}   / (n - p)\)</span></td>
 <td></td>
 </tr>
 <tr class="odd">
@@ -1222,7 +1222,7 @@ <h2><span class="header-section-number">9.4</span> Nested Models<a href="multipl
 \]</span></p>
 <p>Notice that the row for “Diff” compares the sum of the squared differences of the fitted values. The degrees of freedom is then the difference of the number of <span class="math inline">\(\beta\)</span>-parameters estimated between the two models.</p>
 <p>For example, the <code>autompg</code> dataset has a number of additional variables that we have yet to use.</p>
-<div class="sourceCode" id="cb652"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb652-1"><a href="multiple-linear-regression.html#cb652-1" aria-hidden="true" tabindex="-1"></a><span class="fu">names</span>(autompg)</span></code></pre></div>
+<div class="sourceCode" id="cb652"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb652-1"><a href="multiple-linear-regression.html#cb652-1" tabindex="-1"></a><span class="fu">names</span>(autompg)</span></code></pre></div>
 <pre><code>## [1] &quot;mpg&quot;  &quot;cyl&quot;  &quot;disp&quot; &quot;hp&quot;   &quot;wt&quot;   &quot;acc&quot;  &quot;year&quot;</code></pre>
 <p>We’ll continue to use <code>mpg</code> as the response, but now we will consider two different models.</p>
 <ul>
@@ -1235,10 +1235,10 @@ <h2><span class="header-section-number">9.4</span> Nested Models<a href="multipl
 \]</span></p>
 <p>The alternative is simply that at least one of the <span class="math inline">\(\beta_{j}\)</span> from the null is not 0.</p>
 <p>To perform this test in <code>R</code> we first define both models, then give them to the <code>anova()</code> commands.</p>
-<div class="sourceCode" id="cb654"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb654-1"><a href="multiple-linear-regression.html#cb654-1" aria-hidden="true" tabindex="-1"></a>null_mpg_model <span class="ot">=</span> <span class="fu">lm</span>(mpg <span class="sc">~</span> wt <span class="sc">+</span> year, <span class="at">data =</span> autompg)</span>
-<span id="cb654-2"><a href="multiple-linear-regression.html#cb654-2" aria-hidden="true" tabindex="-1"></a><span class="co">#full_mpg_model = lm(mpg ~ wt + year + cyl + disp + hp + acc, data = autompg)</span></span>
-<span id="cb654-3"><a href="multiple-linear-regression.html#cb654-3" aria-hidden="true" tabindex="-1"></a>full_mpg_model <span class="ot">=</span> <span class="fu">lm</span>(mpg <span class="sc">~</span> ., <span class="at">data =</span> autompg)</span>
-<span id="cb654-4"><a href="multiple-linear-regression.html#cb654-4" aria-hidden="true" tabindex="-1"></a><span class="fu">anova</span>(null_mpg_model, full_mpg_model)</span></code></pre></div>
+<div class="sourceCode" id="cb654"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb654-1"><a href="multiple-linear-regression.html#cb654-1" tabindex="-1"></a>null_mpg_model <span class="ot">=</span> <span class="fu">lm</span>(mpg <span class="sc">~</span> wt <span class="sc">+</span> year, <span class="at">data =</span> autompg)</span>
+<span id="cb654-2"><a href="multiple-linear-regression.html#cb654-2" tabindex="-1"></a><span class="co">#full_mpg_model = lm(mpg ~ wt + year + cyl + disp + hp + acc, data = autompg)</span></span>
+<span id="cb654-3"><a href="multiple-linear-regression.html#cb654-3" tabindex="-1"></a>full_mpg_model <span class="ot">=</span> <span class="fu">lm</span>(mpg <span class="sc">~</span> ., <span class="at">data =</span> autompg)</span>
+<span id="cb654-4"><a href="multiple-linear-regression.html#cb654-4" tabindex="-1"></a><span class="fu">anova</span>(null_mpg_model, full_mpg_model)</span></code></pre></div>
 <pre><code>## Analysis of Variance Table
 ## 
 ## Model 1: mpg ~ wt + year
@@ -1249,23 +1249,23 @@ <h2><span class="header-section-number">9.4</span> Nested Models<a href="multipl
 <p>Here we have used the formula <code>mpg ~ .</code> to define to full model. This is the same as the commented out line. Specifically, this is a common shortcut in <code>R</code> which reads, “model <code>mpg</code> as the response with each of the remaining variables in the data frame as predictors.”</p>
 <p>Here we see that the value of the <span class="math inline">\(F\)</span> statistic is 0.553, and the p-value is very large, so we fail to reject the null hypothesis at any reasonable <span class="math inline">\(\alpha\)</span> and say that none of <code>cyl</code>, <code>disp</code>, <code>hp</code>, and <code>acc</code> are significant with <code>wt</code> and <code>year</code> already in the model.</p>
 <p>Again, we verify the sums of squares and degrees of freedom directly in <code>R</code>. You should match these to the table from <code>R</code>, and use this to match <code>R</code>’s output to the written table above.</p>
-<div class="sourceCode" id="cb656"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb656-1"><a href="multiple-linear-regression.html#cb656-1" aria-hidden="true" tabindex="-1"></a><span class="co"># SSDiff</span></span>
-<span id="cb656-2"><a href="multiple-linear-regression.html#cb656-2" aria-hidden="true" tabindex="-1"></a><span class="fu">sum</span>((<span class="fu">fitted</span>(full_mpg_model) <span class="sc">-</span> <span class="fu">fitted</span>(null_mpg_model)) <span class="sc">^</span> <span class="dv">2</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb656"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb656-1"><a href="multiple-linear-regression.html#cb656-1" tabindex="-1"></a><span class="co"># SSDiff</span></span>
+<span id="cb656-2"><a href="multiple-linear-regression.html#cb656-2" tabindex="-1"></a><span class="fu">sum</span>((<span class="fu">fitted</span>(full_mpg_model) <span class="sc">-</span> <span class="fu">fitted</span>(null_mpg_model)) <span class="sc">^</span> <span class="dv">2</span>)</span></code></pre></div>
 <pre><code>## [1] 26.17981</code></pre>
-<div class="sourceCode" id="cb658"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb658-1"><a href="multiple-linear-regression.html#cb658-1" aria-hidden="true" tabindex="-1"></a><span class="co"># SSE (For Full)</span></span>
-<span id="cb658-2"><a href="multiple-linear-regression.html#cb658-2" aria-hidden="true" tabindex="-1"></a><span class="fu">sum</span>(<span class="fu">resid</span>(full_mpg_model) <span class="sc">^</span> <span class="dv">2</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb658"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb658-1"><a href="multiple-linear-regression.html#cb658-1" tabindex="-1"></a><span class="co"># SSE (For Full)</span></span>
+<span id="cb658-2"><a href="multiple-linear-regression.html#cb658-2" tabindex="-1"></a><span class="fu">sum</span>(<span class="fu">resid</span>(full_mpg_model) <span class="sc">^</span> <span class="dv">2</span>)</span></code></pre></div>
 <pre><code>## [1] 4530.466</code></pre>
-<div class="sourceCode" id="cb660"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb660-1"><a href="multiple-linear-regression.html#cb660-1" aria-hidden="true" tabindex="-1"></a><span class="co"># SST (For Null)</span></span>
-<span id="cb660-2"><a href="multiple-linear-regression.html#cb660-2" aria-hidden="true" tabindex="-1"></a><span class="fu">sum</span>(<span class="fu">resid</span>(null_mpg_model) <span class="sc">^</span> <span class="dv">2</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb660"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb660-1"><a href="multiple-linear-regression.html#cb660-1" tabindex="-1"></a><span class="co"># SST (For Null)</span></span>
+<span id="cb660-2"><a href="multiple-linear-regression.html#cb660-2" tabindex="-1"></a><span class="fu">sum</span>(<span class="fu">resid</span>(null_mpg_model) <span class="sc">^</span> <span class="dv">2</span>)</span></code></pre></div>
 <pre><code>## [1] 4556.646</code></pre>
-<div class="sourceCode" id="cb662"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb662-1"><a href="multiple-linear-regression.html#cb662-1" aria-hidden="true" tabindex="-1"></a><span class="co"># Degrees of Freedom: Diff</span></span>
-<span id="cb662-2"><a href="multiple-linear-regression.html#cb662-2" aria-hidden="true" tabindex="-1"></a><span class="fu">length</span>(<span class="fu">coef</span>(full_mpg_model)) <span class="sc">-</span> <span class="fu">length</span>(<span class="fu">coef</span>(null_mpg_model))</span></code></pre></div>
+<div class="sourceCode" id="cb662"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb662-1"><a href="multiple-linear-regression.html#cb662-1" tabindex="-1"></a><span class="co"># Degrees of Freedom: Diff</span></span>
+<span id="cb662-2"><a href="multiple-linear-regression.html#cb662-2" tabindex="-1"></a><span class="fu">length</span>(<span class="fu">coef</span>(full_mpg_model)) <span class="sc">-</span> <span class="fu">length</span>(<span class="fu">coef</span>(null_mpg_model))</span></code></pre></div>
 <pre><code>## [1] 4</code></pre>
-<div class="sourceCode" id="cb664"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb664-1"><a href="multiple-linear-regression.html#cb664-1" aria-hidden="true" tabindex="-1"></a><span class="co"># Degrees of Freedom: Full</span></span>
-<span id="cb664-2"><a href="multiple-linear-regression.html#cb664-2" aria-hidden="true" tabindex="-1"></a><span class="fu">length</span>(<span class="fu">resid</span>(full_mpg_model)) <span class="sc">-</span> <span class="fu">length</span>(<span class="fu">coef</span>(full_mpg_model))</span></code></pre></div>
+<div class="sourceCode" id="cb664"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb664-1"><a href="multiple-linear-regression.html#cb664-1" tabindex="-1"></a><span class="co"># Degrees of Freedom: Full</span></span>
+<span id="cb664-2"><a href="multiple-linear-regression.html#cb664-2" tabindex="-1"></a><span class="fu">length</span>(<span class="fu">resid</span>(full_mpg_model)) <span class="sc">-</span> <span class="fu">length</span>(<span class="fu">coef</span>(full_mpg_model))</span></code></pre></div>
 <pre><code>## [1] 383</code></pre>
-<div class="sourceCode" id="cb666"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb666-1"><a href="multiple-linear-regression.html#cb666-1" aria-hidden="true" tabindex="-1"></a><span class="co"># Degrees of Freedom: Null</span></span>
-<span id="cb666-2"><a href="multiple-linear-regression.html#cb666-2" aria-hidden="true" tabindex="-1"></a><span class="fu">length</span>(<span class="fu">resid</span>(null_mpg_model)) <span class="sc">-</span> <span class="fu">length</span>(<span class="fu">coef</span>(null_mpg_model))</span></code></pre></div>
+<div class="sourceCode" id="cb666"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb666-1"><a href="multiple-linear-regression.html#cb666-1" tabindex="-1"></a><span class="co"># Degrees of Freedom: Null</span></span>
+<span id="cb666-2"><a href="multiple-linear-regression.html#cb666-2" tabindex="-1"></a><span class="fu">length</span>(<span class="fu">resid</span>(null_mpg_model)) <span class="sc">-</span> <span class="fu">length</span>(<span class="fu">coef</span>(null_mpg_model))</span></code></pre></div>
 <pre><code>## [1] 387</code></pre>
 </div>
 <div id="simulation-1" class="section level2 hasAnchor" number="9.5">
@@ -1275,47 +1275,47 @@ <h2><span class="header-section-number">9.5</span> Simulation<a href="multiple-l
 Y_i = 5 + -2 x_{i1} + 6 x_{i2} + \epsilon_i, \qquad i = 1, 2, \ldots, n
 \]</span></p>
 <p>where <span class="math inline">\(\epsilon_i \sim N(0, \sigma^2 = 16)\)</span>. Here we have two predictors, so <span class="math inline">\(p = 3\)</span>.</p>
-<div class="sourceCode" id="cb668"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb668-1"><a href="multiple-linear-regression.html#cb668-1" aria-hidden="true" tabindex="-1"></a><span class="fu">set.seed</span>(<span class="dv">1337</span>)</span>
-<span id="cb668-2"><a href="multiple-linear-regression.html#cb668-2" aria-hidden="true" tabindex="-1"></a>n <span class="ot">=</span> <span class="dv">100</span> <span class="co"># sample size</span></span>
-<span id="cb668-3"><a href="multiple-linear-regression.html#cb668-3" aria-hidden="true" tabindex="-1"></a>p <span class="ot">=</span> <span class="dv">3</span></span>
-<span id="cb668-4"><a href="multiple-linear-regression.html#cb668-4" aria-hidden="true" tabindex="-1"></a></span>
-<span id="cb668-5"><a href="multiple-linear-regression.html#cb668-5" aria-hidden="true" tabindex="-1"></a>beta_0 <span class="ot">=</span> <span class="dv">5</span></span>
-<span id="cb668-6"><a href="multiple-linear-regression.html#cb668-6" aria-hidden="true" tabindex="-1"></a>beta_1 <span class="ot">=</span> <span class="sc">-</span><span class="dv">2</span></span>
-<span id="cb668-7"><a href="multiple-linear-regression.html#cb668-7" aria-hidden="true" tabindex="-1"></a>beta_2 <span class="ot">=</span> <span class="dv">6</span></span>
-<span id="cb668-8"><a href="multiple-linear-regression.html#cb668-8" aria-hidden="true" tabindex="-1"></a>sigma  <span class="ot">=</span> <span class="dv">4</span></span></code></pre></div>
+<div class="sourceCode" id="cb668"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb668-1"><a href="multiple-linear-regression.html#cb668-1" tabindex="-1"></a><span class="fu">set.seed</span>(<span class="dv">1337</span>)</span>
+<span id="cb668-2"><a href="multiple-linear-regression.html#cb668-2" tabindex="-1"></a>n <span class="ot">=</span> <span class="dv">100</span> <span class="co"># sample size</span></span>
+<span id="cb668-3"><a href="multiple-linear-regression.html#cb668-3" tabindex="-1"></a>p <span class="ot">=</span> <span class="dv">3</span></span>
+<span id="cb668-4"><a href="multiple-linear-regression.html#cb668-4" tabindex="-1"></a></span>
+<span id="cb668-5"><a href="multiple-linear-regression.html#cb668-5" tabindex="-1"></a>beta_0 <span class="ot">=</span> <span class="dv">5</span></span>
+<span id="cb668-6"><a href="multiple-linear-regression.html#cb668-6" tabindex="-1"></a>beta_1 <span class="ot">=</span> <span class="sc">-</span><span class="dv">2</span></span>
+<span id="cb668-7"><a href="multiple-linear-regression.html#cb668-7" tabindex="-1"></a>beta_2 <span class="ot">=</span> <span class="dv">6</span></span>
+<span id="cb668-8"><a href="multiple-linear-regression.html#cb668-8" tabindex="-1"></a>sigma  <span class="ot">=</span> <span class="dv">4</span></span></code></pre></div>
 <p>As is the norm with regression, the <span class="math inline">\(x\)</span> values are considered fixed and known quantities, so we will simulate those first, and they remain the same for the rest of the simulation study. Also note we create an <code>x0</code> which is all <code>1</code>, which we need to create our <code>X</code> matrix. If you look at the matrix formulation of regression, this unit vector of all <code>1</code>s is a “predictor” that puts the intercept into the model. We also calculate the <code>C</code> matrix for later use.</p>
-<div class="sourceCode" id="cb669"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb669-1"><a href="multiple-linear-regression.html#cb669-1" aria-hidden="true" tabindex="-1"></a>x0 <span class="ot">=</span> <span class="fu">rep</span>(<span class="dv">1</span>, n)</span>
-<span id="cb669-2"><a href="multiple-linear-regression.html#cb669-2" aria-hidden="true" tabindex="-1"></a>x1 <span class="ot">=</span> <span class="fu">sample</span>(<span class="fu">seq</span>(<span class="dv">1</span>, <span class="dv">10</span>, <span class="at">length =</span> n))</span>
-<span id="cb669-3"><a href="multiple-linear-regression.html#cb669-3" aria-hidden="true" tabindex="-1"></a>x2 <span class="ot">=</span> <span class="fu">sample</span>(<span class="fu">seq</span>(<span class="dv">1</span>, <span class="dv">10</span>, <span class="at">length =</span> n))</span>
-<span id="cb669-4"><a href="multiple-linear-regression.html#cb669-4" aria-hidden="true" tabindex="-1"></a>X <span class="ot">=</span> <span class="fu">cbind</span>(x0, x1, x2)</span>
-<span id="cb669-5"><a href="multiple-linear-regression.html#cb669-5" aria-hidden="true" tabindex="-1"></a>C <span class="ot">=</span> <span class="fu">solve</span>(<span class="fu">t</span>(X) <span class="sc">%*%</span> X)</span></code></pre></div>
+<div class="sourceCode" id="cb669"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb669-1"><a href="multiple-linear-regression.html#cb669-1" tabindex="-1"></a>x0 <span class="ot">=</span> <span class="fu">rep</span>(<span class="dv">1</span>, n)</span>
+<span id="cb669-2"><a href="multiple-linear-regression.html#cb669-2" tabindex="-1"></a>x1 <span class="ot">=</span> <span class="fu">sample</span>(<span class="fu">seq</span>(<span class="dv">1</span>, <span class="dv">10</span>, <span class="at">length =</span> n))</span>
+<span id="cb669-3"><a href="multiple-linear-regression.html#cb669-3" tabindex="-1"></a>x2 <span class="ot">=</span> <span class="fu">sample</span>(<span class="fu">seq</span>(<span class="dv">1</span>, <span class="dv">10</span>, <span class="at">length =</span> n))</span>
+<span id="cb669-4"><a href="multiple-linear-regression.html#cb669-4" tabindex="-1"></a>X <span class="ot">=</span> <span class="fu">cbind</span>(x0, x1, x2)</span>
+<span id="cb669-5"><a href="multiple-linear-regression.html#cb669-5" tabindex="-1"></a>C <span class="ot">=</span> <span class="fu">solve</span>(<span class="fu">t</span>(X) <span class="sc">%*%</span> X)</span></code></pre></div>
 <p>We then simulate the response according the model above. Lastly, we place the two predictors and response into a data frame. Note that we do <strong>not</strong> place <code>x0</code> in the data frame. This is a result of <code>R</code> adding an intercept by default.</p>
-<div class="sourceCode" id="cb670"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb670-1"><a href="multiple-linear-regression.html#cb670-1" aria-hidden="true" tabindex="-1"></a>eps      <span class="ot">=</span> <span class="fu">rnorm</span>(n, <span class="at">mean =</span> <span class="dv">0</span>, <span class="at">sd =</span> sigma)</span>
-<span id="cb670-2"><a href="multiple-linear-regression.html#cb670-2" aria-hidden="true" tabindex="-1"></a>y        <span class="ot">=</span> beta_0 <span class="sc">+</span> beta_1 <span class="sc">*</span> x1 <span class="sc">+</span> beta_2 <span class="sc">*</span> x2 <span class="sc">+</span> eps</span>
-<span id="cb670-3"><a href="multiple-linear-regression.html#cb670-3" aria-hidden="true" tabindex="-1"></a>sim_data <span class="ot">=</span> <span class="fu">data.frame</span>(x1, x2, y)</span></code></pre></div>
+<div class="sourceCode" id="cb670"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb670-1"><a href="multiple-linear-regression.html#cb670-1" tabindex="-1"></a>eps      <span class="ot">=</span> <span class="fu">rnorm</span>(n, <span class="at">mean =</span> <span class="dv">0</span>, <span class="at">sd =</span> sigma)</span>
+<span id="cb670-2"><a href="multiple-linear-regression.html#cb670-2" tabindex="-1"></a>y        <span class="ot">=</span> beta_0 <span class="sc">+</span> beta_1 <span class="sc">*</span> x1 <span class="sc">+</span> beta_2 <span class="sc">*</span> x2 <span class="sc">+</span> eps</span>
+<span id="cb670-3"><a href="multiple-linear-regression.html#cb670-3" tabindex="-1"></a>sim_data <span class="ot">=</span> <span class="fu">data.frame</span>(x1, x2, y)</span></code></pre></div>
 <p>Plotting this data and fitting the regression produces the following plot.</p>
 <p><img src="mlr_files/figure-html/unnamed-chunk-30-1.png" width="576" style="display: block; margin: auto;" /></p>
 <p>We then calculate</p>
 <p><span class="math display">\[
 \hat{\beta} = \left(  X^\top X  \right)^{-1}X^\top y.
 \]</span></p>
-<div class="sourceCode" id="cb671"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb671-1"><a href="multiple-linear-regression.html#cb671-1" aria-hidden="true" tabindex="-1"></a>(<span class="at">beta_hat =</span> C <span class="sc">%*%</span> <span class="fu">t</span>(X) <span class="sc">%*%</span> y)</span></code></pre></div>
+<div class="sourceCode" id="cb671"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb671-1"><a href="multiple-linear-regression.html#cb671-1" tabindex="-1"></a>(<span class="at">beta_hat =</span> C <span class="sc">%*%</span> <span class="fu">t</span>(X) <span class="sc">%*%</span> y)</span></code></pre></div>
 <pre><code>##         [,1]
 ## x0  7.290735
 ## x1 -2.282176
 ## x2  5.843424</code></pre>
 <p>Notice that these values are the same as the coefficients found using <code>lm()</code> in <code>R</code>.</p>
-<div class="sourceCode" id="cb673"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb673-1"><a href="multiple-linear-regression.html#cb673-1" aria-hidden="true" tabindex="-1"></a><span class="fu">coef</span>(<span class="fu">lm</span>(y <span class="sc">~</span> x1 <span class="sc">+</span> x2, <span class="at">data =</span> sim_data))</span></code></pre></div>
+<div class="sourceCode" id="cb673"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb673-1"><a href="multiple-linear-regression.html#cb673-1" tabindex="-1"></a><span class="fu">coef</span>(<span class="fu">lm</span>(y <span class="sc">~</span> x1 <span class="sc">+</span> x2, <span class="at">data =</span> sim_data))</span></code></pre></div>
 <pre><code>## (Intercept)          x1          x2 
 ##    7.290735   -2.282176    5.843424</code></pre>
 <p>Also, these values are close to what we would expect.</p>
-<div class="sourceCode" id="cb675"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb675-1"><a href="multiple-linear-regression.html#cb675-1" aria-hidden="true" tabindex="-1"></a><span class="fu">c</span>(beta_0, beta_1, beta_2)</span></code></pre></div>
+<div class="sourceCode" id="cb675"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb675-1"><a href="multiple-linear-regression.html#cb675-1" tabindex="-1"></a><span class="fu">c</span>(beta_0, beta_1, beta_2)</span></code></pre></div>
 <pre><code>## [1]  5 -2  6</code></pre>
 <p>We then calculated the fitted values in order to calculate <span class="math inline">\(s_e\)</span>, which we see is the same as the <code>sigma</code> which is returned by <code>summary()</code>.</p>
-<div class="sourceCode" id="cb677"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb677-1"><a href="multiple-linear-regression.html#cb677-1" aria-hidden="true" tabindex="-1"></a>y_hat <span class="ot">=</span> X <span class="sc">%*%</span> beta_hat</span>
-<span id="cb677-2"><a href="multiple-linear-regression.html#cb677-2" aria-hidden="true" tabindex="-1"></a>(<span class="at">s_e =</span> <span class="fu">sqrt</span>(<span class="fu">sum</span>((y <span class="sc">-</span> y_hat) <span class="sc">^</span> <span class="dv">2</span>) <span class="sc">/</span> (n <span class="sc">-</span> p)))</span></code></pre></div>
+<div class="sourceCode" id="cb677"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb677-1"><a href="multiple-linear-regression.html#cb677-1" tabindex="-1"></a>y_hat <span class="ot">=</span> X <span class="sc">%*%</span> beta_hat</span>
+<span id="cb677-2"><a href="multiple-linear-regression.html#cb677-2" tabindex="-1"></a>(<span class="at">s_e =</span> <span class="fu">sqrt</span>(<span class="fu">sum</span>((y <span class="sc">-</span> y_hat) <span class="sc">^</span> <span class="dv">2</span>) <span class="sc">/</span> (n <span class="sc">-</span> p)))</span></code></pre></div>
 <pre><code>## [1] 4.294307</code></pre>
-<div class="sourceCode" id="cb679"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb679-1"><a href="multiple-linear-regression.html#cb679-1" aria-hidden="true" tabindex="-1"></a><span class="fu">summary</span>(<span class="fu">lm</span>(y <span class="sc">~</span> x1 <span class="sc">+</span> x2, <span class="at">data =</span> sim_data))<span class="sc">$</span>sigma</span></code></pre></div>
+<div class="sourceCode" id="cb679"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb679-1"><a href="multiple-linear-regression.html#cb679-1" tabindex="-1"></a><span class="fu">summary</span>(<span class="fu">lm</span>(y <span class="sc">~</span> x1 <span class="sc">+</span> x2, <span class="at">data =</span> sim_data))<span class="sc">$</span>sigma</span></code></pre></div>
 <pre><code>## [1] 4.294307</code></pre>
 <p>So far so good. Everything checks out. Now we will finally simulate from this model repeatedly in order to obtain an empirical distribution of <span class="math inline">\(\hat{\beta}_2\)</span>.</p>
 <p>We expect <span class="math inline">\(\hat{\beta}_2\)</span> to follow a normal distribution,</p>
@@ -1330,54 +1330,54 @@ <h2><span class="header-section-number">9.5</span> Simulation<a href="multiple-l
 \hat{\beta}_2 \sim N\left(\mu = 6, \sigma^2 = 0.0232549  \right).
 \]</span></p>
 <p>Note that <span class="math inline">\(C_{22}\)</span> corresponds to the element in the <strong>third</strong> row and <strong>third</strong> column since <span class="math inline">\(\beta_2\)</span> is the <strong>third</strong> parameter in the model and because <code>R</code> is indexed starting at <code>1</code>. However, we index the <span class="math inline">\(C\)</span> matrix starting at <code>0</code> to match the diagonal elements to the corresponding <span class="math inline">\(\beta_j\)</span>.</p>
-<div class="sourceCode" id="cb681"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb681-1"><a href="multiple-linear-regression.html#cb681-1" aria-hidden="true" tabindex="-1"></a>C[<span class="dv">3</span>, <span class="dv">3</span>]</span></code></pre></div>
+<div class="sourceCode" id="cb681"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb681-1"><a href="multiple-linear-regression.html#cb681-1" tabindex="-1"></a>C[<span class="dv">3</span>, <span class="dv">3</span>]</span></code></pre></div>
 <pre><code>## [1] 0.00145343</code></pre>
-<div class="sourceCode" id="cb683"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb683-1"><a href="multiple-linear-regression.html#cb683-1" aria-hidden="true" tabindex="-1"></a>C[<span class="dv">2</span> <span class="sc">+</span> <span class="dv">1</span>, <span class="dv">2</span> <span class="sc">+</span> <span class="dv">1</span>]</span></code></pre></div>
+<div class="sourceCode" id="cb683"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb683-1"><a href="multiple-linear-regression.html#cb683-1" tabindex="-1"></a>C[<span class="dv">2</span> <span class="sc">+</span> <span class="dv">1</span>, <span class="dv">2</span> <span class="sc">+</span> <span class="dv">1</span>]</span></code></pre></div>
 <pre><code>## [1] 0.00145343</code></pre>
-<div class="sourceCode" id="cb685"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb685-1"><a href="multiple-linear-regression.html#cb685-1" aria-hidden="true" tabindex="-1"></a>sigma <span class="sc">^</span> <span class="dv">2</span> <span class="sc">*</span> C[<span class="dv">2</span> <span class="sc">+</span> <span class="dv">1</span>, <span class="dv">2</span> <span class="sc">+</span> <span class="dv">1</span>]</span></code></pre></div>
+<div class="sourceCode" id="cb685"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb685-1"><a href="multiple-linear-regression.html#cb685-1" tabindex="-1"></a>sigma <span class="sc">^</span> <span class="dv">2</span> <span class="sc">*</span> C[<span class="dv">2</span> <span class="sc">+</span> <span class="dv">1</span>, <span class="dv">2</span> <span class="sc">+</span> <span class="dv">1</span>]</span></code></pre></div>
 <pre><code>## [1] 0.02325487</code></pre>
 <p>We now perform the simulation a large number of times. Each time, we update the <code>y</code> variable in the data frame, leaving the <code>x</code> variables the same. We then fit a model, and store <span class="math inline">\(\hat{\beta}_2\)</span>.</p>
-<div class="sourceCode" id="cb687"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb687-1"><a href="multiple-linear-regression.html#cb687-1" aria-hidden="true" tabindex="-1"></a>num_sims <span class="ot">=</span> <span class="dv">10000</span></span>
-<span id="cb687-2"><a href="multiple-linear-regression.html#cb687-2" aria-hidden="true" tabindex="-1"></a>beta_hat_2 <span class="ot">=</span> <span class="fu">rep</span>(<span class="dv">0</span>, num_sims)</span>
-<span id="cb687-3"><a href="multiple-linear-regression.html#cb687-3" aria-hidden="true" tabindex="-1"></a><span class="cf">for</span>(i <span class="cf">in</span> <span class="dv">1</span><span class="sc">:</span>num_sims) {</span>
-<span id="cb687-4"><a href="multiple-linear-regression.html#cb687-4" aria-hidden="true" tabindex="-1"></a>  eps           <span class="ot">=</span> <span class="fu">rnorm</span>(n, <span class="at">mean =</span> <span class="dv">0</span> , <span class="at">sd =</span> sigma)</span>
-<span id="cb687-5"><a href="multiple-linear-regression.html#cb687-5" aria-hidden="true" tabindex="-1"></a>  sim_data<span class="sc">$</span>y    <span class="ot">=</span> beta_0 <span class="sc">*</span> x0 <span class="sc">+</span> beta_1 <span class="sc">*</span> x1 <span class="sc">+</span> beta_2 <span class="sc">*</span> x2 <span class="sc">+</span> eps</span>
-<span id="cb687-6"><a href="multiple-linear-regression.html#cb687-6" aria-hidden="true" tabindex="-1"></a>  fit           <span class="ot">=</span> <span class="fu">lm</span>(y <span class="sc">~</span> x1 <span class="sc">+</span> x2, <span class="at">data =</span> sim_data)</span>
-<span id="cb687-7"><a href="multiple-linear-regression.html#cb687-7" aria-hidden="true" tabindex="-1"></a>  beta_hat_2[i] <span class="ot">=</span> <span class="fu">coef</span>(fit)[<span class="dv">3</span>]</span>
-<span id="cb687-8"><a href="multiple-linear-regression.html#cb687-8" aria-hidden="true" tabindex="-1"></a>}</span></code></pre></div>
+<div class="sourceCode" id="cb687"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb687-1"><a href="multiple-linear-regression.html#cb687-1" tabindex="-1"></a>num_sims <span class="ot">=</span> <span class="dv">10000</span></span>
+<span id="cb687-2"><a href="multiple-linear-regression.html#cb687-2" tabindex="-1"></a>beta_hat_2 <span class="ot">=</span> <span class="fu">rep</span>(<span class="dv">0</span>, num_sims)</span>
+<span id="cb687-3"><a href="multiple-linear-regression.html#cb687-3" tabindex="-1"></a><span class="cf">for</span>(i <span class="cf">in</span> <span class="dv">1</span><span class="sc">:</span>num_sims) {</span>
+<span id="cb687-4"><a href="multiple-linear-regression.html#cb687-4" tabindex="-1"></a>  eps           <span class="ot">=</span> <span class="fu">rnorm</span>(n, <span class="at">mean =</span> <span class="dv">0</span> , <span class="at">sd =</span> sigma)</span>
+<span id="cb687-5"><a href="multiple-linear-regression.html#cb687-5" tabindex="-1"></a>  sim_data<span class="sc">$</span>y    <span class="ot">=</span> beta_0 <span class="sc">*</span> x0 <span class="sc">+</span> beta_1 <span class="sc">*</span> x1 <span class="sc">+</span> beta_2 <span class="sc">*</span> x2 <span class="sc">+</span> eps</span>
+<span id="cb687-6"><a href="multiple-linear-regression.html#cb687-6" tabindex="-1"></a>  fit           <span class="ot">=</span> <span class="fu">lm</span>(y <span class="sc">~</span> x1 <span class="sc">+</span> x2, <span class="at">data =</span> sim_data)</span>
+<span id="cb687-7"><a href="multiple-linear-regression.html#cb687-7" tabindex="-1"></a>  beta_hat_2[i] <span class="ot">=</span> <span class="fu">coef</span>(fit)[<span class="dv">3</span>]</span>
+<span id="cb687-8"><a href="multiple-linear-regression.html#cb687-8" tabindex="-1"></a>}</span></code></pre></div>
 <p>We then see that the mean of the simulated values is close to the true value of <span class="math inline">\(\beta_2\)</span>.</p>
-<div class="sourceCode" id="cb688"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb688-1"><a href="multiple-linear-regression.html#cb688-1" aria-hidden="true" tabindex="-1"></a><span class="fu">mean</span>(beta_hat_2)</span></code></pre></div>
+<div class="sourceCode" id="cb688"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb688-1"><a href="multiple-linear-regression.html#cb688-1" tabindex="-1"></a><span class="fu">mean</span>(beta_hat_2)</span></code></pre></div>
 <pre><code>## [1] 5.999723</code></pre>
-<div class="sourceCode" id="cb690"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb690-1"><a href="multiple-linear-regression.html#cb690-1" aria-hidden="true" tabindex="-1"></a>beta_2</span></code></pre></div>
+<div class="sourceCode" id="cb690"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb690-1"><a href="multiple-linear-regression.html#cb690-1" tabindex="-1"></a>beta_2</span></code></pre></div>
 <pre><code>## [1] 6</code></pre>
 <p>We also see that the variance of the simulated values is close to the true variance of <span class="math inline">\(\hat{\beta}_2\)</span>.</p>
 <p><span class="math display">\[
 \text{Var}[\hat{\beta}_2] = \sigma^2 \cdot C_{22} = 16 \times 0.0014534 = 0.0232549
 \]</span></p>
-<div class="sourceCode" id="cb692"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb692-1"><a href="multiple-linear-regression.html#cb692-1" aria-hidden="true" tabindex="-1"></a><span class="fu">var</span>(beta_hat_2)</span></code></pre></div>
+<div class="sourceCode" id="cb692"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb692-1"><a href="multiple-linear-regression.html#cb692-1" tabindex="-1"></a><span class="fu">var</span>(beta_hat_2)</span></code></pre></div>
 <pre><code>## [1] 0.02343408</code></pre>
-<div class="sourceCode" id="cb694"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb694-1"><a href="multiple-linear-regression.html#cb694-1" aria-hidden="true" tabindex="-1"></a>sigma <span class="sc">^</span> <span class="dv">2</span> <span class="sc">*</span> C[<span class="dv">2</span> <span class="sc">+</span> <span class="dv">1</span>, <span class="dv">2</span> <span class="sc">+</span> <span class="dv">1</span>]</span></code></pre></div>
+<div class="sourceCode" id="cb694"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb694-1"><a href="multiple-linear-regression.html#cb694-1" tabindex="-1"></a>sigma <span class="sc">^</span> <span class="dv">2</span> <span class="sc">*</span> C[<span class="dv">2</span> <span class="sc">+</span> <span class="dv">1</span>, <span class="dv">2</span> <span class="sc">+</span> <span class="dv">1</span>]</span></code></pre></div>
 <pre><code>## [1] 0.02325487</code></pre>
 <p>The standard deviations found from the simulated data and the parent population are also very close.</p>
-<div class="sourceCode" id="cb696"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb696-1"><a href="multiple-linear-regression.html#cb696-1" aria-hidden="true" tabindex="-1"></a><span class="fu">sd</span>(beta_hat_2)</span></code></pre></div>
+<div class="sourceCode" id="cb696"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb696-1"><a href="multiple-linear-regression.html#cb696-1" tabindex="-1"></a><span class="fu">sd</span>(beta_hat_2)</span></code></pre></div>
 <pre><code>## [1] 0.1530819</code></pre>
-<div class="sourceCode" id="cb698"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb698-1"><a href="multiple-linear-regression.html#cb698-1" aria-hidden="true" tabindex="-1"></a><span class="fu">sqrt</span>(sigma <span class="sc">^</span> <span class="dv">2</span> <span class="sc">*</span> C[<span class="dv">2</span> <span class="sc">+</span> <span class="dv">1</span>, <span class="dv">2</span> <span class="sc">+</span> <span class="dv">1</span>])</span></code></pre></div>
+<div class="sourceCode" id="cb698"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb698-1"><a href="multiple-linear-regression.html#cb698-1" tabindex="-1"></a><span class="fu">sqrt</span>(sigma <span class="sc">^</span> <span class="dv">2</span> <span class="sc">*</span> C[<span class="dv">2</span> <span class="sc">+</span> <span class="dv">1</span>, <span class="dv">2</span> <span class="sc">+</span> <span class="dv">1</span>])</span></code></pre></div>
 <pre><code>## [1] 0.1524955</code></pre>
 <p>Lastly, we plot a histogram of the <em>simulated values</em>, and overlay the <em>true distribution</em>.</p>
-<div class="sourceCode" id="cb700"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb700-1"><a href="multiple-linear-regression.html#cb700-1" aria-hidden="true" tabindex="-1"></a><span class="fu">hist</span>(beta_hat_2, <span class="at">prob =</span> <span class="cn">TRUE</span>, <span class="at">breaks =</span> <span class="dv">20</span>, </span>
-<span id="cb700-2"><a href="multiple-linear-regression.html#cb700-2" aria-hidden="true" tabindex="-1"></a>     <span class="at">xlab =</span> <span class="fu">expression</span>(<span class="fu">hat</span>(beta)[<span class="dv">2</span>]), <span class="at">main =</span> <span class="st">&quot;&quot;</span>, <span class="at">border =</span> <span class="st">&quot;dodgerblue&quot;</span>)</span>
-<span id="cb700-3"><a href="multiple-linear-regression.html#cb700-3" aria-hidden="true" tabindex="-1"></a><span class="fu">curve</span>(<span class="fu">dnorm</span>(x, <span class="at">mean =</span> beta_2, <span class="at">sd =</span> <span class="fu">sqrt</span>(sigma <span class="sc">^</span> <span class="dv">2</span> <span class="sc">*</span> C[<span class="dv">2</span> <span class="sc">+</span> <span class="dv">1</span>, <span class="dv">2</span> <span class="sc">+</span> <span class="dv">1</span>])), </span>
-<span id="cb700-4"><a href="multiple-linear-regression.html#cb700-4" aria-hidden="true" tabindex="-1"></a>      <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">add =</span> <span class="cn">TRUE</span>, <span class="at">lwd =</span> <span class="dv">3</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb700"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb700-1"><a href="multiple-linear-regression.html#cb700-1" tabindex="-1"></a><span class="fu">hist</span>(beta_hat_2, <span class="at">prob =</span> <span class="cn">TRUE</span>, <span class="at">breaks =</span> <span class="dv">20</span>, </span>
+<span id="cb700-2"><a href="multiple-linear-regression.html#cb700-2" tabindex="-1"></a>     <span class="at">xlab =</span> <span class="fu">expression</span>(<span class="fu">hat</span>(beta)[<span class="dv">2</span>]), <span class="at">main =</span> <span class="st">&quot;&quot;</span>, <span class="at">border =</span> <span class="st">&quot;dodgerblue&quot;</span>)</span>
+<span id="cb700-3"><a href="multiple-linear-regression.html#cb700-3" tabindex="-1"></a><span class="fu">curve</span>(<span class="fu">dnorm</span>(x, <span class="at">mean =</span> beta_2, <span class="at">sd =</span> <span class="fu">sqrt</span>(sigma <span class="sc">^</span> <span class="dv">2</span> <span class="sc">*</span> C[<span class="dv">2</span> <span class="sc">+</span> <span class="dv">1</span>, <span class="dv">2</span> <span class="sc">+</span> <span class="dv">1</span>])), </span>
+<span id="cb700-4"><a href="multiple-linear-regression.html#cb700-4" tabindex="-1"></a>      <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">add =</span> <span class="cn">TRUE</span>, <span class="at">lwd =</span> <span class="dv">3</span>)</span></code></pre></div>
 <p><img src="mlr_files/figure-html/unnamed-chunk-40-1.png" width="672" style="display: block; margin: auto;" /></p>
-<p>This looks good! The simulation-based histogram appears to be Normal with mean 6 and spread of about 0.15 as you measure from center to inflection point. That matches really well with the sampling distribution of <span class="math inline">\(\hat{\beta}_2 \sim N\left(\mu = 6, \sigma^2 = 0.0232549 \right)\)</span>.</p>
+<p>This looks good! The simulation-based histogram appears to be Normal with mean 6 and spread of about 0.15 as you measure from center to inflection point. That matches really well with the sampling distribution of <span class="math inline">\(\hat{\beta}_2 \sim N\left(\mu = 6, \sigma^2 = 0.0232549  \right)\)</span>.</p>
 <p>One last check, we verify the <span class="math inline">\(68 - 95 - 99.7\)</span> rule.</p>
-<div class="sourceCode" id="cb701"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb701-1"><a href="multiple-linear-regression.html#cb701-1" aria-hidden="true" tabindex="-1"></a>sd_bh2 <span class="ot">=</span> <span class="fu">sqrt</span>(sigma <span class="sc">^</span> <span class="dv">2</span> <span class="sc">*</span> C[<span class="dv">2</span> <span class="sc">+</span> <span class="dv">1</span>, <span class="dv">2</span> <span class="sc">+</span> <span class="dv">1</span>])</span>
-<span id="cb701-2"><a href="multiple-linear-regression.html#cb701-2" aria-hidden="true" tabindex="-1"></a><span class="co"># We expect these to be: 0.68, 0.95, 0.997</span></span>
-<span id="cb701-3"><a href="multiple-linear-regression.html#cb701-3" aria-hidden="true" tabindex="-1"></a><span class="fu">mean</span>(beta_2 <span class="sc">-</span> <span class="dv">1</span> <span class="sc">*</span> sd_bh2 <span class="sc">&lt;</span> beta_hat_2 <span class="sc">&amp;</span> beta_hat_2 <span class="sc">&lt;</span> beta_2 <span class="sc">+</span> <span class="dv">1</span> <span class="sc">*</span> sd_bh2)</span></code></pre></div>
+<div class="sourceCode" id="cb701"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb701-1"><a href="multiple-linear-regression.html#cb701-1" tabindex="-1"></a>sd_bh2 <span class="ot">=</span> <span class="fu">sqrt</span>(sigma <span class="sc">^</span> <span class="dv">2</span> <span class="sc">*</span> C[<span class="dv">2</span> <span class="sc">+</span> <span class="dv">1</span>, <span class="dv">2</span> <span class="sc">+</span> <span class="dv">1</span>])</span>
+<span id="cb701-2"><a href="multiple-linear-regression.html#cb701-2" tabindex="-1"></a><span class="co"># We expect these to be: 0.68, 0.95, 0.997</span></span>
+<span id="cb701-3"><a href="multiple-linear-regression.html#cb701-3" tabindex="-1"></a><span class="fu">mean</span>(beta_2 <span class="sc">-</span> <span class="dv">1</span> <span class="sc">*</span> sd_bh2 <span class="sc">&lt;</span> beta_hat_2 <span class="sc">&amp;</span> beta_hat_2 <span class="sc">&lt;</span> beta_2 <span class="sc">+</span> <span class="dv">1</span> <span class="sc">*</span> sd_bh2)</span></code></pre></div>
 <pre><code>## [1] 0.6807</code></pre>
-<div class="sourceCode" id="cb703"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb703-1"><a href="multiple-linear-regression.html#cb703-1" aria-hidden="true" tabindex="-1"></a><span class="fu">mean</span>(beta_2 <span class="sc">-</span> <span class="dv">2</span> <span class="sc">*</span> sd_bh2 <span class="sc">&lt;</span> beta_hat_2 <span class="sc">&amp;</span> beta_hat_2 <span class="sc">&lt;</span> beta_2 <span class="sc">+</span> <span class="dv">2</span> <span class="sc">*</span> sd_bh2)</span></code></pre></div>
+<div class="sourceCode" id="cb703"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb703-1"><a href="multiple-linear-regression.html#cb703-1" tabindex="-1"></a><span class="fu">mean</span>(beta_2 <span class="sc">-</span> <span class="dv">2</span> <span class="sc">*</span> sd_bh2 <span class="sc">&lt;</span> beta_hat_2 <span class="sc">&amp;</span> beta_hat_2 <span class="sc">&lt;</span> beta_2 <span class="sc">+</span> <span class="dv">2</span> <span class="sc">*</span> sd_bh2)</span></code></pre></div>
 <pre><code>## [1] 0.9529</code></pre>
-<div class="sourceCode" id="cb705"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb705-1"><a href="multiple-linear-regression.html#cb705-1" aria-hidden="true" tabindex="-1"></a><span class="fu">mean</span>(beta_2 <span class="sc">-</span> <span class="dv">3</span> <span class="sc">*</span> sd_bh2 <span class="sc">&lt;</span> beta_hat_2 <span class="sc">&amp;</span> beta_hat_2 <span class="sc">&lt;</span> beta_2 <span class="sc">+</span> <span class="dv">3</span> <span class="sc">*</span> sd_bh2)</span></code></pre></div>
+<div class="sourceCode" id="cb705"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb705-1"><a href="multiple-linear-regression.html#cb705-1" tabindex="-1"></a><span class="fu">mean</span>(beta_2 <span class="sc">-</span> <span class="dv">3</span> <span class="sc">*</span> sd_bh2 <span class="sc">&lt;</span> beta_hat_2 <span class="sc">&amp;</span> beta_hat_2 <span class="sc">&lt;</span> beta_2 <span class="sc">+</span> <span class="dv">3</span> <span class="sc">*</span> sd_bh2)</span></code></pre></div>
 <pre><code>## [1] 0.9967</code></pre>
 </div>
 <div id="r-markdown-2" class="section level2 hasAnchor" number="9.6">
@@ -1386,7 +1386,7 @@ <h2><span class="header-section-number">9.6</span> <code>R</code> Markdown<a hre
 <ul>
 <li><a href="mlr.Rmd" target="_blank"><code>mlr.Rmd</code></a></li>
 </ul>
-<p>The file was created using <code>R</code> version <code>4.3.2</code>.</p>
+<p>The file was created using <code>R</code> version <code>4.4.1</code>.</p>
 
 </div>
 </div>
diff --git a/prob-and-stat.md b/prob-and-stat.md
index d8f5bfd..1e382c2 100644
--- a/prob-and-stat.md
+++ b/prob-and-stat.md
@@ -29,7 +29,7 @@ For example, consider a random variable $X$ which is $N(\mu = 2, \sigma^2 = 25)$
 To calculate the value of the pdf at `x = 3`, that is, the height of the curve at `x = 3`, use:
 
 
-```r
+``` r
 dnorm(x = 3, mean = 2, sd = 5)
 ```
 
@@ -40,7 +40,7 @@ dnorm(x = 3, mean = 2, sd = 5)
 To calculate the value of the cdf at `x = 3`, that is, $P(X \leq 3)$, the probability that $X$ is less than or equal to `3`, use:
 
 
-```r
+``` r
 pnorm(q = 3, mean = 2, sd = 5)
 ```
 
@@ -51,7 +51,7 @@ pnorm(q = 3, mean = 2, sd = 5)
 Or, to calculate the quantile for probability 0.975, use:
 
 
-```r
+``` r
 qnorm(p = 0.975, mean = 2, sd = 5)
 ```
 
@@ -62,13 +62,13 @@ qnorm(p = 0.975, mean = 2, sd = 5)
 Lastly, to generate a random sample of size `n = 10`, use:
 
 
-```r
+``` r
 rnorm(n = 10, mean = 2, sd = 5)
 ```
 
 ```
-##  [1]  8.0283428  3.1691218 11.4638375  5.8073649 -1.1075880  2.1513125
-##  [7]  0.2073891  6.2238014  8.8360299  0.4046592
+##  [1] -3.4256036  8.7541787 -0.8925227  0.7328776  8.3390382 -0.2870347
+##  [7]  9.4780333 -3.6029543  7.1857806  7.4014080
 ```
 
 These functions exist for many other distributions, including but not limited to:
@@ -84,7 +84,7 @@ These functions exist for many other distributions, including but not limited to
 Where `*` can be `d`, `p`, `q`, and `r`. Each distribution will have its own set of parameters which need to be passed to the functions as arguments. For example, `dbinom()` would not have arguments for `mean` and `sd`, since those are not parameters of the distribution. Instead a binomial distribution is usually parameterized by $n$ and $p$, however `R` chooses to call them something else. To find the names that `R` uses we would use `?dbinom` and see that `R` instead calls the arguments `size` and `prob`. For example:
 
 
-```r
+``` r
 dbinom(x = 6, size = 10, prob = 0.75)
 ```
 
@@ -134,7 +134,7 @@ where $t_{\alpha/2, n-1}$ is the critical value such that $P\left(t>t_{\alpha/2,
 Suppose a grocery store sells "16 ounce" boxes of *Captain Crisp* cereal. A random sample of 9 boxes was taken and weighed. The weight in ounces is stored in the data frame `capt_crisp`.
 
 
-```r
+``` r
 capt_crisp = data.frame(weight = c(15.5, 16.2, 16.1, 15.8, 15.6, 16.0, 15.8, 15.9, 16.2))
 ```
 
@@ -149,7 +149,7 @@ t = \frac{\bar{x} - \mu_{0}}{s / \sqrt{n}}
 The sample mean $\bar{x}$ and the sample standard deviation $s$ can be easily computed using `R`. We also create variables which store the hypothesized mean and the sample size.
 
 
-```r
+``` r
 x_bar = mean(capt_crisp$weight)
 s     = sd(capt_crisp$weight)
 mu_0  = 16
@@ -159,7 +159,7 @@ n     = 9
 We can then easily compute the test statistic.
 
 
-```r
+``` r
 t = (x_bar - mu_0) / (s / sqrt(n))
 t
 ```
@@ -177,7 +177,7 @@ P(t_{8} < -1.2)
 \]
 
 
-```r
+``` r
 pt(t, df = n - 1)
 ```
 
@@ -190,7 +190,7 @@ We now have the p-value of our test, which is greater than our significance leve
 Alternatively, this entire process could have been completed using one line of `R` code.
 
 
-```r
+``` r
 t.test(x = capt_crisp$weight, mu = 16, alternative = c("less"), conf.level = 0.95)
 ```
 
@@ -219,7 +219,7 @@ We supply `R` with the data, the hypothesized value of $\mu$, the alternative, a
 Since the test was one-sided, `R` returned a one-sided confidence interval. If instead we wanted a two-sided interval for the mean weight of boxes of *Captain Crisp* cereal we could modify our code.
 
 
-```r
+``` r
 capt_test_results = t.test(capt_crisp$weight, mu = 16,
                            alternative = c("two.sided"), conf.level = 0.95)
 ```
@@ -227,7 +227,7 @@ capt_test_results = t.test(capt_crisp$weight, mu = 16,
 This time we have stored the results. By doing so, we can directly access portions of the output from `t.test()`. To see what information is available we use the `names()` function.
 
 
-```r
+``` r
 names(capt_test_results)
 ```
 
@@ -239,7 +239,7 @@ names(capt_test_results)
 We are interested in the confidence interval which is stored in `conf.int`.
 
 
-```r
+``` r
 capt_test_results$conf.int
 ```
 
@@ -252,7 +252,7 @@ capt_test_results$conf.int
 Let's check this interval "by hand." The one piece of information we are missing is the critical value, $t_{\alpha/2, n-1} = t_{8}(0.025)$, which can be calculated in `R` using the `qt()` function.
 
 
-```r
+``` r
 qt(0.975, df = 8)
 ```
 
@@ -267,7 +267,7 @@ So, the 95\% CI for the mean weight of a cereal box is calculated by plugging in
 \]
 
 
-```r
+``` r
 c(mean(capt_crisp$weight) - qt(0.975, df = 8) * sd(capt_crisp$weight) / sqrt(9),
   mean(capt_crisp$weight) + qt(0.975, df = 8) * sd(capt_crisp$weight) / sqrt(9))
 ```
@@ -303,7 +303,7 @@ where $t_{\alpha/2, n+m-2}$ is the critical value such that $P\left(t>t_{\alpha/
 Assume that the distributions of $X$ and $Y$ are $\mathrm{N}(\mu_{1},\sigma^{2})$ and $\mathrm{N}(\mu_{2},\sigma^{2})$, respectively. Given the $n = 6$ observations of $X$,
 
 
-```r
+``` r
 x = c(70, 82, 78, 74, 94, 82)
 n = length(x)
 ```
@@ -311,7 +311,7 @@ n = length(x)
 and the $m = 8$ observations of $Y$,
 
 
-```r
+``` r
 y = c(64, 72, 60, 76, 72, 80, 84, 68)
 m = length(y)
 ```
@@ -321,7 +321,7 @@ we will test $H_{0}: \mu_{1} = \mu_{2}$ versus $H_{1}: \mu_{1} > \mu_{2}$.
 First, note that we can calculate the sample means and standard deviations.
 
 
-```r
+``` r
 x_bar = mean(x)
 s_x   = sd(x)
 y_bar = mean(y)
@@ -335,7 +335,7 @@ s_{p} = \sqrt{\frac{(n-1)s_{x}^{2}+(m-1)s_{y}^{2}}{n+m-2}}
 \]
 
 
-```r
+``` r
 s_p = sqrt(((n - 1) * s_x ^ 2 + (m - 1) * s_y ^ 2) / (n + m - 2))
 ```
 
@@ -346,7 +346,7 @@ t = \frac{(\bar{x}-\bar{y})-\mu_{0}}{s_{p}\sqrt{\frac{1}{n}+\frac{1}{m}}}.
 \]
 
 
-```r
+``` r
 t = ((x_bar - y_bar) - 0) / (s_p * sqrt(1 / n + 1 / m))
 t
 ```
@@ -362,7 +362,7 @@ P(t_{12} > 1.8233692).
 \]
 
 
-```r
+``` r
 1 - pt(t, df = n + m - 2)
 ```
 
@@ -373,7 +373,7 @@ P(t_{12} > 1.8233692).
 But, then again, we could have simply performed this test in one line of `R`.
 
 
-```r
+``` r
 t.test(x, y, alternative = c("greater"), var.equal = TRUE)
 ```
 
@@ -396,7 +396,7 @@ Recall that a two-sample $t$-test can be done with or without an equal variance
 Above we carried out the analysis using two vectors `x` and `y`. In general, we will have a preference for using data frames.
 
 
-```r
+``` r
 t_test_data = data.frame(values = c(x, y),
                          group  = c(rep("A", length(x)), rep("B", length(y))))
 ```
@@ -404,7 +404,7 @@ t_test_data = data.frame(values = c(x, y),
 We now have the data stored in a single variables (`values`) and have created a second variable (`group`) which indicates which "sample" the value belongs to.
 
 
-```r
+``` r
 t_test_data
 ```
 
@@ -429,7 +429,7 @@ t_test_data
 Now to perform the test, we still use the `t.test()` function but with the `~` syntax and a `data` argument.
 
 
-```r
+``` r
 t.test(values ~ group, data = t_test_data,
        alternative = c("greater"), var.equal = TRUE)
 ```
@@ -519,7 +519,7 @@ P(0 < D < 2) = P(D < 2) - P(D < 0).
 This can then be calculated using `R` without a need to first standardize, or use a table.
 
 
-```r
+``` r
 pnorm(2, mean = 1, sd = sqrt(0.32)) - pnorm(0, mean = 1, sd = sqrt(0.32))
 ```
 
@@ -538,7 +538,7 @@ Our strategy will be to repeatedly:
 We will repeat the process a large number of times. Then we will use the distribution of the simulated observations of $d_s$ as an estimate for the true distribution of $D$.
 
 
-```r
+``` r
 set.seed(42)
 num_samples = 10000
 differences = rep(0, num_samples)
@@ -549,7 +549,7 @@ Before starting our `for` loop to perform the operation, we set a seed for repro
 By using `set.seed()` we can reproduce the random results of `rnorm()` each time starting from that line. 
 
 
-```r
+``` r
 for (s in 1:num_samples) {
   x1 = rnorm(n = 25, mean = 6, sd = 2)
   x2 = rnorm(n = 25, mean = 5, sd = 2)
@@ -560,7 +560,7 @@ for (s in 1:num_samples) {
 To estimate $P(0 < D < 2)$ we will find the proportion of values of $d_s$ (among the \ensuremath{10^{4}} values of $d_s$ generated) that are between 0 and 2.
 
 
-```r
+``` r
 mean(0 < differences & differences < 2)
 ```
 
@@ -573,7 +573,7 @@ Recall that above we derived the distribution of $D$ to be $N(\mu = 1, \sigma^2
 If we look at a histogram of the differences, we find that it looks very much like a normal distribution.
 
 
-```r
+``` r
 hist(differences, breaks = 20, 
      main   = "Empirical Distribution of D",
      xlab   = "Simulated Values of D",
@@ -586,7 +586,7 @@ hist(differences, breaks = 20,
 Also the sample mean and variance are very close to what we would expect.
 
 
-```r
+``` r
 mean(differences)
 ```
 
@@ -594,7 +594,7 @@ mean(differences)
 ## [1] 1.001423
 ```
 
-```r
+``` r
 var(differences)
 ```
 
@@ -605,7 +605,7 @@ var(differences)
 We could have also accomplished this task with a single line of more "idiomatic" `R`.
 
 
-```r
+``` r
 set.seed(42)
 diffs = replicate(10000, mean(rnorm(25, 6, 2)) - mean(rnorm(25, 5, 2)))
 ```
@@ -613,7 +613,7 @@ diffs = replicate(10000, mean(rnorm(25, 6, 2)) - mean(rnorm(25, 5, 2)))
 Use `?replicate` to take a look at the documentation for the `replicate` function and see if you can understand how this line performs the same operations that our `for` loop above executed.
 
 
-```r
+``` r
 mean(differences == diffs)
 ```
 
@@ -654,7 +654,7 @@ Also, recall that for a random variable $X$ with finite mean $\mu$ and finite va
 The following verifies this result for a Poisson distribution with $\mu = 10$ and a sample size of $n = 50$.
 
 
-```r
+``` r
 set.seed(1337)
 mu          = 10
 sample_size = 50
@@ -663,14 +663,14 @@ x_bars      = rep(0, samples)
 ```
 
 
-```r
+``` r
 for(i in 1:samples){
   x_bars[i] = mean(rpois(sample_size, lambda = mu))
 }
 ```
 
 
-```r
+``` r
 x_bar_hist = hist(x_bars, breaks = 50, 
                   main = "Histogram of Sample Means",
                   xlab = "Sample Means")
@@ -681,7 +681,7 @@ x_bar_hist = hist(x_bars, breaks = 50,
 Now we will compare sample statistics from the empirical distribution with their known values based on the parent distribution.
 
 
-```r
+``` r
 c(mean(x_bars), mu)
 ```
 
@@ -690,7 +690,7 @@ c(mean(x_bars), mu)
 ```
 
 
-```r
+``` r
 c(var(x_bars), mu / sample_size)
 ```
 
@@ -699,7 +699,7 @@ c(var(x_bars), mu / sample_size)
 ```
 
 
-```r
+``` r
 c(sd(x_bars), sqrt(mu) / sqrt(sample_size))
 ```
 
@@ -710,7 +710,7 @@ c(sd(x_bars), sqrt(mu) / sqrt(sample_size))
 And here, we will calculate the proportion of sample means that are within 2 standard deviations of the population mean.
 
 
-```r
+``` r
 mean(x_bars > mu - 2 * sqrt(mu) / sqrt(sample_size) &
      x_bars < mu + 2 * sqrt(mu) / sqrt(sample_size))
 ```
@@ -722,7 +722,7 @@ mean(x_bars > mu - 2 * sqrt(mu) / sqrt(sample_size) &
 This last histogram uses a bit of a trick to approximately shade the bars that are within two standard deviations of the mean.)
 
 
-```r
+``` r
 shading = ifelse(x_bar_hist$breaks > mu - 2 * sqrt(mu) / sqrt(sample_size) & 
                    x_bar_hist$breaks < mu + 2 * sqrt(mu) / sqrt(sample_size),
                   "darkorange", "dodgerblue")
diff --git a/probability-and-statistics-in-r.html b/probability-and-statistics-in-r.html
index b776034..66204a4 100644
--- a/probability-and-statistics-in-r.html
+++ b/probability-and-statistics-in-r.html
@@ -6,7 +6,7 @@
   <meta http-equiv="X-UA-Compatible" content="IE=edge" />
   <title>Chapter 5 Probability and Statistics in R | Applied Statistics with R</title>
   <meta name="description" content="Chapter 5 Probability and Statistics in R | Applied Statistics with R" />
-  <meta name="generator" content="bookdown 0.37 and GitBook 2.6.7" />
+  <meta name="generator" content="bookdown 0.40 and GitBook 2.6.7" />
 
   <meta property="og:title" content="Chapter 5 Probability and Statistics in R | Applied Statistics with R" />
   <meta property="og:type" content="book" />
@@ -55,7 +55,7 @@
 
 <style type="text/css">
 pre > code.sourceCode { white-space: pre; position: relative; }
-pre > code.sourceCode > span { display: inline-block; line-height: 1.25; }
+pre > code.sourceCode > span { line-height: 1.25; }
 pre > code.sourceCode > span:empty { height: 1.2em; }
 .sourceCode { overflow: visible; }
 code.sourceCode > span { color: inherit; text-decoration: inherit; }
@@ -489,18 +489,18 @@ <h3><span class="header-section-number">5.1.1</span> Distributions<a href="proba
 <p>Note that <code>name</code> represents the name of the given distribution.</p>
 <p>For example, consider a random variable <span class="math inline">\(X\)</span> which is <span class="math inline">\(N(\mu = 2, \sigma^2 = 25)\)</span>. (Note, we are parameterizing using the variance <span class="math inline">\(\sigma^2\)</span>. <code>R</code> however uses the standard deviation.)</p>
 <p>To calculate the value of the pdf at <code>x = 3</code>, that is, the height of the curve at <code>x = 3</code>, use:</p>
-<div class="sourceCode" id="cb354"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb354-1"><a href="probability-and-statistics-in-r.html#cb354-1" aria-hidden="true" tabindex="-1"></a><span class="fu">dnorm</span>(<span class="at">x =</span> <span class="dv">3</span>, <span class="at">mean =</span> <span class="dv">2</span>, <span class="at">sd =</span> <span class="dv">5</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb354"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb354-1"><a href="probability-and-statistics-in-r.html#cb354-1" tabindex="-1"></a><span class="fu">dnorm</span>(<span class="at">x =</span> <span class="dv">3</span>, <span class="at">mean =</span> <span class="dv">2</span>, <span class="at">sd =</span> <span class="dv">5</span>)</span></code></pre></div>
 <pre><code>## [1] 0.07820854</code></pre>
 <p>To calculate the value of the cdf at <code>x = 3</code>, that is, <span class="math inline">\(P(X \leq 3)\)</span>, the probability that <span class="math inline">\(X\)</span> is less than or equal to <code>3</code>, use:</p>
-<div class="sourceCode" id="cb356"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb356-1"><a href="probability-and-statistics-in-r.html#cb356-1" aria-hidden="true" tabindex="-1"></a><span class="fu">pnorm</span>(<span class="at">q =</span> <span class="dv">3</span>, <span class="at">mean =</span> <span class="dv">2</span>, <span class="at">sd =</span> <span class="dv">5</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb356"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb356-1"><a href="probability-and-statistics-in-r.html#cb356-1" tabindex="-1"></a><span class="fu">pnorm</span>(<span class="at">q =</span> <span class="dv">3</span>, <span class="at">mean =</span> <span class="dv">2</span>, <span class="at">sd =</span> <span class="dv">5</span>)</span></code></pre></div>
 <pre><code>## [1] 0.5792597</code></pre>
 <p>Or, to calculate the quantile for probability 0.975, use:</p>
-<div class="sourceCode" id="cb358"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb358-1"><a href="probability-and-statistics-in-r.html#cb358-1" aria-hidden="true" tabindex="-1"></a><span class="fu">qnorm</span>(<span class="at">p =</span> <span class="fl">0.975</span>, <span class="at">mean =</span> <span class="dv">2</span>, <span class="at">sd =</span> <span class="dv">5</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb358"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb358-1"><a href="probability-and-statistics-in-r.html#cb358-1" tabindex="-1"></a><span class="fu">qnorm</span>(<span class="at">p =</span> <span class="fl">0.975</span>, <span class="at">mean =</span> <span class="dv">2</span>, <span class="at">sd =</span> <span class="dv">5</span>)</span></code></pre></div>
 <pre><code>## [1] 11.79982</code></pre>
 <p>Lastly, to generate a random sample of size <code>n = 10</code>, use:</p>
-<div class="sourceCode" id="cb360"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb360-1"><a href="probability-and-statistics-in-r.html#cb360-1" aria-hidden="true" tabindex="-1"></a><span class="fu">rnorm</span>(<span class="at">n =</span> <span class="dv">10</span>, <span class="at">mean =</span> <span class="dv">2</span>, <span class="at">sd =</span> <span class="dv">5</span>)</span></code></pre></div>
-<pre><code>##  [1] 10.761919  5.233989 -1.841460 -2.220900  4.484235  5.504151  1.051961
-##  [8]  8.648773  3.080896  5.714909</code></pre>
+<div class="sourceCode" id="cb360"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb360-1"><a href="probability-and-statistics-in-r.html#cb360-1" tabindex="-1"></a><span class="fu">rnorm</span>(<span class="at">n =</span> <span class="dv">10</span>, <span class="at">mean =</span> <span class="dv">2</span>, <span class="at">sd =</span> <span class="dv">5</span>)</span></code></pre></div>
+<pre><code>##  [1]   1.3334356  -5.0754528  14.1540511   2.1927133  -3.8482335  -0.2280363
+##  [7]   7.6598749  -2.7304969 -10.8570448  -6.9123819</code></pre>
 <p>These functions exist for many other distributions, including but not limited to:</p>
 <table>
 <thead>
@@ -533,7 +533,7 @@ <h3><span class="header-section-number">5.1.1</span> Distributions<a href="proba
 </tbody>
 </table>
 <p>Where <code>*</code> can be <code>d</code>, <code>p</code>, <code>q</code>, and <code>r</code>. Each distribution will have its own set of parameters which need to be passed to the functions as arguments. For example, <code>dbinom()</code> would not have arguments for <code>mean</code> and <code>sd</code>, since those are not parameters of the distribution. Instead a binomial distribution is usually parameterized by <span class="math inline">\(n\)</span> and <span class="math inline">\(p\)</span>, however <code>R</code> chooses to call them something else. To find the names that <code>R</code> uses we would use <code>?dbinom</code> and see that <code>R</code> instead calls the arguments <code>size</code> and <code>prob</code>. For example:</p>
-<div class="sourceCode" id="cb362"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb362-1"><a href="probability-and-statistics-in-r.html#cb362-1" aria-hidden="true" tabindex="-1"></a><span class="fu">dbinom</span>(<span class="at">x =</span> <span class="dv">6</span>, <span class="at">size =</span> <span class="dv">10</span>, <span class="at">prob =</span> <span class="fl">0.75</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb362"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb362-1"><a href="probability-and-statistics-in-r.html#cb362-1" tabindex="-1"></a><span class="fu">dbinom</span>(<span class="at">x =</span> <span class="dv">6</span>, <span class="at">size =</span> <span class="dv">10</span>, <span class="at">prob =</span> <span class="fl">0.75</span>)</span></code></pre></div>
 <pre><code>## [1] 0.145998</code></pre>
 <p>Also note that, when using the <code>dname</code> functions with discrete distributions, they are the pmf of the distribution. For example, the above command is <span class="math inline">\(P(Y = 6)\)</span> if <span class="math inline">\(Y \sim b(n = 10, p = 0.75)\)</span>. (The probability of flipping an unfair coin <code>10</code> times and seeing <code>6</code> heads, if the probability of heads is <code>0.75</code>.)</p>
 </div>
@@ -571,31 +571,31 @@ <h3><span class="header-section-number">5.2.1</span> One Sample t-Test: Review<a
 <div id="one-sample-t-test-example" class="section level3 hasAnchor" number="5.2.2">
 <h3><span class="header-section-number">5.2.2</span> One Sample t-Test: Example<a href="probability-and-statistics-in-r.html#one-sample-t-test-example" class="anchor-section" aria-label="Anchor link to header"></a></h3>
 <p>Suppose a grocery store sells “16 ounce” boxes of <em>Captain Crisp</em> cereal. A random sample of 9 boxes was taken and weighed. The weight in ounces is stored in the data frame <code>capt_crisp</code>.</p>
-<div class="sourceCode" id="cb364"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb364-1"><a href="probability-and-statistics-in-r.html#cb364-1" aria-hidden="true" tabindex="-1"></a>capt_crisp <span class="ot">=</span> <span class="fu">data.frame</span>(<span class="at">weight =</span> <span class="fu">c</span>(<span class="fl">15.5</span>, <span class="fl">16.2</span>, <span class="fl">16.1</span>, <span class="fl">15.8</span>, <span class="fl">15.6</span>, <span class="fl">16.0</span>, <span class="fl">15.8</span>, <span class="fl">15.9</span>, <span class="fl">16.2</span>))</span></code></pre></div>
+<div class="sourceCode" id="cb364"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb364-1"><a href="probability-and-statistics-in-r.html#cb364-1" tabindex="-1"></a>capt_crisp <span class="ot">=</span> <span class="fu">data.frame</span>(<span class="at">weight =</span> <span class="fu">c</span>(<span class="fl">15.5</span>, <span class="fl">16.2</span>, <span class="fl">16.1</span>, <span class="fl">15.8</span>, <span class="fl">15.6</span>, <span class="fl">16.0</span>, <span class="fl">15.8</span>, <span class="fl">15.9</span>, <span class="fl">16.2</span>))</span></code></pre></div>
 <p>The company that makes <em>Captain Crisp</em> cereal claims that the average weight of a box is at least 16 ounces. We will assume the weight of cereal in a box is normally distributed and use a 0.05 level of significance to test the company’s claim.</p>
 <p>To test <span class="math inline">\(H_{0}: \mu \geq 16\)</span> versus <span class="math inline">\(H_{1}: \mu &lt; 16\)</span>, the test statistic is</p>
 <p><span class="math display">\[
 t = \frac{\bar{x} - \mu_{0}}{s / \sqrt{n}}
 \]</span></p>
 <p>The sample mean <span class="math inline">\(\bar{x}\)</span> and the sample standard deviation <span class="math inline">\(s\)</span> can be easily computed using <code>R</code>. We also create variables which store the hypothesized mean and the sample size.</p>
-<div class="sourceCode" id="cb365"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb365-1"><a href="probability-and-statistics-in-r.html#cb365-1" aria-hidden="true" tabindex="-1"></a>x_bar <span class="ot">=</span> <span class="fu">mean</span>(capt_crisp<span class="sc">$</span>weight)</span>
-<span id="cb365-2"><a href="probability-and-statistics-in-r.html#cb365-2" aria-hidden="true" tabindex="-1"></a>s     <span class="ot">=</span> <span class="fu">sd</span>(capt_crisp<span class="sc">$</span>weight)</span>
-<span id="cb365-3"><a href="probability-and-statistics-in-r.html#cb365-3" aria-hidden="true" tabindex="-1"></a>mu_0  <span class="ot">=</span> <span class="dv">16</span></span>
-<span id="cb365-4"><a href="probability-and-statistics-in-r.html#cb365-4" aria-hidden="true" tabindex="-1"></a>n     <span class="ot">=</span> <span class="dv">9</span></span></code></pre></div>
+<div class="sourceCode" id="cb365"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb365-1"><a href="probability-and-statistics-in-r.html#cb365-1" tabindex="-1"></a>x_bar <span class="ot">=</span> <span class="fu">mean</span>(capt_crisp<span class="sc">$</span>weight)</span>
+<span id="cb365-2"><a href="probability-and-statistics-in-r.html#cb365-2" tabindex="-1"></a>s     <span class="ot">=</span> <span class="fu">sd</span>(capt_crisp<span class="sc">$</span>weight)</span>
+<span id="cb365-3"><a href="probability-and-statistics-in-r.html#cb365-3" tabindex="-1"></a>mu_0  <span class="ot">=</span> <span class="dv">16</span></span>
+<span id="cb365-4"><a href="probability-and-statistics-in-r.html#cb365-4" tabindex="-1"></a>n     <span class="ot">=</span> <span class="dv">9</span></span></code></pre></div>
 <p>We can then easily compute the test statistic.</p>
-<div class="sourceCode" id="cb366"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb366-1"><a href="probability-and-statistics-in-r.html#cb366-1" aria-hidden="true" tabindex="-1"></a>t <span class="ot">=</span> (x_bar <span class="sc">-</span> mu_0) <span class="sc">/</span> (s <span class="sc">/</span> <span class="fu">sqrt</span>(n))</span>
-<span id="cb366-2"><a href="probability-and-statistics-in-r.html#cb366-2" aria-hidden="true" tabindex="-1"></a>t</span></code></pre></div>
+<div class="sourceCode" id="cb366"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb366-1"><a href="probability-and-statistics-in-r.html#cb366-1" tabindex="-1"></a>t <span class="ot">=</span> (x_bar <span class="sc">-</span> mu_0) <span class="sc">/</span> (s <span class="sc">/</span> <span class="fu">sqrt</span>(n))</span>
+<span id="cb366-2"><a href="probability-and-statistics-in-r.html#cb366-2" tabindex="-1"></a>t</span></code></pre></div>
 <pre><code>## [1] -1.2</code></pre>
 <p>Under the null hypothesis, the test statistic has a <span class="math inline">\(t\)</span> distribution with <span class="math inline">\(n - 1\)</span> degrees of freedom, in this case 8.</p>
 <p>To complete the test, we need to obtain the p-value of the test. Since this is a one-sided test with a less-than alternative, we need the area to the left of -1.2 for a <span class="math inline">\(t\)</span> distribution with 8 degrees of freedom. That is,</p>
 <p><span class="math display">\[
 P(t_{8} &lt; -1.2)
 \]</span></p>
-<div class="sourceCode" id="cb368"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb368-1"><a href="probability-and-statistics-in-r.html#cb368-1" aria-hidden="true" tabindex="-1"></a><span class="fu">pt</span>(t, <span class="at">df =</span> n <span class="sc">-</span> <span class="dv">1</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb368"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb368-1"><a href="probability-and-statistics-in-r.html#cb368-1" tabindex="-1"></a><span class="fu">pt</span>(t, <span class="at">df =</span> n <span class="sc">-</span> <span class="dv">1</span>)</span></code></pre></div>
 <pre><code>## [1] 0.1322336</code></pre>
 <p>We now have the p-value of our test, which is greater than our significance level (0.05), so we fail to reject the null hypothesis.</p>
 <p>Alternatively, this entire process could have been completed using one line of <code>R</code> code.</p>
-<div class="sourceCode" id="cb370"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb370-1"><a href="probability-and-statistics-in-r.html#cb370-1" aria-hidden="true" tabindex="-1"></a><span class="fu">t.test</span>(<span class="at">x =</span> capt_crisp<span class="sc">$</span>weight, <span class="at">mu =</span> <span class="dv">16</span>, <span class="at">alternative =</span> <span class="fu">c</span>(<span class="st">&quot;less&quot;</span>), <span class="at">conf.level =</span> <span class="fl">0.95</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb370"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb370-1"><a href="probability-and-statistics-in-r.html#cb370-1" tabindex="-1"></a><span class="fu">t.test</span>(<span class="at">x =</span> capt_crisp<span class="sc">$</span>weight, <span class="at">mu =</span> <span class="dv">16</span>, <span class="at">alternative =</span> <span class="fu">c</span>(<span class="st">&quot;less&quot;</span>), <span class="at">conf.level =</span> <span class="fl">0.95</span>)</span></code></pre></div>
 <pre><code>## 
 ##  One Sample t-test
 ## 
@@ -616,26 +616,26 @@ <h3><span class="header-section-number">5.2.2</span> One Sample t-Test: Example<
 <li>An estimate of <span class="math inline">\(\mu\)</span>.</li>
 </ul>
 <p>Since the test was one-sided, <code>R</code> returned a one-sided confidence interval. If instead we wanted a two-sided interval for the mean weight of boxes of <em>Captain Crisp</em> cereal we could modify our code.</p>
-<div class="sourceCode" id="cb372"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb372-1"><a href="probability-and-statistics-in-r.html#cb372-1" aria-hidden="true" tabindex="-1"></a>capt_test_results <span class="ot">=</span> <span class="fu">t.test</span>(capt_crisp<span class="sc">$</span>weight, <span class="at">mu =</span> <span class="dv">16</span>,</span>
-<span id="cb372-2"><a href="probability-and-statistics-in-r.html#cb372-2" aria-hidden="true" tabindex="-1"></a>                           <span class="at">alternative =</span> <span class="fu">c</span>(<span class="st">&quot;two.sided&quot;</span>), <span class="at">conf.level =</span> <span class="fl">0.95</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb372"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb372-1"><a href="probability-and-statistics-in-r.html#cb372-1" tabindex="-1"></a>capt_test_results <span class="ot">=</span> <span class="fu">t.test</span>(capt_crisp<span class="sc">$</span>weight, <span class="at">mu =</span> <span class="dv">16</span>,</span>
+<span id="cb372-2"><a href="probability-and-statistics-in-r.html#cb372-2" tabindex="-1"></a>                           <span class="at">alternative =</span> <span class="fu">c</span>(<span class="st">&quot;two.sided&quot;</span>), <span class="at">conf.level =</span> <span class="fl">0.95</span>)</span></code></pre></div>
 <p>This time we have stored the results. By doing so, we can directly access portions of the output from <code>t.test()</code>. To see what information is available we use the <code>names()</code> function.</p>
-<div class="sourceCode" id="cb373"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb373-1"><a href="probability-and-statistics-in-r.html#cb373-1" aria-hidden="true" tabindex="-1"></a><span class="fu">names</span>(capt_test_results)</span></code></pre></div>
+<div class="sourceCode" id="cb373"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb373-1"><a href="probability-and-statistics-in-r.html#cb373-1" tabindex="-1"></a><span class="fu">names</span>(capt_test_results)</span></code></pre></div>
 <pre><code>##  [1] &quot;statistic&quot;   &quot;parameter&quot;   &quot;p.value&quot;     &quot;conf.int&quot;    &quot;estimate&quot;   
 ##  [6] &quot;null.value&quot;  &quot;stderr&quot;      &quot;alternative&quot; &quot;method&quot;      &quot;data.name&quot;</code></pre>
 <p>We are interested in the confidence interval which is stored in <code>conf.int</code>.</p>
-<div class="sourceCode" id="cb375"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb375-1"><a href="probability-and-statistics-in-r.html#cb375-1" aria-hidden="true" tabindex="-1"></a>capt_test_results<span class="sc">$</span>conf.int</span></code></pre></div>
+<div class="sourceCode" id="cb375"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb375-1"><a href="probability-and-statistics-in-r.html#cb375-1" tabindex="-1"></a>capt_test_results<span class="sc">$</span>conf.int</span></code></pre></div>
 <pre><code>## [1] 15.70783 16.09217
 ## attr(,&quot;conf.level&quot;)
 ## [1] 0.95</code></pre>
 <p>Let’s check this interval “by hand.” The one piece of information we are missing is the critical value, <span class="math inline">\(t_{\alpha/2, n-1} = t_{8}(0.025)\)</span>, which can be calculated in <code>R</code> using the <code>qt()</code> function.</p>
-<div class="sourceCode" id="cb377"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb377-1"><a href="probability-and-statistics-in-r.html#cb377-1" aria-hidden="true" tabindex="-1"></a><span class="fu">qt</span>(<span class="fl">0.975</span>, <span class="at">df =</span> <span class="dv">8</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb377"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb377-1"><a href="probability-and-statistics-in-r.html#cb377-1" tabindex="-1"></a><span class="fu">qt</span>(<span class="fl">0.975</span>, <span class="at">df =</span> <span class="dv">8</span>)</span></code></pre></div>
 <pre><code>## [1] 2.306004</code></pre>
 <p>So, the 95% CI for the mean weight of a cereal box is calculated by plugging into the formula,</p>
 <p><span class="math display">\[
 \bar{x} \pm t_{\alpha/2, n-1} \frac{s}{\sqrt{n}}
 \]</span></p>
-<div class="sourceCode" id="cb379"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb379-1"><a href="probability-and-statistics-in-r.html#cb379-1" aria-hidden="true" tabindex="-1"></a><span class="fu">c</span>(<span class="fu">mean</span>(capt_crisp<span class="sc">$</span>weight) <span class="sc">-</span> <span class="fu">qt</span>(<span class="fl">0.975</span>, <span class="at">df =</span> <span class="dv">8</span>) <span class="sc">*</span> <span class="fu">sd</span>(capt_crisp<span class="sc">$</span>weight) <span class="sc">/</span> <span class="fu">sqrt</span>(<span class="dv">9</span>),</span>
-<span id="cb379-2"><a href="probability-and-statistics-in-r.html#cb379-2" aria-hidden="true" tabindex="-1"></a>  <span class="fu">mean</span>(capt_crisp<span class="sc">$</span>weight) <span class="sc">+</span> <span class="fu">qt</span>(<span class="fl">0.975</span>, <span class="at">df =</span> <span class="dv">8</span>) <span class="sc">*</span> <span class="fu">sd</span>(capt_crisp<span class="sc">$</span>weight) <span class="sc">/</span> <span class="fu">sqrt</span>(<span class="dv">9</span>))</span></code></pre></div>
+<div class="sourceCode" id="cb379"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb379-1"><a href="probability-and-statistics-in-r.html#cb379-1" tabindex="-1"></a><span class="fu">c</span>(<span class="fu">mean</span>(capt_crisp<span class="sc">$</span>weight) <span class="sc">-</span> <span class="fu">qt</span>(<span class="fl">0.975</span>, <span class="at">df =</span> <span class="dv">8</span>) <span class="sc">*</span> <span class="fu">sd</span>(capt_crisp<span class="sc">$</span>weight) <span class="sc">/</span> <span class="fu">sqrt</span>(<span class="dv">9</span>),</span>
+<span id="cb379-2"><a href="probability-and-statistics-in-r.html#cb379-2" tabindex="-1"></a>  <span class="fu">mean</span>(capt_crisp<span class="sc">$</span>weight) <span class="sc">+</span> <span class="fu">qt</span>(<span class="fl">0.975</span>, <span class="at">df =</span> <span class="dv">8</span>) <span class="sc">*</span> <span class="fu">sd</span>(capt_crisp<span class="sc">$</span>weight) <span class="sc">/</span> <span class="fu">sqrt</span>(<span class="dv">9</span>))</span></code></pre></div>
 <pre><code>## [1] 15.70783 16.09217</code></pre>
 </div>
 <div id="two-sample-t-test-review" class="section level3 hasAnchor" number="5.2.3">
@@ -656,37 +656,37 @@ <h3><span class="header-section-number">5.2.3</span> Two Sample t-Test: Review<a
 <div id="two-sample-t-test-example" class="section level3 hasAnchor" number="5.2.4">
 <h3><span class="header-section-number">5.2.4</span> Two Sample t-Test: Example<a href="probability-and-statistics-in-r.html#two-sample-t-test-example" class="anchor-section" aria-label="Anchor link to header"></a></h3>
 <p>Assume that the distributions of <span class="math inline">\(X\)</span> and <span class="math inline">\(Y\)</span> are <span class="math inline">\(\mathrm{N}(\mu_{1},\sigma^{2})\)</span> and <span class="math inline">\(\mathrm{N}(\mu_{2},\sigma^{2})\)</span>, respectively. Given the <span class="math inline">\(n = 6\)</span> observations of <span class="math inline">\(X\)</span>,</p>
-<div class="sourceCode" id="cb381"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb381-1"><a href="probability-and-statistics-in-r.html#cb381-1" aria-hidden="true" tabindex="-1"></a>x <span class="ot">=</span> <span class="fu">c</span>(<span class="dv">70</span>, <span class="dv">82</span>, <span class="dv">78</span>, <span class="dv">74</span>, <span class="dv">94</span>, <span class="dv">82</span>)</span>
-<span id="cb381-2"><a href="probability-and-statistics-in-r.html#cb381-2" aria-hidden="true" tabindex="-1"></a>n <span class="ot">=</span> <span class="fu">length</span>(x)</span></code></pre></div>
+<div class="sourceCode" id="cb381"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb381-1"><a href="probability-and-statistics-in-r.html#cb381-1" tabindex="-1"></a>x <span class="ot">=</span> <span class="fu">c</span>(<span class="dv">70</span>, <span class="dv">82</span>, <span class="dv">78</span>, <span class="dv">74</span>, <span class="dv">94</span>, <span class="dv">82</span>)</span>
+<span id="cb381-2"><a href="probability-and-statistics-in-r.html#cb381-2" tabindex="-1"></a>n <span class="ot">=</span> <span class="fu">length</span>(x)</span></code></pre></div>
 <p>and the <span class="math inline">\(m = 8\)</span> observations of <span class="math inline">\(Y\)</span>,</p>
-<div class="sourceCode" id="cb382"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb382-1"><a href="probability-and-statistics-in-r.html#cb382-1" aria-hidden="true" tabindex="-1"></a>y <span class="ot">=</span> <span class="fu">c</span>(<span class="dv">64</span>, <span class="dv">72</span>, <span class="dv">60</span>, <span class="dv">76</span>, <span class="dv">72</span>, <span class="dv">80</span>, <span class="dv">84</span>, <span class="dv">68</span>)</span>
-<span id="cb382-2"><a href="probability-and-statistics-in-r.html#cb382-2" aria-hidden="true" tabindex="-1"></a>m <span class="ot">=</span> <span class="fu">length</span>(y)</span></code></pre></div>
+<div class="sourceCode" id="cb382"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb382-1"><a href="probability-and-statistics-in-r.html#cb382-1" tabindex="-1"></a>y <span class="ot">=</span> <span class="fu">c</span>(<span class="dv">64</span>, <span class="dv">72</span>, <span class="dv">60</span>, <span class="dv">76</span>, <span class="dv">72</span>, <span class="dv">80</span>, <span class="dv">84</span>, <span class="dv">68</span>)</span>
+<span id="cb382-2"><a href="probability-and-statistics-in-r.html#cb382-2" tabindex="-1"></a>m <span class="ot">=</span> <span class="fu">length</span>(y)</span></code></pre></div>
 <p>we will test <span class="math inline">\(H_{0}: \mu_{1} = \mu_{2}\)</span> versus <span class="math inline">\(H_{1}: \mu_{1} &gt; \mu_{2}\)</span>.</p>
 <p>First, note that we can calculate the sample means and standard deviations.</p>
-<div class="sourceCode" id="cb383"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb383-1"><a href="probability-and-statistics-in-r.html#cb383-1" aria-hidden="true" tabindex="-1"></a>x_bar <span class="ot">=</span> <span class="fu">mean</span>(x)</span>
-<span id="cb383-2"><a href="probability-and-statistics-in-r.html#cb383-2" aria-hidden="true" tabindex="-1"></a>s_x   <span class="ot">=</span> <span class="fu">sd</span>(x)</span>
-<span id="cb383-3"><a href="probability-and-statistics-in-r.html#cb383-3" aria-hidden="true" tabindex="-1"></a>y_bar <span class="ot">=</span> <span class="fu">mean</span>(y)</span>
-<span id="cb383-4"><a href="probability-and-statistics-in-r.html#cb383-4" aria-hidden="true" tabindex="-1"></a>s_y   <span class="ot">=</span> <span class="fu">sd</span>(y)</span></code></pre></div>
+<div class="sourceCode" id="cb383"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb383-1"><a href="probability-and-statistics-in-r.html#cb383-1" tabindex="-1"></a>x_bar <span class="ot">=</span> <span class="fu">mean</span>(x)</span>
+<span id="cb383-2"><a href="probability-and-statistics-in-r.html#cb383-2" tabindex="-1"></a>s_x   <span class="ot">=</span> <span class="fu">sd</span>(x)</span>
+<span id="cb383-3"><a href="probability-and-statistics-in-r.html#cb383-3" tabindex="-1"></a>y_bar <span class="ot">=</span> <span class="fu">mean</span>(y)</span>
+<span id="cb383-4"><a href="probability-and-statistics-in-r.html#cb383-4" tabindex="-1"></a>s_y   <span class="ot">=</span> <span class="fu">sd</span>(y)</span></code></pre></div>
 <p>We can then calculate the pooled standard deviation.</p>
 <p><span class="math display">\[
 s_{p} = \sqrt{\frac{(n-1)s_{x}^{2}+(m-1)s_{y}^{2}}{n+m-2}}
 \]</span></p>
-<div class="sourceCode" id="cb384"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb384-1"><a href="probability-and-statistics-in-r.html#cb384-1" aria-hidden="true" tabindex="-1"></a>s_p <span class="ot">=</span> <span class="fu">sqrt</span>(((n <span class="sc">-</span> <span class="dv">1</span>) <span class="sc">*</span> s_x <span class="sc">^</span> <span class="dv">2</span> <span class="sc">+</span> (m <span class="sc">-</span> <span class="dv">1</span>) <span class="sc">*</span> s_y <span class="sc">^</span> <span class="dv">2</span>) <span class="sc">/</span> (n <span class="sc">+</span> m <span class="sc">-</span> <span class="dv">2</span>))</span></code></pre></div>
+<div class="sourceCode" id="cb384"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb384-1"><a href="probability-and-statistics-in-r.html#cb384-1" tabindex="-1"></a>s_p <span class="ot">=</span> <span class="fu">sqrt</span>(((n <span class="sc">-</span> <span class="dv">1</span>) <span class="sc">*</span> s_x <span class="sc">^</span> <span class="dv">2</span> <span class="sc">+</span> (m <span class="sc">-</span> <span class="dv">1</span>) <span class="sc">*</span> s_y <span class="sc">^</span> <span class="dv">2</span>) <span class="sc">/</span> (n <span class="sc">+</span> m <span class="sc">-</span> <span class="dv">2</span>))</span></code></pre></div>
 <p>Thus, the relevant <span class="math inline">\(t\)</span> test statistic is given by</p>
 <p><span class="math display">\[
 t = \frac{(\bar{x}-\bar{y})-\mu_{0}}{s_{p}\sqrt{\frac{1}{n}+\frac{1}{m}}}.
 \]</span></p>
-<div class="sourceCode" id="cb385"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb385-1"><a href="probability-and-statistics-in-r.html#cb385-1" aria-hidden="true" tabindex="-1"></a>t <span class="ot">=</span> ((x_bar <span class="sc">-</span> y_bar) <span class="sc">-</span> <span class="dv">0</span>) <span class="sc">/</span> (s_p <span class="sc">*</span> <span class="fu">sqrt</span>(<span class="dv">1</span> <span class="sc">/</span> n <span class="sc">+</span> <span class="dv">1</span> <span class="sc">/</span> m))</span>
-<span id="cb385-2"><a href="probability-and-statistics-in-r.html#cb385-2" aria-hidden="true" tabindex="-1"></a>t</span></code></pre></div>
+<div class="sourceCode" id="cb385"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb385-1"><a href="probability-and-statistics-in-r.html#cb385-1" tabindex="-1"></a>t <span class="ot">=</span> ((x_bar <span class="sc">-</span> y_bar) <span class="sc">-</span> <span class="dv">0</span>) <span class="sc">/</span> (s_p <span class="sc">*</span> <span class="fu">sqrt</span>(<span class="dv">1</span> <span class="sc">/</span> n <span class="sc">+</span> <span class="dv">1</span> <span class="sc">/</span> m))</span>
+<span id="cb385-2"><a href="probability-and-statistics-in-r.html#cb385-2" tabindex="-1"></a>t</span></code></pre></div>
 <pre><code>## [1] 1.823369</code></pre>
 <p>Note that <span class="math inline">\(t \sim t_{n + m - 2} = t_{12}\)</span>, so we can calculate the p-value, which is</p>
 <p><span class="math display">\[
 P(t_{12} &gt; 1.8233692).
 \]</span></p>
-<div class="sourceCode" id="cb387"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb387-1"><a href="probability-and-statistics-in-r.html#cb387-1" aria-hidden="true" tabindex="-1"></a><span class="dv">1</span> <span class="sc">-</span> <span class="fu">pt</span>(t, <span class="at">df =</span> n <span class="sc">+</span> m <span class="sc">-</span> <span class="dv">2</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb387"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb387-1"><a href="probability-and-statistics-in-r.html#cb387-1" tabindex="-1"></a><span class="dv">1</span> <span class="sc">-</span> <span class="fu">pt</span>(t, <span class="at">df =</span> n <span class="sc">+</span> m <span class="sc">-</span> <span class="dv">2</span>)</span></code></pre></div>
 <pre><code>## [1] 0.04661961</code></pre>
 <p>But, then again, we could have simply performed this test in one line of <code>R</code>.</p>
-<div class="sourceCode" id="cb389"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb389-1"><a href="probability-and-statistics-in-r.html#cb389-1" aria-hidden="true" tabindex="-1"></a><span class="fu">t.test</span>(x, y, <span class="at">alternative =</span> <span class="fu">c</span>(<span class="st">&quot;greater&quot;</span>), <span class="at">var.equal =</span> <span class="cn">TRUE</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb389"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb389-1"><a href="probability-and-statistics-in-r.html#cb389-1" tabindex="-1"></a><span class="fu">t.test</span>(x, y, <span class="at">alternative =</span> <span class="fu">c</span>(<span class="st">&quot;greater&quot;</span>), <span class="at">var.equal =</span> <span class="cn">TRUE</span>)</span></code></pre></div>
 <pre><code>## 
 ##  Two Sample t-test
 ## 
@@ -700,10 +700,10 @@ <h3><span class="header-section-number">5.2.4</span> Two Sample t-Test: Example<
 ##        80        72</code></pre>
 <p>Recall that a two-sample <span class="math inline">\(t\)</span>-test can be done with or without an equal variance assumption. Here <code>var.equal = TRUE</code> tells <code>R</code> we would like to perform the test under the equal variance assumption.</p>
 <p>Above we carried out the analysis using two vectors <code>x</code> and <code>y</code>. In general, we will have a preference for using data frames.</p>
-<div class="sourceCode" id="cb391"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb391-1"><a href="probability-and-statistics-in-r.html#cb391-1" aria-hidden="true" tabindex="-1"></a>t_test_data <span class="ot">=</span> <span class="fu">data.frame</span>(<span class="at">values =</span> <span class="fu">c</span>(x, y),</span>
-<span id="cb391-2"><a href="probability-and-statistics-in-r.html#cb391-2" aria-hidden="true" tabindex="-1"></a>                         <span class="at">group  =</span> <span class="fu">c</span>(<span class="fu">rep</span>(<span class="st">&quot;A&quot;</span>, <span class="fu">length</span>(x)), <span class="fu">rep</span>(<span class="st">&quot;B&quot;</span>, <span class="fu">length</span>(y))))</span></code></pre></div>
+<div class="sourceCode" id="cb391"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb391-1"><a href="probability-and-statistics-in-r.html#cb391-1" tabindex="-1"></a>t_test_data <span class="ot">=</span> <span class="fu">data.frame</span>(<span class="at">values =</span> <span class="fu">c</span>(x, y),</span>
+<span id="cb391-2"><a href="probability-and-statistics-in-r.html#cb391-2" tabindex="-1"></a>                         <span class="at">group  =</span> <span class="fu">c</span>(<span class="fu">rep</span>(<span class="st">&quot;A&quot;</span>, <span class="fu">length</span>(x)), <span class="fu">rep</span>(<span class="st">&quot;B&quot;</span>, <span class="fu">length</span>(y))))</span></code></pre></div>
 <p>We now have the data stored in a single variables (<code>values</code>) and have created a second variable (<code>group</code>) which indicates which “sample” the value belongs to.</p>
-<div class="sourceCode" id="cb392"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb392-1"><a href="probability-and-statistics-in-r.html#cb392-1" aria-hidden="true" tabindex="-1"></a>t_test_data</span></code></pre></div>
+<div class="sourceCode" id="cb392"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb392-1"><a href="probability-and-statistics-in-r.html#cb392-1" tabindex="-1"></a>t_test_data</span></code></pre></div>
 <pre><code>##    values group
 ## 1      70     A
 ## 2      82     A
@@ -720,8 +720,8 @@ <h3><span class="header-section-number">5.2.4</span> Two Sample t-Test: Example<
 ## 13     84     B
 ## 14     68     B</code></pre>
 <p>Now to perform the test, we still use the <code>t.test()</code> function but with the <code>~</code> syntax and a <code>data</code> argument.</p>
-<div class="sourceCode" id="cb394"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb394-1"><a href="probability-and-statistics-in-r.html#cb394-1" aria-hidden="true" tabindex="-1"></a><span class="fu">t.test</span>(values <span class="sc">~</span> group, <span class="at">data =</span> t_test_data,</span>
-<span id="cb394-2"><a href="probability-and-statistics-in-r.html#cb394-2" aria-hidden="true" tabindex="-1"></a>       <span class="at">alternative =</span> <span class="fu">c</span>(<span class="st">&quot;greater&quot;</span>), <span class="at">var.equal =</span> <span class="cn">TRUE</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb394"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb394-1"><a href="probability-and-statistics-in-r.html#cb394-1" tabindex="-1"></a><span class="fu">t.test</span>(values <span class="sc">~</span> group, <span class="at">data =</span> t_test_data,</span>
+<span id="cb394-2"><a href="probability-and-statistics-in-r.html#cb394-2" tabindex="-1"></a>       <span class="at">alternative =</span> <span class="fu">c</span>(<span class="st">&quot;greater&quot;</span>), <span class="at">var.equal =</span> <span class="cn">TRUE</span>)</span></code></pre></div>
 <pre><code>## 
 ##  Two Sample t-test
 ## 
@@ -742,9 +742,9 @@ <h2><span class="header-section-number">5.3</span> Simulation<a href="probabilit
 <li>In <strong>simulation</strong>, the <em>data generating process</em> is known. We will know the form of the model as well as the value of each of the parameters. In particular, we will often control the distribution and parameters which define the randomness, or noise in the data.</li>
 <li>In <strong>model fitting</strong>, the <em>data</em> is known. We will then assume a certain form of model and find the best possible values of the parameters given the observed data. Essentially we are seeking to uncover the truth. Often we will attempt to fit many models, and we will learn metrics to assess which model fits best.</li>
 </ul>
-<div class="figure">
-<img src="images/simulation.png" alt="" />
-<p class="caption">Simulation vs Modeling</p>
+<div class="float">
+<img src="images/simulation.png" alt="Simulation vs Modeling" />
+<div class="figcaption">Simulation vs Modeling</div>
 </div>
 <p>Often we will simulate data according to a process we decide, then use a modeling method seen in class. We can then verify how well the method works, since we know the data generating process.</p>
 <p>One of the biggest strengths of <code>R</code> is its ability to carry out simulations using built-in functions for generating random samples from certain distributions. We’ll look at two very simple examples here, however simulation will be a topic we revisit several times throughout the course.</p>
@@ -788,7 +788,7 @@ <h3><span class="header-section-number">5.3.1</span> Paired Differences<a href="
 P(0 &lt; D &lt; 2) = P(D &lt; 2) - P(D &lt; 0).
 \]</span></p>
 <p>This can then be calculated using <code>R</code> without a need to first standardize, or use a table.</p>
-<div class="sourceCode" id="cb396"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb396-1"><a href="probability-and-statistics-in-r.html#cb396-1" aria-hidden="true" tabindex="-1"></a><span class="fu">pnorm</span>(<span class="dv">2</span>, <span class="at">mean =</span> <span class="dv">1</span>, <span class="at">sd =</span> <span class="fu">sqrt</span>(<span class="fl">0.32</span>)) <span class="sc">-</span> <span class="fu">pnorm</span>(<span class="dv">0</span>, <span class="at">mean =</span> <span class="dv">1</span>, <span class="at">sd =</span> <span class="fu">sqrt</span>(<span class="fl">0.32</span>))</span></code></pre></div>
+<div class="sourceCode" id="cb396"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb396-1"><a href="probability-and-statistics-in-r.html#cb396-1" tabindex="-1"></a><span class="fu">pnorm</span>(<span class="dv">2</span>, <span class="at">mean =</span> <span class="dv">1</span>, <span class="at">sd =</span> <span class="fu">sqrt</span>(<span class="fl">0.32</span>)) <span class="sc">-</span> <span class="fu">pnorm</span>(<span class="dv">0</span>, <span class="at">mean =</span> <span class="dv">1</span>, <span class="at">sd =</span> <span class="fu">sqrt</span>(<span class="fl">0.32</span>))</span></code></pre></div>
 <pre><code>## [1] 0.9229001</code></pre>
 <p>An alternative approach, would be to <strong>simulate</strong> a large number of observations of <span class="math inline">\(D\)</span> then use the <strong>empirical distribution</strong> to calculate the probability.</p>
 <p>Our strategy will be to repeatedly:</p>
@@ -798,37 +798,37 @@ <h3><span class="header-section-number">5.3.1</span> Paired Differences<a href="
 <li>Calculate the differences in the means, <span class="math inline">\(d_s = \bar{x}_{1s} - \bar{x}_{2s}\)</span>.</li>
 </ul>
 <p>We will repeat the process a large number of times. Then we will use the distribution of the simulated observations of <span class="math inline">\(d_s\)</span> as an estimate for the true distribution of <span class="math inline">\(D\)</span>.</p>
-<div class="sourceCode" id="cb398"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb398-1"><a href="probability-and-statistics-in-r.html#cb398-1" aria-hidden="true" tabindex="-1"></a><span class="fu">set.seed</span>(<span class="dv">42</span>)</span>
-<span id="cb398-2"><a href="probability-and-statistics-in-r.html#cb398-2" aria-hidden="true" tabindex="-1"></a>num_samples <span class="ot">=</span> <span class="dv">10000</span></span>
-<span id="cb398-3"><a href="probability-and-statistics-in-r.html#cb398-3" aria-hidden="true" tabindex="-1"></a>differences <span class="ot">=</span> <span class="fu">rep</span>(<span class="dv">0</span>, num_samples)</span></code></pre></div>
+<div class="sourceCode" id="cb398"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb398-1"><a href="probability-and-statistics-in-r.html#cb398-1" tabindex="-1"></a><span class="fu">set.seed</span>(<span class="dv">42</span>)</span>
+<span id="cb398-2"><a href="probability-and-statistics-in-r.html#cb398-2" tabindex="-1"></a>num_samples <span class="ot">=</span> <span class="dv">10000</span></span>
+<span id="cb398-3"><a href="probability-and-statistics-in-r.html#cb398-3" tabindex="-1"></a>differences <span class="ot">=</span> <span class="fu">rep</span>(<span class="dv">0</span>, num_samples)</span></code></pre></div>
 <p>Before starting our <code>for</code> loop to perform the operation, we set a seed for reproducibility, create and set a variable <code>num_samples</code> which will define the number of repetitions, and lastly create a variables <code>differences</code> which will store the simulate values, <span class="math inline">\(d_s\)</span>.</p>
 <p>By using <code>set.seed()</code> we can reproduce the random results of <code>rnorm()</code> each time starting from that line.</p>
-<div class="sourceCode" id="cb399"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb399-1"><a href="probability-and-statistics-in-r.html#cb399-1" aria-hidden="true" tabindex="-1"></a><span class="cf">for</span> (s <span class="cf">in</span> <span class="dv">1</span><span class="sc">:</span>num_samples) {</span>
-<span id="cb399-2"><a href="probability-and-statistics-in-r.html#cb399-2" aria-hidden="true" tabindex="-1"></a>  x1 <span class="ot">=</span> <span class="fu">rnorm</span>(<span class="at">n =</span> <span class="dv">25</span>, <span class="at">mean =</span> <span class="dv">6</span>, <span class="at">sd =</span> <span class="dv">2</span>)</span>
-<span id="cb399-3"><a href="probability-and-statistics-in-r.html#cb399-3" aria-hidden="true" tabindex="-1"></a>  x2 <span class="ot">=</span> <span class="fu">rnorm</span>(<span class="at">n =</span> <span class="dv">25</span>, <span class="at">mean =</span> <span class="dv">5</span>, <span class="at">sd =</span> <span class="dv">2</span>)</span>
-<span id="cb399-4"><a href="probability-and-statistics-in-r.html#cb399-4" aria-hidden="true" tabindex="-1"></a>  differences[s] <span class="ot">=</span> <span class="fu">mean</span>(x1) <span class="sc">-</span> <span class="fu">mean</span>(x2)</span>
-<span id="cb399-5"><a href="probability-and-statistics-in-r.html#cb399-5" aria-hidden="true" tabindex="-1"></a>}</span></code></pre></div>
+<div class="sourceCode" id="cb399"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb399-1"><a href="probability-and-statistics-in-r.html#cb399-1" tabindex="-1"></a><span class="cf">for</span> (s <span class="cf">in</span> <span class="dv">1</span><span class="sc">:</span>num_samples) {</span>
+<span id="cb399-2"><a href="probability-and-statistics-in-r.html#cb399-2" tabindex="-1"></a>  x1 <span class="ot">=</span> <span class="fu">rnorm</span>(<span class="at">n =</span> <span class="dv">25</span>, <span class="at">mean =</span> <span class="dv">6</span>, <span class="at">sd =</span> <span class="dv">2</span>)</span>
+<span id="cb399-3"><a href="probability-and-statistics-in-r.html#cb399-3" tabindex="-1"></a>  x2 <span class="ot">=</span> <span class="fu">rnorm</span>(<span class="at">n =</span> <span class="dv">25</span>, <span class="at">mean =</span> <span class="dv">5</span>, <span class="at">sd =</span> <span class="dv">2</span>)</span>
+<span id="cb399-4"><a href="probability-and-statistics-in-r.html#cb399-4" tabindex="-1"></a>  differences[s] <span class="ot">=</span> <span class="fu">mean</span>(x1) <span class="sc">-</span> <span class="fu">mean</span>(x2)</span>
+<span id="cb399-5"><a href="probability-and-statistics-in-r.html#cb399-5" tabindex="-1"></a>}</span></code></pre></div>
 <p>To estimate <span class="math inline">\(P(0 &lt; D &lt; 2)\)</span> we will find the proportion of values of <span class="math inline">\(d_s\)</span> (among the 10^{4} values of <span class="math inline">\(d_s\)</span> generated) that are between 0 and 2.</p>
-<div class="sourceCode" id="cb400"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb400-1"><a href="probability-and-statistics-in-r.html#cb400-1" aria-hidden="true" tabindex="-1"></a><span class="fu">mean</span>(<span class="dv">0</span> <span class="sc">&lt;</span> differences <span class="sc">&amp;</span> differences <span class="sc">&lt;</span> <span class="dv">2</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb400"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb400-1"><a href="probability-and-statistics-in-r.html#cb400-1" tabindex="-1"></a><span class="fu">mean</span>(<span class="dv">0</span> <span class="sc">&lt;</span> differences <span class="sc">&amp;</span> differences <span class="sc">&lt;</span> <span class="dv">2</span>)</span></code></pre></div>
 <pre><code>## [1] 0.9222</code></pre>
 <p>Recall that above we derived the distribution of <span class="math inline">\(D\)</span> to be <span class="math inline">\(N(\mu = 1, \sigma^2 = 0.32)\)</span></p>
 <p>If we look at a histogram of the differences, we find that it looks very much like a normal distribution.</p>
-<div class="sourceCode" id="cb402"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb402-1"><a href="probability-and-statistics-in-r.html#cb402-1" aria-hidden="true" tabindex="-1"></a><span class="fu">hist</span>(differences, <span class="at">breaks =</span> <span class="dv">20</span>, </span>
-<span id="cb402-2"><a href="probability-and-statistics-in-r.html#cb402-2" aria-hidden="true" tabindex="-1"></a>     <span class="at">main   =</span> <span class="st">&quot;Empirical Distribution of D&quot;</span>,</span>
-<span id="cb402-3"><a href="probability-and-statistics-in-r.html#cb402-3" aria-hidden="true" tabindex="-1"></a>     <span class="at">xlab   =</span> <span class="st">&quot;Simulated Values of D&quot;</span>,</span>
-<span id="cb402-4"><a href="probability-and-statistics-in-r.html#cb402-4" aria-hidden="true" tabindex="-1"></a>     <span class="at">col    =</span> <span class="st">&quot;dodgerblue&quot;</span>,</span>
-<span id="cb402-5"><a href="probability-and-statistics-in-r.html#cb402-5" aria-hidden="true" tabindex="-1"></a>     <span class="at">border =</span> <span class="st">&quot;darkorange&quot;</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb402"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb402-1"><a href="probability-and-statistics-in-r.html#cb402-1" tabindex="-1"></a><span class="fu">hist</span>(differences, <span class="at">breaks =</span> <span class="dv">20</span>, </span>
+<span id="cb402-2"><a href="probability-and-statistics-in-r.html#cb402-2" tabindex="-1"></a>     <span class="at">main   =</span> <span class="st">&quot;Empirical Distribution of D&quot;</span>,</span>
+<span id="cb402-3"><a href="probability-and-statistics-in-r.html#cb402-3" tabindex="-1"></a>     <span class="at">xlab   =</span> <span class="st">&quot;Simulated Values of D&quot;</span>,</span>
+<span id="cb402-4"><a href="probability-and-statistics-in-r.html#cb402-4" tabindex="-1"></a>     <span class="at">col    =</span> <span class="st">&quot;dodgerblue&quot;</span>,</span>
+<span id="cb402-5"><a href="probability-and-statistics-in-r.html#cb402-5" tabindex="-1"></a>     <span class="at">border =</span> <span class="st">&quot;darkorange&quot;</span>)</span></code></pre></div>
 <p><img src="prob-and-stat_files/figure-html/unnamed-chunk-30-1.png" width="672" /></p>
 <p>Also the sample mean and variance are very close to what we would expect.</p>
-<div class="sourceCode" id="cb403"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb403-1"><a href="probability-and-statistics-in-r.html#cb403-1" aria-hidden="true" tabindex="-1"></a><span class="fu">mean</span>(differences)</span></code></pre></div>
+<div class="sourceCode" id="cb403"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb403-1"><a href="probability-and-statistics-in-r.html#cb403-1" tabindex="-1"></a><span class="fu">mean</span>(differences)</span></code></pre></div>
 <pre><code>## [1] 1.001423</code></pre>
-<div class="sourceCode" id="cb405"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb405-1"><a href="probability-and-statistics-in-r.html#cb405-1" aria-hidden="true" tabindex="-1"></a><span class="fu">var</span>(differences)</span></code></pre></div>
+<div class="sourceCode" id="cb405"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb405-1"><a href="probability-and-statistics-in-r.html#cb405-1" tabindex="-1"></a><span class="fu">var</span>(differences)</span></code></pre></div>
 <pre><code>## [1] 0.3230183</code></pre>
 <p>We could have also accomplished this task with a single line of more “idiomatic” <code>R</code>.</p>
-<div class="sourceCode" id="cb407"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb407-1"><a href="probability-and-statistics-in-r.html#cb407-1" aria-hidden="true" tabindex="-1"></a><span class="fu">set.seed</span>(<span class="dv">42</span>)</span>
-<span id="cb407-2"><a href="probability-and-statistics-in-r.html#cb407-2" aria-hidden="true" tabindex="-1"></a>diffs <span class="ot">=</span> <span class="fu">replicate</span>(<span class="dv">10000</span>, <span class="fu">mean</span>(<span class="fu">rnorm</span>(<span class="dv">25</span>, <span class="dv">6</span>, <span class="dv">2</span>)) <span class="sc">-</span> <span class="fu">mean</span>(<span class="fu">rnorm</span>(<span class="dv">25</span>, <span class="dv">5</span>, <span class="dv">2</span>)))</span></code></pre></div>
+<div class="sourceCode" id="cb407"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb407-1"><a href="probability-and-statistics-in-r.html#cb407-1" tabindex="-1"></a><span class="fu">set.seed</span>(<span class="dv">42</span>)</span>
+<span id="cb407-2"><a href="probability-and-statistics-in-r.html#cb407-2" tabindex="-1"></a>diffs <span class="ot">=</span> <span class="fu">replicate</span>(<span class="dv">10000</span>, <span class="fu">mean</span>(<span class="fu">rnorm</span>(<span class="dv">25</span>, <span class="dv">6</span>, <span class="dv">2</span>)) <span class="sc">-</span> <span class="fu">mean</span>(<span class="fu">rnorm</span>(<span class="dv">25</span>, <span class="dv">5</span>, <span class="dv">2</span>)))</span></code></pre></div>
 <p>Use <code>?replicate</code> to take a look at the documentation for the <code>replicate</code> function and see if you can understand how this line performs the same operations that our <code>for</code> loop above executed.</p>
-<div class="sourceCode" id="cb408"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb408-1"><a href="probability-and-statistics-in-r.html#cb408-1" aria-hidden="true" tabindex="-1"></a><span class="fu">mean</span>(differences <span class="sc">==</span> diffs)</span></code></pre></div>
+<div class="sourceCode" id="cb408"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb408-1"><a href="probability-and-statistics-in-r.html#cb408-1" tabindex="-1"></a><span class="fu">mean</span>(differences <span class="sc">==</span> diffs)</span></code></pre></div>
 <pre><code>## [1] 1</code></pre>
 <p>We see that by setting the same seed for the randomization, we actually obtain identical results!</p>
 </div>
@@ -852,37 +852,37 @@ <h3><span class="header-section-number">5.3.2</span> Distribution of a Sample Me
 \bar{X} \overset{d}{\to} N\left(\mu, \frac{\sigma^2}{n}\right).
 \]</span></p>
 <p>The following verifies this result for a Poisson distribution with <span class="math inline">\(\mu = 10\)</span> and a sample size of <span class="math inline">\(n = 50\)</span>.</p>
-<div class="sourceCode" id="cb410"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb410-1"><a href="probability-and-statistics-in-r.html#cb410-1" aria-hidden="true" tabindex="-1"></a><span class="fu">set.seed</span>(<span class="dv">1337</span>)</span>
-<span id="cb410-2"><a href="probability-and-statistics-in-r.html#cb410-2" aria-hidden="true" tabindex="-1"></a>mu          <span class="ot">=</span> <span class="dv">10</span></span>
-<span id="cb410-3"><a href="probability-and-statistics-in-r.html#cb410-3" aria-hidden="true" tabindex="-1"></a>sample_size <span class="ot">=</span> <span class="dv">50</span></span>
-<span id="cb410-4"><a href="probability-and-statistics-in-r.html#cb410-4" aria-hidden="true" tabindex="-1"></a>samples     <span class="ot">=</span> <span class="dv">100000</span></span>
-<span id="cb410-5"><a href="probability-and-statistics-in-r.html#cb410-5" aria-hidden="true" tabindex="-1"></a>x_bars      <span class="ot">=</span> <span class="fu">rep</span>(<span class="dv">0</span>, samples)</span></code></pre></div>
-<div class="sourceCode" id="cb411"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb411-1"><a href="probability-and-statistics-in-r.html#cb411-1" aria-hidden="true" tabindex="-1"></a><span class="cf">for</span>(i <span class="cf">in</span> <span class="dv">1</span><span class="sc">:</span>samples){</span>
-<span id="cb411-2"><a href="probability-and-statistics-in-r.html#cb411-2" aria-hidden="true" tabindex="-1"></a>  x_bars[i] <span class="ot">=</span> <span class="fu">mean</span>(<span class="fu">rpois</span>(sample_size, <span class="at">lambda =</span> mu))</span>
-<span id="cb411-3"><a href="probability-and-statistics-in-r.html#cb411-3" aria-hidden="true" tabindex="-1"></a>}</span></code></pre></div>
-<div class="sourceCode" id="cb412"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb412-1"><a href="probability-and-statistics-in-r.html#cb412-1" aria-hidden="true" tabindex="-1"></a>x_bar_hist <span class="ot">=</span> <span class="fu">hist</span>(x_bars, <span class="at">breaks =</span> <span class="dv">50</span>, </span>
-<span id="cb412-2"><a href="probability-and-statistics-in-r.html#cb412-2" aria-hidden="true" tabindex="-1"></a>                  <span class="at">main =</span> <span class="st">&quot;Histogram of Sample Means&quot;</span>,</span>
-<span id="cb412-3"><a href="probability-and-statistics-in-r.html#cb412-3" aria-hidden="true" tabindex="-1"></a>                  <span class="at">xlab =</span> <span class="st">&quot;Sample Means&quot;</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb410"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb410-1"><a href="probability-and-statistics-in-r.html#cb410-1" tabindex="-1"></a><span class="fu">set.seed</span>(<span class="dv">1337</span>)</span>
+<span id="cb410-2"><a href="probability-and-statistics-in-r.html#cb410-2" tabindex="-1"></a>mu          <span class="ot">=</span> <span class="dv">10</span></span>
+<span id="cb410-3"><a href="probability-and-statistics-in-r.html#cb410-3" tabindex="-1"></a>sample_size <span class="ot">=</span> <span class="dv">50</span></span>
+<span id="cb410-4"><a href="probability-and-statistics-in-r.html#cb410-4" tabindex="-1"></a>samples     <span class="ot">=</span> <span class="dv">100000</span></span>
+<span id="cb410-5"><a href="probability-and-statistics-in-r.html#cb410-5" tabindex="-1"></a>x_bars      <span class="ot">=</span> <span class="fu">rep</span>(<span class="dv">0</span>, samples)</span></code></pre></div>
+<div class="sourceCode" id="cb411"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb411-1"><a href="probability-and-statistics-in-r.html#cb411-1" tabindex="-1"></a><span class="cf">for</span>(i <span class="cf">in</span> <span class="dv">1</span><span class="sc">:</span>samples){</span>
+<span id="cb411-2"><a href="probability-and-statistics-in-r.html#cb411-2" tabindex="-1"></a>  x_bars[i] <span class="ot">=</span> <span class="fu">mean</span>(<span class="fu">rpois</span>(sample_size, <span class="at">lambda =</span> mu))</span>
+<span id="cb411-3"><a href="probability-and-statistics-in-r.html#cb411-3" tabindex="-1"></a>}</span></code></pre></div>
+<div class="sourceCode" id="cb412"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb412-1"><a href="probability-and-statistics-in-r.html#cb412-1" tabindex="-1"></a>x_bar_hist <span class="ot">=</span> <span class="fu">hist</span>(x_bars, <span class="at">breaks =</span> <span class="dv">50</span>, </span>
+<span id="cb412-2"><a href="probability-and-statistics-in-r.html#cb412-2" tabindex="-1"></a>                  <span class="at">main =</span> <span class="st">&quot;Histogram of Sample Means&quot;</span>,</span>
+<span id="cb412-3"><a href="probability-and-statistics-in-r.html#cb412-3" tabindex="-1"></a>                  <span class="at">xlab =</span> <span class="st">&quot;Sample Means&quot;</span>)</span></code></pre></div>
 <p><img src="prob-and-stat_files/figure-html/unnamed-chunk-36-1.png" width="672" /></p>
 <p>Now we will compare sample statistics from the empirical distribution with their known values based on the parent distribution.</p>
-<div class="sourceCode" id="cb413"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb413-1"><a href="probability-and-statistics-in-r.html#cb413-1" aria-hidden="true" tabindex="-1"></a><span class="fu">c</span>(<span class="fu">mean</span>(x_bars), mu)</span></code></pre></div>
+<div class="sourceCode" id="cb413"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb413-1"><a href="probability-and-statistics-in-r.html#cb413-1" tabindex="-1"></a><span class="fu">c</span>(<span class="fu">mean</span>(x_bars), mu)</span></code></pre></div>
 <pre><code>## [1] 10.00008 10.00000</code></pre>
-<div class="sourceCode" id="cb415"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb415-1"><a href="probability-and-statistics-in-r.html#cb415-1" aria-hidden="true" tabindex="-1"></a><span class="fu">c</span>(<span class="fu">var</span>(x_bars), mu <span class="sc">/</span> sample_size)</span></code></pre></div>
+<div class="sourceCode" id="cb415"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb415-1"><a href="probability-and-statistics-in-r.html#cb415-1" tabindex="-1"></a><span class="fu">c</span>(<span class="fu">var</span>(x_bars), mu <span class="sc">/</span> sample_size)</span></code></pre></div>
 <pre><code>## [1] 0.1989732 0.2000000</code></pre>
-<div class="sourceCode" id="cb417"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb417-1"><a href="probability-and-statistics-in-r.html#cb417-1" aria-hidden="true" tabindex="-1"></a><span class="fu">c</span>(<span class="fu">sd</span>(x_bars), <span class="fu">sqrt</span>(mu) <span class="sc">/</span> <span class="fu">sqrt</span>(sample_size))</span></code></pre></div>
+<div class="sourceCode" id="cb417"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb417-1"><a href="probability-and-statistics-in-r.html#cb417-1" tabindex="-1"></a><span class="fu">c</span>(<span class="fu">sd</span>(x_bars), <span class="fu">sqrt</span>(mu) <span class="sc">/</span> <span class="fu">sqrt</span>(sample_size))</span></code></pre></div>
 <pre><code>## [1] 0.4460641 0.4472136</code></pre>
 <p>And here, we will calculate the proportion of sample means that are within 2 standard deviations of the population mean.</p>
-<div class="sourceCode" id="cb419"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb419-1"><a href="probability-and-statistics-in-r.html#cb419-1" aria-hidden="true" tabindex="-1"></a><span class="fu">mean</span>(x_bars <span class="sc">&gt;</span> mu <span class="sc">-</span> <span class="dv">2</span> <span class="sc">*</span> <span class="fu">sqrt</span>(mu) <span class="sc">/</span> <span class="fu">sqrt</span>(sample_size) <span class="sc">&amp;</span></span>
-<span id="cb419-2"><a href="probability-and-statistics-in-r.html#cb419-2" aria-hidden="true" tabindex="-1"></a>     x_bars <span class="sc">&lt;</span> mu <span class="sc">+</span> <span class="dv">2</span> <span class="sc">*</span> <span class="fu">sqrt</span>(mu) <span class="sc">/</span> <span class="fu">sqrt</span>(sample_size))</span></code></pre></div>
+<div class="sourceCode" id="cb419"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb419-1"><a href="probability-and-statistics-in-r.html#cb419-1" tabindex="-1"></a><span class="fu">mean</span>(x_bars <span class="sc">&gt;</span> mu <span class="sc">-</span> <span class="dv">2</span> <span class="sc">*</span> <span class="fu">sqrt</span>(mu) <span class="sc">/</span> <span class="fu">sqrt</span>(sample_size) <span class="sc">&amp;</span></span>
+<span id="cb419-2"><a href="probability-and-statistics-in-r.html#cb419-2" tabindex="-1"></a>     x_bars <span class="sc">&lt;</span> mu <span class="sc">+</span> <span class="dv">2</span> <span class="sc">*</span> <span class="fu">sqrt</span>(mu) <span class="sc">/</span> <span class="fu">sqrt</span>(sample_size))</span></code></pre></div>
 <pre><code>## [1] 0.95429</code></pre>
 <p>This last histogram uses a bit of a trick to approximately shade the bars that are within two standard deviations of the mean.)</p>
-<div class="sourceCode" id="cb421"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb421-1"><a href="probability-and-statistics-in-r.html#cb421-1" aria-hidden="true" tabindex="-1"></a>shading <span class="ot">=</span> <span class="fu">ifelse</span>(x_bar_hist<span class="sc">$</span>breaks <span class="sc">&gt;</span> mu <span class="sc">-</span> <span class="dv">2</span> <span class="sc">*</span> <span class="fu">sqrt</span>(mu) <span class="sc">/</span> <span class="fu">sqrt</span>(sample_size) <span class="sc">&amp;</span> </span>
-<span id="cb421-2"><a href="probability-and-statistics-in-r.html#cb421-2" aria-hidden="true" tabindex="-1"></a>                   x_bar_hist<span class="sc">$</span>breaks <span class="sc">&lt;</span> mu <span class="sc">+</span> <span class="dv">2</span> <span class="sc">*</span> <span class="fu">sqrt</span>(mu) <span class="sc">/</span> <span class="fu">sqrt</span>(sample_size),</span>
-<span id="cb421-3"><a href="probability-and-statistics-in-r.html#cb421-3" aria-hidden="true" tabindex="-1"></a>                  <span class="st">&quot;darkorange&quot;</span>, <span class="st">&quot;dodgerblue&quot;</span>)</span>
-<span id="cb421-4"><a href="probability-and-statistics-in-r.html#cb421-4" aria-hidden="true" tabindex="-1"></a></span>
-<span id="cb421-5"><a href="probability-and-statistics-in-r.html#cb421-5" aria-hidden="true" tabindex="-1"></a>x_bar_hist <span class="ot">=</span> <span class="fu">hist</span>(x_bars, <span class="at">breaks =</span> <span class="dv">50</span>, <span class="at">col =</span> shading,</span>
-<span id="cb421-6"><a href="probability-and-statistics-in-r.html#cb421-6" aria-hidden="true" tabindex="-1"></a>                  <span class="at">main =</span> <span class="st">&quot;Histogram of Sample Means, Two Standard Deviations&quot;</span>,</span>
-<span id="cb421-7"><a href="probability-and-statistics-in-r.html#cb421-7" aria-hidden="true" tabindex="-1"></a>                  <span class="at">xlab =</span> <span class="st">&quot;Sample Means&quot;</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb421"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb421-1"><a href="probability-and-statistics-in-r.html#cb421-1" tabindex="-1"></a>shading <span class="ot">=</span> <span class="fu">ifelse</span>(x_bar_hist<span class="sc">$</span>breaks <span class="sc">&gt;</span> mu <span class="sc">-</span> <span class="dv">2</span> <span class="sc">*</span> <span class="fu">sqrt</span>(mu) <span class="sc">/</span> <span class="fu">sqrt</span>(sample_size) <span class="sc">&amp;</span> </span>
+<span id="cb421-2"><a href="probability-and-statistics-in-r.html#cb421-2" tabindex="-1"></a>                   x_bar_hist<span class="sc">$</span>breaks <span class="sc">&lt;</span> mu <span class="sc">+</span> <span class="dv">2</span> <span class="sc">*</span> <span class="fu">sqrt</span>(mu) <span class="sc">/</span> <span class="fu">sqrt</span>(sample_size),</span>
+<span id="cb421-3"><a href="probability-and-statistics-in-r.html#cb421-3" tabindex="-1"></a>                  <span class="st">&quot;darkorange&quot;</span>, <span class="st">&quot;dodgerblue&quot;</span>)</span>
+<span id="cb421-4"><a href="probability-and-statistics-in-r.html#cb421-4" tabindex="-1"></a></span>
+<span id="cb421-5"><a href="probability-and-statistics-in-r.html#cb421-5" tabindex="-1"></a>x_bar_hist <span class="ot">=</span> <span class="fu">hist</span>(x_bars, <span class="at">breaks =</span> <span class="dv">50</span>, <span class="at">col =</span> shading,</span>
+<span id="cb421-6"><a href="probability-and-statistics-in-r.html#cb421-6" tabindex="-1"></a>                  <span class="at">main =</span> <span class="st">&quot;Histogram of Sample Means, Two Standard Deviations&quot;</span>,</span>
+<span id="cb421-7"><a href="probability-and-statistics-in-r.html#cb421-7" tabindex="-1"></a>                  <span class="at">xlab =</span> <span class="st">&quot;Sample Means&quot;</span>)</span></code></pre></div>
 <p><img src="prob-and-stat_files/figure-html/unnamed-chunk-41-1.png" width="672" /></p>
 
 </div>
diff --git a/r-intro.md b/r-intro.md
index 516271f..bf21745 100644
--- a/r-intro.md
+++ b/r-intro.md
@@ -5,7 +5,7 @@
 `R` is both a programming language and software environment for statistical computing, which is *free* and *open-source*. To get started, you will need to install two pieces of software:
 
 - [`R`, the actual programming language.](http://cran.r-project.org/){target="_blank"}
-    - Chose your operating system, and select the most recent version, 4.3.2.
+    - Chose your operating system, and select the most recent version, 4.4.1.
 - [RStudio, an excellent IDE for working with `R`.](http://www.rstudio.com/){target="_blank"}
     - Note, you must have `R` installed to use RStudio. RStudio is simply an interface used to interact with `R`.
 
@@ -83,7 +83,7 @@ Note that we will use $\ln$ and $\log$ interchangeably to mean the natural logar
 In using `R` as a calculator, we have seen a number of functions: `sqrt()`, `exp()`, `log()` and `sin()`. To get documentation about a function in `R`, simply put a question mark in front of the function name and RStudio will display the documentation, for example: 
 
 
-```r
+``` r
 ?log
 ?sin
 ?paste
@@ -107,14 +107,14 @@ If you follow these steps, you will get your issue resolved much quicker, and po
 To install a package, use the `install.packages()` function. Think of this as buying a recipe book from the store, bringing it home, and putting it on your shelf.
 
 
-```r
+``` r
 install.packages("ggplot2")
 ```
 
 Once a package is installed, it must be loaded into your current `R` session before being used. Think of this as taking the book off of the shelf and opening it up to read.
 
 
-```r
+``` r
 library(ggplot2)
 ```
 
diff --git a/r-resources.html b/r-resources.html
index 0d341c2..0628701 100644
--- a/r-resources.html
+++ b/r-resources.html
@@ -6,7 +6,7 @@
   <meta http-equiv="X-UA-Compatible" content="IE=edge" />
   <title>Chapter 6 R Resources | Applied Statistics with R</title>
   <meta name="description" content="Chapter 6 R Resources | Applied Statistics with R" />
-  <meta name="generator" content="bookdown 0.37 and GitBook 2.6.7" />
+  <meta name="generator" content="bookdown 0.40 and GitBook 2.6.7" />
 
   <meta property="og:title" content="Chapter 6 R Resources | Applied Statistics with R" />
   <meta property="og:type" content="book" />
@@ -55,7 +55,7 @@
 
 <style type="text/css">
 pre > code.sourceCode { white-space: pre; position: relative; }
-pre > code.sourceCode > span { display: inline-block; line-height: 1.25; }
+pre > code.sourceCode > span { line-height: 1.25; }
 pre > code.sourceCode > span:empty { height: 1.2em; }
 .sourceCode { overflow: visible; }
 code.sourceCode > span { color: inherit; text-decoration: inherit; }
diff --git a/search_index.json b/search_index.json
index 3d27f3f..4abff0f 100644
--- a/search_index.json
+++ b/search_index.json
@@ -1 +1 @@
-[["index.html", "Applied Statistics with R Chapter 1 Introduction 1.1 About This Book 1.2 Conventions 1.3 Acknowledgements 1.4 License", " Applied Statistics with R David Dalpiaz Chapter 1 Introduction Welcome to Applied Statistics with R! 1.1 About This Book This book was originally (and currently) designed for use with STAT 420, Methods of Applied Statistics, at the University of Illinois Urbana-Champaign. It may certainly be used elsewhere, but any references to “this course” in this book specifically refer to STAT 420. This book is under active development. When possible, it would be best to always access the text online to be sure you are using the most up-to-date version. Also, the html version provides additional features such as changing text size, font, and colors. If you are in need of a local copy, a pdf version is continuously maintained, however, because a pdf uses pages, the formatting may not be as functional. (In other words, the author needs to go back and spend some time working on the pdf formatting.) Since this book is under active development you may encounter errors ranging from typos, to broken code, to poorly explained topics. If you do, please let us know! Simply send an email and we will make the changes as soon as possible. (dalpiaz2 AT illinois DOT edu) Or, if you know RMarkdown and are familiar with GitHub, make a pull request and fix an issue yourself! This process is partially automated by the edit button in the top-left corner of the html version. If your suggestion or fix becomes part of the book, you will be added to the list at the end of this chapter. We’ll also link to your GitHub account, or personal website upon request. This text uses MathJax to render mathematical notation for the web. Occasionally, but rarely, a JavaScript error will prevent MathJax from rendering correctly. In this case, you will see the “code” instead of the expected mathematical equations. From experience, this is almost always fixed by simply refreshing the page. You’ll also notice that if you right-click any equation you can obtain the MathML Code (for copying into Microsoft Word) or the TeX command used to generate the equation. \\[ a^2 + b^2 = c^2 \\] 1.2 Conventions R code will be typeset using a monospace font which is syntax highlighted. a = 3 b = 4 sqrt(a^2 + b^2) R output lines, which would appear in the console will begin with ##. They will generally not be syntax highlighted. ## [1] 5 We use the quantity \\(p\\) to refer to the number of \\(\\beta\\) parameters in a linear model, not the number of predictors. Don’t worry if you don’t know what this means yet! 1.3 Acknowledgements Material in this book was heavily influenced by: Alex Stepanov Longtime instructor of STAT 420 at the University of Illinois at Urbana-Champaign. The author of this book actually took Alex’s STAT 420 class many years ago! Alex provided or inspired many of the examples in the text. David Unger Another STAT 420 instructor at the University of Illinois at Urbana-Champaign. Co-taught with the author during the summer of 2016 while this book was first being developed. Provided endless hours of copy editing and countless suggestions. James Balamuta Current graduate student at the University of Illinois at Urbana-Champaign. Provided the initial push to write this book by introducing the author to the bookdown package in R. Also a frequent contributor via GitHub. Your name could be here! Suggest an edit! Correct a typo! If you submit a correction and would like to be listed below, please provide your name as you would like it to appear, as well as a link to a GitHub, LinkedIn, or personal website. Daniel McQuillan Mason Rubenstein Yuhang Wang Zhao Liu Jinfeng Xiao Somu Palaniappan Michael Hung-Yiu Chan Eloise Rosen Kiomars Nassiri Jeff Gerlach Brandon Ching Ray Fix Tyler Kim Yeongho Kim Elmar Langholz Thai Duy Cuong Nguyen Junyoung Kim Sezgin Kucukcoban Tony Ma Radu Manolescu Dileep Pasumarthi Sihun Wang Joseph Wilson Yingkui Lin Andy Siddall Nishant Balepur Durga Krovi Raj Krishnan Ed Pureza Siddharth Singh Schillaci Mcinnis Ivan Valdes Castillo Tony Mu Salman Yousaf Yutaro Nishiyama Regina Sahani Goonetilleke Paul Zuradzki Will Tsai Ellen Veomett David Newman Yixing Zheng Ossama Sybesma 1.4 License This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License. "],["introduction-to-r.html", "Chapter 2 Introduction to R 2.1 Getting Started 2.2 Basic Calculations 2.3 Getting Help 2.4 Installing Packages", " Chapter 2 Introduction to R 2.1 Getting Started R is both a programming language and software environment for statistical computing, which is free and open-source. To get started, you will need to install two pieces of software: R, the actual programming language. Chose your operating system, and select the most recent version, 4.3.2. RStudio, an excellent IDE for working with R. Note, you must have R installed to use RStudio. RStudio is simply an interface used to interact with R. The popularity of R is on the rise, and every day it becomes a better tool for statistical analysis. It even generated this book! (A skill you will learn in this course.) There are many good resources for learning R. The following few chapters will serve as a whirlwind introduction to R. They are by no means meant to be a complete reference for the R language, but simply an introduction to the basics that we will need along the way. Several of the more important topics will be re-stressed as they are actually needed for analyses. These introductory R chapters may feel like an overwhelming amount of information. You are not expected to pick up everything the first time through. You should try all of the code from these chapters, then return to them a number of times as you return to the concepts when performing analyses. R is used both for software development and data analysis. We will operate in a grey area, somewhere between these two tasks. Our main goal will be to analyze data, but we will also perform programming exercises that help illustrate certain concepts. RStudio has a large number of useful keyboard shortcuts. A list of these can be found using a keyboard shortcut – the keyboard shortcut to rule them all: On Windows: Alt + Shift + K On Mac: Option + Shift + K The RStudio team has developed a number of “cheatsheets” for working with both R and RStudio. This particular cheatsheet for “Base” R will summarize many of the concepts in this document. (“Base” R is a name used to differentiate the practice of using built-in R functions, as opposed to using functions from outside packages, in particular, those from the tidyverse. More on this later.) When programming, it is often a good practice to follow a style guide. (Where do spaces go? Tabs or spaces? Underscores or CamelCase when naming variables?) No style guide is “correct,” but it helps to be aware of what others do. The more important thing is to be consistent within your own code. Hadley Wickham Style Guide from Advanced R Google Style Guide For this course, our main deviation from these two guides is the use of = in place of &lt;-. (More on that later.) 2.2 Basic Calculations To get started, we’ll use R like a simple calculator. Addition, Subtraction, Multiplication, and Division Math R Result \\(3 + 2\\) 3 + 2 5 \\(3 - 2\\) 3 - 2 1 \\(3 \\cdot2\\) 3 * 2 6 \\(3 / 2\\) 3 / 2 1.5 Exponents Math R Result \\(3^2\\) 3 ^ 2 9 \\(2^{(-3)}\\) 2 ^ (-3) 0.125 \\(100^{1/2}\\) 100 ^ (1 / 2) 10 \\(\\sqrt{100}\\) sqrt(100) 10 Mathematical Constants Math R Result \\(\\pi\\) pi 3.1415927 \\(e\\) exp(1) 2.7182818 Logarithms Note that we will use \\(\\ln\\) and \\(\\log\\) interchangeably to mean the natural logarithm. There is no ln() in R, instead it uses log() to mean the natural logarithm. Math R Result \\(\\log(e)\\) log(exp(1)) 1 \\(\\log_{10}(1000)\\) log10(1000) 3 \\(\\log_{2}(8)\\) log2(8) 3 \\(\\log_{4}(16)\\) log(16, base = 4) 2 Trigonometry Math R Result \\(\\sin(\\pi / 2)\\) sin(pi / 2) 1 \\(\\cos(0)\\) cos(0) 1 2.3 Getting Help In using R as a calculator, we have seen a number of functions: sqrt(), exp(), log() and sin(). To get documentation about a function in R, simply put a question mark in front of the function name and RStudio will display the documentation, for example: ?log ?sin ?paste ?lm Frequently one of the most difficult things to do when learning R is asking for help. First, you need to decide to ask for help, then you need to know how to ask for help. Your very first line of defense should be to Google your error message or a short description of your issue. (The ability to solve problems using this method is quickly becoming an extremely valuable skill.) If that fails, and it eventually will, you should ask for help. There are a number of things you should include when emailing an instructor, or posting to a help website such as Stack Exchange. Describe what you expect the code to do. State the end goal you are trying to achieve. (Sometimes what you expect the code to do, is not what you want to actually do.) Provide the full text of any errors you have received. Provide enough code to recreate the error. Often for the purpose of this course, you could simply email your entire .R or .Rmd file. Sometimes it is also helpful to include a screenshot of your entire RStudio window when the error occurs. If you follow these steps, you will get your issue resolved much quicker, and possibly learn more in the process. Do not be discouraged by running into errors and difficulties when learning R. (Or any technical skill.) It is simply part of the learning process. 2.4 Installing Packages R comes with a number of built-in functions and datasets, but one of the main strengths of R as an open-source project is its package system. Packages add additional functions and data. Frequently if you want to do something in R, and it is not available by default, there is a good chance that there is a package that will fulfill your needs. To install a package, use the install.packages() function. Think of this as buying a recipe book from the store, bringing it home, and putting it on your shelf. install.packages(&quot;ggplot2&quot;) Once a package is installed, it must be loaded into your current R session before being used. Think of this as taking the book off of the shelf and opening it up to read. library(ggplot2) Once you close R, all the packages are closed and put back on the imaginary shelf. The next time you open R, you do not have to install the package again, but you do have to load any packages you intend to use by invoking library(). "],["data-and-programming.html", "Chapter 3 Data and Programming 3.1 Data Types 3.2 Data Structures 3.3 Programming Basics", " Chapter 3 Data and Programming 3.1 Data Types R has a number of basic data types. Numeric Also known as Double. The default type when dealing with numbers. Examples: 1, 1.0, 42.5 Integer Examples: 1L, 2L, 42L Complex Example: 4 + 2i Logical Two possible values: TRUE and FALSE You can also use T and F, but this is not recommended. NA is also considered logical. Character Examples: \"a\", \"Statistics\", \"1 plus 2.\" 3.2 Data Structures R also has a number of basic data structures. A data structure is either homogeneous (all elements are of the same data type) or heterogeneous (elements can be of more than one data type). Dimension Homogeneous Heterogeneous 1 Vector List 2 Matrix Data Frame 3+ Array 3.2.1 Vectors Many operations in R make heavy use of vectors. Vectors in R are indexed starting at 1. That is what the [1] in the output is indicating, that the first element of the row being displayed is the first element of the vector. Larger vectors will start additional rows with [*] where * is the index of the first element of the row. Possibly the most common way to create a vector in R is using the c() function, which is short for “combine.” As the name suggests, it combines a list of elements separated by commas. c(1, 3, 5, 7, 8, 9) ## [1] 1 3 5 7 8 9 Here R simply outputs this vector. If we would like to store this vector in a variable, we can do so with the assignment operator =. In this case the variable x now holds the vector we just created, and we can access the vector by typing x. x = c(1, 3, 5, 7, 8, 9) x ## [1] 1 3 5 7 8 9 As an aside, there is a long history of the assignment operator in R, partially due to the keys available on the keyboards of the creators of the S language. (Which preceded R.) For simplicity we will use =, but know that often you will see &lt;- as the assignment operator. The pros and cons of these two are well beyond the scope of this book, but know that for our purposes you will have no issue if you simply use =. If you are interested in the weird cases where the difference matters, check out The R Inferno. If you wish to use &lt;-, you will still need to use =, however only for argument passing. Some users like to keep assignment (&lt;-) and argument passing (=) separate. No matter what you choose, the more important thing is that you stay consistent. Also, if working on a larger collaborative project, you should use whatever style is already in place. Because vectors must contain elements that are all the same type, R will automatically coerce to a single type when attempting to create a vector that combines multiple types. c(42, &quot;Statistics&quot;, TRUE) ## [1] &quot;42&quot; &quot;Statistics&quot; &quot;TRUE&quot; c(42, TRUE) ## [1] 42 1 Frequently you may wish to create a vector based on a sequence of numbers. The quickest and easiest way to do this is with the : operator, which creates a sequence of integers between two specified integers. (y = 1:100) ## [1] 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 ## [19] 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 ## [37] 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 ## [55] 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 ## [73] 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 ## [91] 91 92 93 94 95 96 97 98 99 100 Here we see R labeling the rows after the first since this is a large vector. Also, we see that by putting parentheses around the assignment, R both stores the vector in a variable called y and automatically outputs y to the console. Note that scalars do not exist in R. They are simply vectors of length 1. 2 ## [1] 2 To create a sequence that is not limited to consecutive integers, use the seq() function to define a sequence by its start, end, and increment. seq(from = 1.5, to = 4.2, by = 0.1) ## [1] 1.5 1.6 1.7 1.8 1.9 2.0 2.1 2.2 2.3 2.4 2.5 2.6 2.7 2.8 2.9 3.0 3.1 3.2 3.3 ## [20] 3.4 3.5 3.6 3.7 3.8 3.9 4.0 4.1 4.2 We will discuss functions in detail later, but note here that the input labels from, to, and by are optional. seq(1.5, 4.2, 0.1) ## [1] 1.5 1.6 1.7 1.8 1.9 2.0 2.1 2.2 2.3 2.4 2.5 2.6 2.7 2.8 2.9 3.0 3.1 3.2 3.3 ## [20] 3.4 3.5 3.6 3.7 3.8 3.9 4.0 4.1 4.2 Another common operation to create a vector is rep(), which can repeat a single value a number of times. rep(&quot;A&quot;, times = 10) ## [1] &quot;A&quot; &quot;A&quot; &quot;A&quot; &quot;A&quot; &quot;A&quot; &quot;A&quot; &quot;A&quot; &quot;A&quot; &quot;A&quot; &quot;A&quot; The rep() function can be used to repeat a vector some number of times. rep(x, times = 3) ## [1] 1 3 5 7 8 9 1 3 5 7 8 9 1 3 5 7 8 9 We have now seen four different ways to create vectors: c() : seq() rep() So far we have mostly used them in isolation, but they are often used together. c(x, rep(seq(1, 9, 2), 3), c(1, 2, 3), 42, 2:4) ## [1] 1 3 5 7 8 9 1 3 5 7 9 1 3 5 7 9 1 3 5 7 9 1 2 3 42 ## [26] 2 3 4 The length of a vector can be obtained with the length() function. length(x) ## [1] 6 length(y) ## [1] 100 3.2.1.1 Subsetting To subset a vector, we use square brackets, []. x ## [1] 1 3 5 7 8 9 x[1] ## [1] 1 x[3] ## [1] 5 We see that x[1] returns the first element, and x[3] returns the third element. x[-2] ## [1] 1 5 7 8 9 We can also exclude certain indexes, in this case the second element. x[1:3] ## [1] 1 3 5 x[c(1,3,4)] ## [1] 1 5 7 Lastly we see that we can subset based on a vector of indices. All of the above are subsetting a vector using a vector of indexes. (Remember a single number is still a vector.) We could instead use a vector of logical values. z = c(TRUE, TRUE, FALSE, TRUE, TRUE, FALSE) z ## [1] TRUE TRUE FALSE TRUE TRUE FALSE x[z] ## [1] 1 3 7 8 3.2.2 Vectorization One of the biggest strengths of R is its use of vectorized operations. (Frequently the lack of understanding of this concept leads to a belief that R is slow. R is not the fastest language, but it has a reputation for being slower than it really is.) x = 1:10 x + 1 ## [1] 2 3 4 5 6 7 8 9 10 11 2 * x ## [1] 2 4 6 8 10 12 14 16 18 20 2 ^ x ## [1] 2 4 8 16 32 64 128 256 512 1024 sqrt(x) ## [1] 1.000000 1.414214 1.732051 2.000000 2.236068 2.449490 2.645751 2.828427 ## [9] 3.000000 3.162278 log(x) ## [1] 0.0000000 0.6931472 1.0986123 1.3862944 1.6094379 1.7917595 1.9459101 ## [8] 2.0794415 2.1972246 2.3025851 We see that when a function like log() is called on a vector x, a vector is returned which has applied the function to each element of the vector x. 3.2.3 Logical Operators Operator Summary Example Result x &lt; y x less than y 3 &lt; 42 TRUE x &gt; y x greater than y 3 &gt; 42 FALSE x &lt;= y x less than or equal to y 3 &lt;= 42 TRUE x &gt;= y x greater than or equal to y 3 &gt;= 42 FALSE x == y xequal to y 3 == 42 FALSE x != y x not equal to y 3 != 42 TRUE !x not x !(3 &gt; 42) TRUE x | y x or y (3 &gt; 42) | TRUE TRUE x &amp; y x and y (3 &lt; 4) &amp; ( 42 &gt; 13) TRUE In R, logical operators are vectorized. x = c(1, 3, 5, 7, 8, 9) x &gt; 3 ## [1] FALSE FALSE TRUE TRUE TRUE TRUE x &lt; 3 ## [1] TRUE FALSE FALSE FALSE FALSE FALSE x == 3 ## [1] FALSE TRUE FALSE FALSE FALSE FALSE x != 3 ## [1] TRUE FALSE TRUE TRUE TRUE TRUE x == 3 &amp; x != 3 ## [1] FALSE FALSE FALSE FALSE FALSE FALSE x == 3 | x != 3 ## [1] TRUE TRUE TRUE TRUE TRUE TRUE This is extremely useful for subsetting. x[x &gt; 3] ## [1] 5 7 8 9 x[x != 3] ## [1] 1 5 7 8 9 sum(x &gt; 3) ## [1] 4 as.numeric(x &gt; 3) ## [1] 0 0 1 1 1 1 Here we see that using the sum() function on a vector of logical TRUE and FALSE values that is the result of x &gt; 3 results in a numeric result. R is first automatically coercing the logical to numeric where TRUE is 1 and FALSE is 0. This coercion from logical to numeric happens for most mathematical operations. If you are interested in more detail, check out Advanced R. which(x &gt; 3) ## [1] 3 4 5 6 x[which(x &gt; 3)] ## [1] 5 7 8 9 max(x) ## [1] 9 which(x == max(x)) ## [1] 6 which.max(x) ## [1] 6 3.2.4 More Vectorization x = c(1, 3, 5, 7, 8, 9) y = 1:100 x + 2 ## [1] 3 5 7 9 10 11 x + rep(2, 6) ## [1] 3 5 7 9 10 11 x &gt; 3 ## [1] FALSE FALSE TRUE TRUE TRUE TRUE x &gt; rep(3, 6) ## [1] FALSE FALSE TRUE TRUE TRUE TRUE x + y ## Warning in x + y: longer object length is not a multiple of shorter object ## length ## [1] 2 5 8 11 13 15 8 11 14 17 19 21 14 17 20 23 25 27 ## [19] 20 23 26 29 31 33 26 29 32 35 37 39 32 35 38 41 43 45 ## [37] 38 41 44 47 49 51 44 47 50 53 55 57 50 53 56 59 61 63 ## [55] 56 59 62 65 67 69 62 65 68 71 73 75 68 71 74 77 79 81 ## [73] 74 77 80 83 85 87 80 83 86 89 91 93 86 89 92 95 97 99 ## [91] 92 95 98 101 103 105 98 101 104 107 length(x) ## [1] 6 length(y) ## [1] 100 length(y) / length(x) ## [1] 16.66667 (x + y) - y ## Warning in x + y: longer object length is not a multiple of shorter object ## length ## [1] 1 3 5 7 8 9 1 3 5 7 8 9 1 3 5 7 8 9 1 3 5 7 8 9 1 3 5 7 8 9 1 3 5 7 8 9 1 ## [38] 3 5 7 8 9 1 3 5 7 8 9 1 3 5 7 8 9 1 3 5 7 8 9 1 3 5 7 8 9 1 3 5 7 8 9 1 3 ## [75] 5 7 8 9 1 3 5 7 8 9 1 3 5 7 8 9 1 3 5 7 8 9 1 3 5 7 y = 1:60 x + y ## [1] 2 5 8 11 13 15 8 11 14 17 19 21 14 17 20 23 25 27 20 23 26 29 31 33 26 ## [26] 29 32 35 37 39 32 35 38 41 43 45 38 41 44 47 49 51 44 47 50 53 55 57 50 53 ## [51] 56 59 61 63 56 59 62 65 67 69 length(y) / length(x) ## [1] 10 rep(x, 10) + y ## [1] 2 5 8 11 13 15 8 11 14 17 19 21 14 17 20 23 25 27 20 23 26 29 31 33 26 ## [26] 29 32 35 37 39 32 35 38 41 43 45 38 41 44 47 49 51 44 47 50 53 55 57 50 53 ## [51] 56 59 61 63 56 59 62 65 67 69 all(x + y == rep(x, 10) + y) ## [1] TRUE identical(x + y, rep(x, 10) + y) ## [1] TRUE # ?any # ?all.equal x = c(1, 3, 5) y = c(1, 2, 4) x == y ## [1] TRUE FALSE FALSE all(x == y) ## [1] FALSE any(x == y) ## [1] TRUE While all returns TRUE only when all of its arguments are TRUE, any returns TRUE when at least one of its arguments is TRUE. x = c(10 ^ (-8)) y = c(10 ^ (-9)) all(x == y) ## [1] FALSE all.equal(x, y) ## [1] TRUE The all.equal function tests “near equality” with a default tolerance value around 1.5e-8 and returns TRUE if all of its arguments have differences smaller than the tolerance. 3.2.5 Matrices R can also be used for matrix calculations. Matrices have rows and columns containing a single data type. In a matrix, the order of rows and columns is important. (This is not true of data frames, which we will see later.) Matrices can be created using the matrix function. x = 1:9 x ## [1] 1 2 3 4 5 6 7 8 9 X = matrix(x, nrow = 3, ncol = 3) X ## [,1] [,2] [,3] ## [1,] 1 4 7 ## [2,] 2 5 8 ## [3,] 3 6 9 Note here that we are using two different variables: lower case x, which stores a vector and capital X, which stores a matrix. (Following the usual mathematical convention.) We can do this because R is case sensitive. By default the matrix function reorders a vector into columns, but we can also tell R to use rows instead. Y = matrix(x, nrow = 3, ncol = 3, byrow = TRUE) Y ## [,1] [,2] [,3] ## [1,] 1 2 3 ## [2,] 4 5 6 ## [3,] 7 8 9 We can also create a matrix of a specified dimension where every element is the same, in this case 0. Z = matrix(0, 2, 4) Z ## [,1] [,2] [,3] [,4] ## [1,] 0 0 0 0 ## [2,] 0 0 0 0 Like vectors, matrices can be subsetted using square brackets, []. However, since matrices are two-dimensional, we need to specify both a row and a column when subsetting. X ## [,1] [,2] [,3] ## [1,] 1 4 7 ## [2,] 2 5 8 ## [3,] 3 6 9 X[1, 2] ## [1] 4 Here we accessed the element in the first row and the second column. We could also subset an entire row or column. X[1, ] ## [1] 1 4 7 X[, 2] ## [1] 4 5 6 We can also use vectors to subset more than one row or column at a time. Here we subset to the first and third column of the second row. X[2, c(1, 3)] ## [1] 2 8 Matrices can also be created by combining vectors as columns, using cbind, or combining vectors as rows, using rbind. x = 1:9 rev(x) ## [1] 9 8 7 6 5 4 3 2 1 rep(1, 9) ## [1] 1 1 1 1 1 1 1 1 1 rbind(x, rev(x), rep(1, 9)) ## [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] ## x 1 2 3 4 5 6 7 8 9 ## 9 8 7 6 5 4 3 2 1 ## 1 1 1 1 1 1 1 1 1 cbind(col_1 = x, col_2 = rev(x), col_3 = rep(1, 9)) ## col_1 col_2 col_3 ## [1,] 1 9 1 ## [2,] 2 8 1 ## [3,] 3 7 1 ## [4,] 4 6 1 ## [5,] 5 5 1 ## [6,] 6 4 1 ## [7,] 7 3 1 ## [8,] 8 2 1 ## [9,] 9 1 1 When using rbind and cbind you can specify “argument” names that will be used as column names. R can then be used to perform matrix calculations. x = 1:9 y = 9:1 X = matrix(x, 3, 3) Y = matrix(y, 3, 3) X ## [,1] [,2] [,3] ## [1,] 1 4 7 ## [2,] 2 5 8 ## [3,] 3 6 9 Y ## [,1] [,2] [,3] ## [1,] 9 6 3 ## [2,] 8 5 2 ## [3,] 7 4 1 X + Y ## [,1] [,2] [,3] ## [1,] 10 10 10 ## [2,] 10 10 10 ## [3,] 10 10 10 X - Y ## [,1] [,2] [,3] ## [1,] -8 -2 4 ## [2,] -6 0 6 ## [3,] -4 2 8 X * Y ## [,1] [,2] [,3] ## [1,] 9 24 21 ## [2,] 16 25 16 ## [3,] 21 24 9 X / Y ## [,1] [,2] [,3] ## [1,] 0.1111111 0.6666667 2.333333 ## [2,] 0.2500000 1.0000000 4.000000 ## [3,] 0.4285714 1.5000000 9.000000 Note that X * Y is not matrix multiplication. It is element by element multiplication. (Same for X / Y). Instead, matrix multiplication uses %*%. Other matrix functions include t() which gives the transpose of a matrix and solve() which returns the inverse of a square matrix if it is invertible. X %*% Y ## [,1] [,2] [,3] ## [1,] 90 54 18 ## [2,] 114 69 24 ## [3,] 138 84 30 t(X) ## [,1] [,2] [,3] ## [1,] 1 2 3 ## [2,] 4 5 6 ## [3,] 7 8 9 Z = matrix(c(9, 2, -3, 2, 4, -2, -3, -2, 16), 3, byrow = TRUE) Z ## [,1] [,2] [,3] ## [1,] 9 2 -3 ## [2,] 2 4 -2 ## [3,] -3 -2 16 solve(Z) ## [,1] [,2] [,3] ## [1,] 0.12931034 -0.05603448 0.01724138 ## [2,] -0.05603448 0.29094828 0.02586207 ## [3,] 0.01724138 0.02586207 0.06896552 To verify that solve(Z) returns the inverse, we multiply it by Z. We would expect this to return the identity matrix, however we see that this is not the case due to some computational issues. However, R also has the all.equal() function which checks for equality, with some small tolerance which accounts for some computational issues. The identical() function is used to check for exact equality. solve(Z) %*% Z ## [,1] [,2] [,3] ## [1,] 1.000000e+00 -6.245005e-17 0.000000e+00 ## [2,] 8.326673e-17 1.000000e+00 5.551115e-17 ## [3,] 2.775558e-17 0.000000e+00 1.000000e+00 diag(3) ## [,1] [,2] [,3] ## [1,] 1 0 0 ## [2,] 0 1 0 ## [3,] 0 0 1 all.equal(solve(Z) %*% Z, diag(3)) ## [1] TRUE R has a number of matrix specific functions for obtaining dimension and summary information. X = matrix(1:6, 2, 3) X ## [,1] [,2] [,3] ## [1,] 1 3 5 ## [2,] 2 4 6 dim(X) ## [1] 2 3 rowSums(X) ## [1] 9 12 colSums(X) ## [1] 3 7 11 rowMeans(X) ## [1] 3 4 colMeans(X) ## [1] 1.5 3.5 5.5 The diag() function can be used in a number of ways. We can extract the diagonal of a matrix. diag(Z) ## [1] 9 4 16 Or create a matrix with specified elements on the diagonal. (And 0 on the off-diagonals.) diag(1:5) ## [,1] [,2] [,3] [,4] [,5] ## [1,] 1 0 0 0 0 ## [2,] 0 2 0 0 0 ## [3,] 0 0 3 0 0 ## [4,] 0 0 0 4 0 ## [5,] 0 0 0 0 5 Or, lastly, create a square matrix of a certain dimension with 1 for every element of the diagonal and 0 for the off-diagonals. diag(5) ## [,1] [,2] [,3] [,4] [,5] ## [1,] 1 0 0 0 0 ## [2,] 0 1 0 0 0 ## [3,] 0 0 1 0 0 ## [4,] 0 0 0 1 0 ## [5,] 0 0 0 0 1 Calculations with Vectors and Matrices Certain operations in R, for example %*% have different behavior on vectors and matrices. To illustrate this, we will first create two vectors. a_vec = c(1, 2, 3) b_vec = c(2, 2, 2) Note that these are indeed vectors. They are not matrices. c(is.vector(a_vec), is.vector(b_vec)) ## [1] TRUE TRUE c(is.matrix(a_vec), is.matrix(b_vec)) ## [1] FALSE FALSE When this is the case, the %*% operator is used to calculate the dot product, also known as the inner product of the two vectors. The dot product of vectors \\(\\boldsymbol{a} = \\lbrack a_1, a_2, \\cdots a_n \\rbrack\\) and \\(\\boldsymbol{b} = \\lbrack b_1, b_2, \\cdots b_n \\rbrack\\) is defined to be \\[ \\boldsymbol{a} \\cdot \\boldsymbol{b} = \\sum_{i = 1}^{n} a_i b_i = a_1 b_1 + a_2 b_2 + \\cdots a_n b_n. \\] a_vec %*% b_vec # inner product ## [,1] ## [1,] 12 a_vec %o% b_vec # outer product ## [,1] [,2] [,3] ## [1,] 2 2 2 ## [2,] 4 4 4 ## [3,] 6 6 6 The %o% operator is used to calculate the outer product of the two vectors. When vectors are coerced to become matrices, they are column vectors. So a vector of length \\(n\\) becomes an \\(n \\times 1\\) matrix after coercion. as.matrix(a_vec) ## [,1] ## [1,] 1 ## [2,] 2 ## [3,] 3 If we use the %*% operator on matrices, %*% again performs the expected matrix multiplication. So you might expect the following to produce an error, because the dimensions are incorrect. as.matrix(a_vec) %*% b_vec ## [,1] [,2] [,3] ## [1,] 2 2 2 ## [2,] 4 4 4 ## [3,] 6 6 6 At face value this is a \\(3 \\times 1\\) matrix, multiplied by a \\(3 \\times 1\\) matrix. However, when b_vec is automatically coerced to be a matrix, R decided to make it a “row vector,” a \\(1 \\times 3\\) matrix, so that the multiplication has conformable dimensions. If we had coerced both, then R would produce an error. as.matrix(a_vec) %*% as.matrix(b_vec) Another way to calculate a dot product is with the crossprod() function. Given two vectors, the crossprod() function calculates their dot product. The function has a rather misleading name. crossprod(a_vec, b_vec) # inner product ## [,1] ## [1,] 12 tcrossprod(a_vec, b_vec) # outer product ## [,1] [,2] [,3] ## [1,] 2 2 2 ## [2,] 4 4 4 ## [3,] 6 6 6 These functions could be very useful later. When used with matrices \\(X\\) and \\(Y\\) as arguments, it calculates \\[ X^\\top Y. \\] When dealing with linear models, the calculation \\[ X^\\top X \\] is used repeatedly. C_mat = matrix(c(1, 2, 3, 4, 5, 6), 2, 3) D_mat = matrix(c(2, 2, 2, 2, 2, 2), 2, 3) This is useful both as a shortcut for a frequent calculation and as a more efficient implementation than using t() and %*%. crossprod(C_mat, D_mat) ## [,1] [,2] [,3] ## [1,] 6 6 6 ## [2,] 14 14 14 ## [3,] 22 22 22 t(C_mat) %*% D_mat ## [,1] [,2] [,3] ## [1,] 6 6 6 ## [2,] 14 14 14 ## [3,] 22 22 22 all.equal(crossprod(C_mat, D_mat), t(C_mat) %*% D_mat) ## [1] TRUE crossprod(C_mat, C_mat) ## [,1] [,2] [,3] ## [1,] 5 11 17 ## [2,] 11 25 39 ## [3,] 17 39 61 t(C_mat) %*% C_mat ## [,1] [,2] [,3] ## [1,] 5 11 17 ## [2,] 11 25 39 ## [3,] 17 39 61 all.equal(crossprod(C_mat, C_mat), t(C_mat) %*% C_mat) ## [1] TRUE 3.2.6 Lists A list is a one-dimensional heterogeneous data structure. So it is indexed like a vector with a single integer value, but each element can contain an element of any type. # creation list(42, &quot;Hello&quot;, TRUE) ## [[1]] ## [1] 42 ## ## [[2]] ## [1] &quot;Hello&quot; ## ## [[3]] ## [1] TRUE ex_list = list( a = c(1, 2, 3, 4), b = TRUE, c = &quot;Hello!&quot;, d = function(arg = 42) {print(&quot;Hello World!&quot;)}, e = diag(5) ) Lists can be subset using two syntaxes, the $ operator, and square brackets []. The $ operator returns a named element of a list. The [] syntax returns a list, while the [[]] returns an element of a list. ex_list[1] returns a list containing the first element. ex_list[[1]] returns the first element of the list, in this case, a vector. # subsetting ex_list$e ## [,1] [,2] [,3] [,4] [,5] ## [1,] 1 0 0 0 0 ## [2,] 0 1 0 0 0 ## [3,] 0 0 1 0 0 ## [4,] 0 0 0 1 0 ## [5,] 0 0 0 0 1 ex_list[1:2] ## $a ## [1] 1 2 3 4 ## ## $b ## [1] TRUE ex_list[1] ## $a ## [1] 1 2 3 4 ex_list[[1]] ## [1] 1 2 3 4 ex_list[c(&quot;e&quot;, &quot;a&quot;)] ## $e ## [,1] [,2] [,3] [,4] [,5] ## [1,] 1 0 0 0 0 ## [2,] 0 1 0 0 0 ## [3,] 0 0 1 0 0 ## [4,] 0 0 0 1 0 ## [5,] 0 0 0 0 1 ## ## $a ## [1] 1 2 3 4 ex_list[&quot;e&quot;] ## $e ## [,1] [,2] [,3] [,4] [,5] ## [1,] 1 0 0 0 0 ## [2,] 0 1 0 0 0 ## [3,] 0 0 1 0 0 ## [4,] 0 0 0 1 0 ## [5,] 0 0 0 0 1 ex_list[[&quot;e&quot;]] ## [,1] [,2] [,3] [,4] [,5] ## [1,] 1 0 0 0 0 ## [2,] 0 1 0 0 0 ## [3,] 0 0 1 0 0 ## [4,] 0 0 0 1 0 ## [5,] 0 0 0 0 1 ex_list$d ## function(arg = 42) {print(&quot;Hello World!&quot;)} ex_list$d(arg = 1) ## [1] &quot;Hello World!&quot; 3.2.7 Data Frames We have previously seen vectors and matrices for storing data as we introduced R. We will now introduce a data frame which will be the most common way that we store and interact with data in this course. example_data = data.frame(x = c(1, 3, 5, 7, 9, 1, 3, 5, 7, 9), y = c(rep(&quot;Hello&quot;, 9), &quot;Goodbye&quot;), z = rep(c(TRUE, FALSE), 5)) Unlike a matrix, which can be thought of as a vector rearranged into rows and columns, a data frame is not required to have the same data type for each element. A data frame is a list of vectors. So, each vector must contain the same data type, but the different vectors can store different data types. example_data ## x y z ## 1 1 Hello TRUE ## 2 3 Hello FALSE ## 3 5 Hello TRUE ## 4 7 Hello FALSE ## 5 9 Hello TRUE ## 6 1 Hello FALSE ## 7 3 Hello TRUE ## 8 5 Hello FALSE ## 9 7 Hello TRUE ## 10 9 Goodbye FALSE Unlike a list which has more flexibility, the elements of a data frame must all be vectors, and have the same length. example_data$x ## [1] 1 3 5 7 9 1 3 5 7 9 all.equal(length(example_data$x), length(example_data$y), length(example_data$z)) ## [1] TRUE str(example_data) ## &#39;data.frame&#39;: 10 obs. of 3 variables: ## $ x: num 1 3 5 7 9 1 3 5 7 9 ## $ y: chr &quot;Hello&quot; &quot;Hello&quot; &quot;Hello&quot; &quot;Hello&quot; ... ## $ z: logi TRUE FALSE TRUE FALSE TRUE FALSE ... nrow(example_data) ## [1] 10 ncol(example_data) ## [1] 3 dim(example_data) ## [1] 10 3 The data.frame() function above is one way to create a data frame. We can also import data from various file types into R, as well as use data stored in packages. The example data above can also be found here as a .csv file. To read this data into R, we would use the read_csv() function from the readr package. Note that R has a built-in function read.csv() that operates very similarly. The readr function read_csv() has a number of advantages. For example, it is much faster reading larger data. It also uses the tibble package to read the data as a tibble. library(readr) example_data_from_csv = read_csv(&quot;data/example-data.csv&quot;) This particular line of code assumes that the file example_data.csv exists in a folder called data in your current working directory. example_data_from_csv ## # A tibble: 10 × 3 ## x y z ## &lt;dbl&gt; &lt;chr&gt; &lt;lgl&gt; ## 1 1 Hello TRUE ## 2 3 Hello FALSE ## 3 5 Hello TRUE ## 4 7 Hello FALSE ## 5 9 Hello TRUE ## 6 1 Hello FALSE ## 7 3 Hello TRUE ## 8 5 Hello FALSE ## 9 7 Hello TRUE ## 10 9 Goodbye FALSE A tibble is simply a data frame that prints with sanity. Notice in the output above that we are given additional information such as dimension and variable type. The as_tibble() function can be used to coerce a regular data frame to a tibble. library(tibble) example_data = as_tibble(example_data) example_data ## # A tibble: 10 × 3 ## x y z ## &lt;dbl&gt; &lt;chr&gt; &lt;lgl&gt; ## 1 1 Hello TRUE ## 2 3 Hello FALSE ## 3 5 Hello TRUE ## 4 7 Hello FALSE ## 5 9 Hello TRUE ## 6 1 Hello FALSE ## 7 3 Hello TRUE ## 8 5 Hello FALSE ## 9 7 Hello TRUE ## 10 9 Goodbye FALSE Alternatively, we could use the “Import Dataset” feature in RStudio which can be found in the environment window. (By default, the top-right pane of RStudio.) Once completed, this process will automatically generate the code to import a file. The resulting code will be shown in the console window. In recent versions of RStudio, read_csv() is used by default, thus reading in a tibble. Earlier we looked at installing packages, in particular the ggplot2 package. (A package for visualization. While not necessary for this course, it is quickly growing in popularity.) library(ggplot2) Inside the ggplot2 package is a dataset called mpg. By loading the package using the library() function, we can now access mpg. When using data from inside a package, there are three things we would generally like to do: Look at the raw data. Understand the data. (Where did it come from? What are the variables? Etc.) Visualize the data. To look at the data, we have two useful commands: head() and str(). head(mpg, n = 10) ## # A tibble: 10 × 11 ## manufacturer model displ year cyl trans drv cty hwy fl class ## &lt;chr&gt; &lt;chr&gt; &lt;dbl&gt; &lt;int&gt; &lt;int&gt; &lt;chr&gt; &lt;chr&gt; &lt;int&gt; &lt;int&gt; &lt;chr&gt; &lt;chr&gt; ## 1 audi a4 1.8 1999 4 auto… f 18 29 p comp… ## 2 audi a4 1.8 1999 4 manu… f 21 29 p comp… ## 3 audi a4 2 2008 4 manu… f 20 31 p comp… ## 4 audi a4 2 2008 4 auto… f 21 30 p comp… ## 5 audi a4 2.8 1999 6 auto… f 16 26 p comp… ## 6 audi a4 2.8 1999 6 manu… f 18 26 p comp… ## 7 audi a4 3.1 2008 6 auto… f 18 27 p comp… ## 8 audi a4 quattro 1.8 1999 4 manu… 4 18 26 p comp… ## 9 audi a4 quattro 1.8 1999 4 auto… 4 16 25 p comp… ## 10 audi a4 quattro 2 2008 4 manu… 4 20 28 p comp… The function head() will display the first n observations of the data frame. The head() function was more useful before tibbles. Notice that mpg is a tibble already, so the output from head() indicates there are only 10 observations. Note that this applies to head(mpg, n = 10) and not mpg itself. Also note that tibbles print a limited number of rows and columns by default. The last line of the printed output indicates which rows and columns were omitted. mpg ## # A tibble: 234 × 11 ## manufacturer model displ year cyl trans drv cty hwy fl class ## &lt;chr&gt; &lt;chr&gt; &lt;dbl&gt; &lt;int&gt; &lt;int&gt; &lt;chr&gt; &lt;chr&gt; &lt;int&gt; &lt;int&gt; &lt;chr&gt; &lt;chr&gt; ## 1 audi a4 1.8 1999 4 auto… f 18 29 p comp… ## 2 audi a4 1.8 1999 4 manu… f 21 29 p comp… ## 3 audi a4 2 2008 4 manu… f 20 31 p comp… ## 4 audi a4 2 2008 4 auto… f 21 30 p comp… ## 5 audi a4 2.8 1999 6 auto… f 16 26 p comp… ## 6 audi a4 2.8 1999 6 manu… f 18 26 p comp… ## 7 audi a4 3.1 2008 6 auto… f 18 27 p comp… ## 8 audi a4 quattro 1.8 1999 4 manu… 4 18 26 p comp… ## 9 audi a4 quattro 1.8 1999 4 auto… 4 16 25 p comp… ## 10 audi a4 quattro 2 2008 4 manu… 4 20 28 p comp… ## # ℹ 224 more rows The function str() will display the “structure” of the data frame. It will display the number of observations and variables, list the variables, give the type of each variable, and show some elements of each variable. This information can also be found in the “Environment” window in RStudio. str(mpg) ## tibble [234 × 11] (S3: tbl_df/tbl/data.frame) ## $ manufacturer: chr [1:234] &quot;audi&quot; &quot;audi&quot; &quot;audi&quot; &quot;audi&quot; ... ## $ model : chr [1:234] &quot;a4&quot; &quot;a4&quot; &quot;a4&quot; &quot;a4&quot; ... ## $ displ : num [1:234] 1.8 1.8 2 2 2.8 2.8 3.1 1.8 1.8 2 ... ## $ year : int [1:234] 1999 1999 2008 2008 1999 1999 2008 1999 1999 2008 ... ## $ cyl : int [1:234] 4 4 4 4 6 6 6 4 4 4 ... ## $ trans : chr [1:234] &quot;auto(l5)&quot; &quot;manual(m5)&quot; &quot;manual(m6)&quot; &quot;auto(av)&quot; ... ## $ drv : chr [1:234] &quot;f&quot; &quot;f&quot; &quot;f&quot; &quot;f&quot; ... ## $ cty : int [1:234] 18 21 20 21 16 18 18 18 16 20 ... ## $ hwy : int [1:234] 29 29 31 30 26 26 27 26 25 28 ... ## $ fl : chr [1:234] &quot;p&quot; &quot;p&quot; &quot;p&quot; &quot;p&quot; ... ## $ class : chr [1:234] &quot;compact&quot; &quot;compact&quot; &quot;compact&quot; &quot;compact&quot; ... It is important to note that while matrices have rows and columns, data frames (tibbles) instead have observations and variables. When displayed in the console or viewer, each row is an observation and each column is a variable. However generally speaking, their order does not matter, it is simply a side-effect of how the data was entered or stored. In this dataset an observation is for a particular model-year of a car, and the variables describe attributes of the car, for example its highway fuel efficiency. To understand more about the data set, we use the ? operator to pull up the documentation for the data. ?mpg R has a number of functions for quickly working with and extracting basic information from data frames. To quickly obtain a vector of the variable names, we use the names() function. names(mpg) ## [1] &quot;manufacturer&quot; &quot;model&quot; &quot;displ&quot; &quot;year&quot; &quot;cyl&quot; ## [6] &quot;trans&quot; &quot;drv&quot; &quot;cty&quot; &quot;hwy&quot; &quot;fl&quot; ## [11] &quot;class&quot; To access one of the variables as a vector, we use the $ operator. mpg$year ## [1] 1999 1999 2008 2008 1999 1999 2008 1999 1999 2008 2008 1999 1999 2008 2008 ## [16] 1999 2008 2008 2008 2008 2008 1999 2008 1999 1999 2008 2008 2008 2008 2008 ## [31] 1999 1999 1999 2008 1999 2008 2008 1999 1999 1999 1999 2008 2008 2008 1999 ## [46] 1999 2008 2008 2008 2008 1999 1999 2008 2008 2008 1999 1999 1999 2008 2008 ## [61] 2008 1999 2008 1999 2008 2008 2008 2008 2008 2008 1999 1999 2008 1999 1999 ## [76] 1999 2008 1999 1999 1999 2008 2008 1999 1999 1999 1999 1999 2008 1999 2008 ## [91] 1999 1999 2008 2008 1999 1999 2008 2008 2008 1999 1999 1999 1999 1999 2008 ## [106] 2008 2008 2008 1999 1999 2008 2008 1999 1999 2008 1999 1999 2008 2008 2008 ## [121] 2008 2008 2008 2008 1999 1999 2008 2008 2008 2008 1999 2008 2008 1999 1999 ## [136] 1999 2008 1999 2008 2008 1999 1999 1999 2008 2008 2008 2008 1999 1999 2008 ## [151] 1999 1999 2008 2008 1999 1999 1999 2008 2008 1999 1999 2008 2008 2008 2008 ## [166] 1999 1999 1999 1999 2008 2008 2008 2008 1999 1999 1999 1999 2008 2008 1999 ## [181] 1999 2008 2008 1999 1999 2008 1999 1999 2008 2008 1999 1999 2008 1999 1999 ## [196] 1999 2008 2008 1999 2008 1999 1999 2008 1999 1999 2008 2008 1999 1999 2008 ## [211] 2008 1999 1999 1999 1999 2008 2008 2008 2008 1999 1999 1999 1999 1999 1999 ## [226] 2008 2008 1999 1999 2008 2008 1999 1999 2008 mpg$hwy ## [1] 29 29 31 30 26 26 27 26 25 28 27 25 25 25 25 24 25 23 20 15 20 17 17 26 23 ## [26] 26 25 24 19 14 15 17 27 30 26 29 26 24 24 22 22 24 24 17 22 21 23 23 19 18 ## [51] 17 17 19 19 12 17 15 17 17 12 17 16 18 15 16 12 17 17 16 12 15 16 17 15 17 ## [76] 17 18 17 19 17 19 19 17 17 17 16 16 17 15 17 26 25 26 24 21 22 23 22 20 33 ## [101] 32 32 29 32 34 36 36 29 26 27 30 31 26 26 28 26 29 28 27 24 24 24 22 19 20 ## [126] 17 12 19 18 14 15 18 18 15 17 16 18 17 19 19 17 29 27 31 32 27 26 26 25 25 ## [151] 17 17 20 18 26 26 27 28 25 25 24 27 25 26 23 26 26 26 26 25 27 25 27 20 20 ## [176] 19 17 20 17 29 27 31 31 26 26 28 27 29 31 31 26 26 27 30 33 35 37 35 15 18 ## [201] 20 20 22 17 19 18 20 29 26 29 29 24 44 29 26 29 29 29 29 23 24 44 41 29 26 ## [226] 28 29 29 29 28 29 26 26 26 We can use the dim(), nrow() and ncol() functions to obtain information about the dimension of the data frame. dim(mpg) ## [1] 234 11 nrow(mpg) ## [1] 234 ncol(mpg) ## [1] 11 Here nrow() is also the number of observations, which in most cases is the sample size. Subsetting data frames can work much like subsetting matrices using square brackets, [,]. Here, we find fuel efficient vehicles earning over 35 miles per gallon and only display manufacturer, model and year. mpg[mpg$hwy &gt; 35, c(&quot;manufacturer&quot;, &quot;model&quot;, &quot;year&quot;)] ## # A tibble: 6 × 3 ## manufacturer model year ## &lt;chr&gt; &lt;chr&gt; &lt;int&gt; ## 1 honda civic 2008 ## 2 honda civic 2008 ## 3 toyota corolla 2008 ## 4 volkswagen jetta 1999 ## 5 volkswagen new beetle 1999 ## 6 volkswagen new beetle 1999 An alternative would be to use the subset() function, which has a much more readable syntax. subset(mpg, subset = hwy &gt; 35, select = c(&quot;manufacturer&quot;, &quot;model&quot;, &quot;year&quot;)) Lastly, we could use the filter and select functions from the dplyr package which introduces the %&gt;% operator from the magrittr package. This is not necessary for this course, however the dplyr package is something you should be aware of as it is becoming a popular tool in the R world. library(dplyr) mpg %&gt;% filter(hwy &gt; 35) %&gt;% select(manufacturer, model, year) All three approaches produce the same results. Which you use will be largely based on a given situation as well as user preference. When subsetting a data frame, be aware of what is being returned, as sometimes it may be a vector instead of a data frame. Also note that there are differences between subsetting a data frame and a tibble. A data frame operates more like a matrix where it is possible to reduce the subset to a vector. A tibble operates more like a list where it always subsets to another tibble. 3.3 Programming Basics 3.3.1 Control Flow In R, the if/else syntax is: if (...) { some R code } else { more R code } For example, x = 1 y = 3 if (x &gt; y) { z = x * y print(&quot;x is larger than y&quot;) } else { z = x + 5 * y print(&quot;x is less than or equal to y&quot;) } ## [1] &quot;x is less than or equal to y&quot; z ## [1] 16 R also has a special function ifelse() which is very useful. It returns one of two specified values based on a conditional statement. ifelse(4 &gt; 3, 1, 0) ## [1] 1 The real power of ifelse() comes from its ability to be applied to vectors. fib = c(1, 1, 2, 3, 5, 8, 13, 21) ifelse(fib &gt; 6, &quot;Foo&quot;, &quot;Bar&quot;) ## [1] &quot;Bar&quot; &quot;Bar&quot; &quot;Bar&quot; &quot;Bar&quot; &quot;Bar&quot; &quot;Foo&quot; &quot;Foo&quot; &quot;Foo&quot; Now a for loop example, x = 11:15 for (i in 1:5) { x[i] = x[i] * 2 } x ## [1] 22 24 26 28 30 Note that this for loop is very normal in many programming languages, but not in R. In R we would not use a loop, instead we would simply use a vectorized operation. x = 11:15 x = x * 2 x ## [1] 22 24 26 28 30 3.3.2 Functions So far we have been using functions, but haven’t actually discussed some of their details. function_name(arg1 = 10, arg2 = 20) To use a function, you simply type its name, followed by an open parenthesis, then specify values of its arguments, then finish with a closing parenthesis. An argument is a variable which is used in the body of the function. Specifying the values of the arguments is essentially providing the inputs to the function. We can also write our own functions in R. For example, we often like to “standardize” variables, that is, subtracting the sample mean, and dividing by the sample standard deviation. \\[ \\frac{x - \\bar{x}}{s} \\] In R we would write a function to do this. When writing a function, there are three things you must do. Give the function a name. Preferably something that is short, but descriptive. Specify the arguments using function(). Write the body of the function within curly braces, {}. standardize = function(x) { m = mean(x) std = sd(x) result = (x - m) / std result } Here the name of the function is standardize, and the function has a single argument x which is used in the body of function. Note that the output of the final line of the body is what is returned by the function. In this case the function returns the vector stored in the variable result. To test our function, we will take a random sample of size n = 10 from a normal distribution with a mean of 2 and a standard deviation of 5. (test_sample = rnorm(n = 10, mean = 2, sd = 5)) ## [1] 14.1326555 3.6826188 0.4917934 2.7671252 0.2775572 -2.1484609 ## [7] -2.6526105 7.5995334 3.8200937 -9.5182079 standardize(x = test_sample) ## [1] 1.9379339 0.2897899 -0.2134562 0.1454013 -0.2472448 -0.6298681 ## [7] -0.7093808 0.9075522 0.3114719 -1.7921993 This function could be written much more succinctly, simply performing all the operations on one line and immediately returning the result, without storing any of the intermediate results. standardize = function(x) { (x - mean(x)) / sd(x) } When specifying arguments, you can provide default arguments. power_of_num = function(num, power = 2) { num ^ power } Let’s look at a number of ways that we could run this function to perform the operation 10^2 resulting in 100. power_of_num(10) ## [1] 100 power_of_num(10, 2) ## [1] 100 power_of_num(num = 10, power = 2) ## [1] 100 power_of_num(power = 2, num = 10) ## [1] 100 Note that without using the argument names, the order matters. The following code will not evaluate to the same output as the previous example. power_of_num(2, 10) ## [1] 1024 Also, the following line of code would produce an error since arguments without a default value must be specified. power_of_num(power = 5) To further illustrate a function with a default argument, we will write a function that calculates sample variance two ways. By default, it will calculate the unbiased estimate of \\(\\sigma^2\\), which we will call \\(s^2\\). \\[ s^2 = \\frac{1}{n - 1}\\sum_{i=1}^{n}(x - \\bar{x})^2 \\] It will also have the ability to return the biased estimate (based on maximum likelihood) which we will call \\(\\hat{\\sigma}^2\\). \\[ \\hat{\\sigma}^2 = \\frac{1}{n}\\sum_{i=1}^{n}(x - \\bar{x})^2 \\] get_var = function(x, biased = FALSE) { n = length(x) - 1 * !biased (1 / n) * sum((x - mean(x)) ^ 2) } get_var(test_sample) ## [1] 40.20178 get_var(test_sample, biased = FALSE) ## [1] 40.20178 var(test_sample) ## [1] 40.20178 We see the function is working as expected, and when returning the unbiased estimate it matches R’s built-in function var(). Finally, let’s examine the biased estimate of \\(\\sigma^2\\). get_var(test_sample, biased = TRUE) ## [1] 36.18161 "],["summarizing-data.html", "Chapter 4 Summarizing Data 4.1 Summary Statistics 4.2 Plotting", " Chapter 4 Summarizing Data 4.1 Summary Statistics R has built-in functions for a large number of summary statistics. For numeric variables, we can summarize data with the center and spread. We’ll again look at the mpg dataset from the ggplot2 package. library(ggplot2) Central Tendency Measure R Result Mean mean(mpg$cty) 16.8589744 Median median(mpg$cty) 17 Spread Measure R Result Variance var(mpg$cty) 18.1130736 Standard Deviation sd(mpg$cty) 4.2559457 IQR IQR(mpg$cty) 5 Minimum min(mpg$cty) 9 Maximum max(mpg$cty) 35 Range range(mpg$cty) 9, 35 Categorical For categorical variables, counts and percentages can be used for summary. table(mpg$drv) ## ## 4 f r ## 103 106 25 table(mpg$drv) / nrow(mpg) ## ## 4 f r ## 0.4401709 0.4529915 0.1068376 4.2 Plotting Now that we have some data to work with, and we have learned about the data at the most basic level, our next task is to visualize the data. Often, a proper visualization can illuminate features of the data that can inform further analysis. We will look at four methods of visualizing data that we will use throughout the course: Histograms Barplots Boxplots Scatterplots 4.2.1 Histograms When visualizing a single numerical variable, a histogram will be our go-to tool, which can be created in R using the hist() function. hist(mpg$cty) The histogram function has a number of parameters which can be changed to make our plot look much nicer. Use the ? operator to read the documentation for the hist() to see a full list of these parameters. hist(mpg$cty, xlab = &quot;Miles Per Gallon (City)&quot;, main = &quot;Histogram of MPG (City)&quot;, breaks = 12, col = &quot;dodgerblue&quot;, border = &quot;darkorange&quot;) Importantly, you should always be sure to label your axes and give the plot a title. The argument breaks is specific to hist(). Entering an integer will give a suggestion to R for how many bars to use for the histogram. By default R will attempt to intelligently guess a good number of breaks, but as we can see here, it is sometimes useful to modify this yourself. 4.2.2 Barplots Somewhat similar to a histogram, a barplot can provide a visual summary of a categorical variable, or a numeric variable with a finite number of values, like a ranking from 1 to 10. barplot(table(mpg$drv)) barplot(table(mpg$drv), xlab = &quot;Drivetrain (f = FWD, r = RWD, 4 = 4WD)&quot;, ylab = &quot;Frequency&quot;, main = &quot;Drivetrains&quot;, col = &quot;dodgerblue&quot;, border = &quot;darkorange&quot;) 4.2.3 Boxplots To visualize the relationship between a numerical and categorical variable, we will use a boxplot. In the mpg dataset, the drv variable takes a small, finite number of values. A car can only be front wheel drive, 4 wheel drive, or rear wheel drive. unique(mpg$drv) ## [1] &quot;f&quot; &quot;4&quot; &quot;r&quot; First note that we can use a single boxplot as an alternative to a histogram for visualizing a single numerical variable. To do so in R, we use the boxplot() function. boxplot(mpg$hwy) However, more often we will use boxplots to compare a numerical variable for different values of a categorical variable. boxplot(hwy ~ drv, data = mpg) Here we used the boxplot() command to create side-by-side boxplots. However, since we are now dealing with two variables, the syntax has changed. The R syntax hwy ~ drv, data = mpg reads “Plot the hwy variable against the drv variable using the dataset mpg.” We see the use of a ~ (which specifies a formula) and also a data = argument. This will be a syntax that is common to many functions we will use in this course. boxplot(hwy ~ drv, data = mpg, xlab = &quot;Drivetrain (f = FWD, r = RWD, 4 = 4WD)&quot;, ylab = &quot;Miles Per Gallon (Highway)&quot;, main = &quot;MPG (Highway) vs Drivetrain&quot;, pch = 20, cex = 2, col = &quot;darkorange&quot;, border = &quot;dodgerblue&quot;) Again, boxplot() has a number of additional arguments which have the ability to make our plot more visually appealing. 4.2.4 Scatterplots Lastly, to visualize the relationship between two numeric variables we will use a scatterplot. This can be done with the plot() function and the ~ syntax we just used with a boxplot. (The function plot() can also be used more generally; see the documentation for details.) plot(hwy ~ displ, data = mpg) plot(hwy ~ displ, data = mpg, xlab = &quot;Engine Displacement (in Liters)&quot;, ylab = &quot;Miles Per Gallon (Highway)&quot;, main = &quot;MPG (Highway) vs Engine Displacement&quot;, pch = 20, cex = 2, col = &quot;dodgerblue&quot;) "],["probability-and-statistics-in-r.html", "Chapter 5 Probability and Statistics in R 5.1 Probability in R 5.2 Hypothesis Tests in R 5.3 Simulation", " Chapter 5 Probability and Statistics in R 5.1 Probability in R 5.1.1 Distributions When working with different statistical distributions, we often want to make probabilistic statements based on the distribution. We typically want to know one of four things: The density (pdf) at a particular value. The distribution (cdf) at a particular value. The quantile value corresponding to a particular probability. A random draw of values from a particular distribution. This used to be done with statistical tables printed in the back of textbooks. Now, R has functions for obtaining density, distribution, quantile and random values. The general naming structure of the relevant R functions is: dname calculates density (pdf) at input x. pname calculates distribution (cdf) at input x. qname calculates the quantile at an input probability. rname generates a random draw from a particular distribution. Note that name represents the name of the given distribution. For example, consider a random variable \\(X\\) which is \\(N(\\mu = 2, \\sigma^2 = 25)\\). (Note, we are parameterizing using the variance \\(\\sigma^2\\). R however uses the standard deviation.) To calculate the value of the pdf at x = 3, that is, the height of the curve at x = 3, use: dnorm(x = 3, mean = 2, sd = 5) ## [1] 0.07820854 To calculate the value of the cdf at x = 3, that is, \\(P(X \\leq 3)\\), the probability that \\(X\\) is less than or equal to 3, use: pnorm(q = 3, mean = 2, sd = 5) ## [1] 0.5792597 Or, to calculate the quantile for probability 0.975, use: qnorm(p = 0.975, mean = 2, sd = 5) ## [1] 11.79982 Lastly, to generate a random sample of size n = 10, use: rnorm(n = 10, mean = 2, sd = 5) ## [1] 10.761919 5.233989 -1.841460 -2.220900 4.484235 5.504151 1.051961 ## [8] 8.648773 3.080896 5.714909 These functions exist for many other distributions, including but not limited to: Command Distribution *binom Binomial *t t *pois Poisson *f F *chisq Chi-Squared Where * can be d, p, q, and r. Each distribution will have its own set of parameters which need to be passed to the functions as arguments. For example, dbinom() would not have arguments for mean and sd, since those are not parameters of the distribution. Instead a binomial distribution is usually parameterized by \\(n\\) and \\(p\\), however R chooses to call them something else. To find the names that R uses we would use ?dbinom and see that R instead calls the arguments size and prob. For example: dbinom(x = 6, size = 10, prob = 0.75) ## [1] 0.145998 Also note that, when using the dname functions with discrete distributions, they are the pmf of the distribution. For example, the above command is \\(P(Y = 6)\\) if \\(Y \\sim b(n = 10, p = 0.75)\\). (The probability of flipping an unfair coin 10 times and seeing 6 heads, if the probability of heads is 0.75.) 5.2 Hypothesis Tests in R A prerequisite for STAT 420 is an understanding of the basics of hypothesis testing. Recall the basic structure of hypothesis tests: An overall model and related assumptions are made. (The most common being observations following a normal distribution.) The null (\\(H_{0}\\)) and alternative (\\(H_{1}\\) or \\(H_{A}\\)) hypotheses are specified. Usually the null specifies a particular value of a parameter. With given data, the value of the test statistic is calculated. Under the general assumptions, as well as assuming the null hypothesis is true, the distribution of the test statistic is known. Given the distribution and value of the test statistic, as well as the form of the alternative hypothesis, we can calculate a p-value of the test. Based on the p-value and pre-specified level of significance, we make a decision. One of: Fail to reject the null hypothesis. Reject the null hypothesis. We’ll do some quick review of two of the most common tests to show how they are performed using R. 5.2.1 One Sample t-Test: Review Suppose \\(x_{i} \\sim \\mathrm{N}(\\mu,\\sigma^{2})\\) and we want to test \\(H_{0}: \\mu = \\mu_{0}\\) versus \\(H_{1}: \\mu \\neq \\mu_{0}.\\) Assuming \\(\\sigma\\) is unknown, we use the one-sample Student’s \\(t\\) test statistic: \\[ t = \\frac{\\bar{x}-\\mu_{0}}{s/\\sqrt{n}} \\sim t_{n-1}, \\] where \\(\\bar{x} = \\displaystyle\\frac{\\sum_{i=1}^{n}x_{i}}{n}\\) and \\(s = \\sqrt{\\displaystyle\\frac{1}{n - 1}\\sum_{i=1}^{n}(x_i - \\bar{x})^2}\\). A \\(100(1 - \\alpha)\\)% confidence interval for \\(\\mu\\) is given by, \\[ \\bar{x} \\pm t_{\\alpha/2, n-1}\\frac{s}{\\sqrt{n}} \\] where \\(t_{\\alpha/2, n-1}\\) is the critical value such that \\(P\\left(t&gt;t_{\\alpha/2, n-1}\\right) = \\alpha/2\\) for \\(n-1\\) degrees of freedom. 5.2.2 One Sample t-Test: Example Suppose a grocery store sells “16 ounce” boxes of Captain Crisp cereal. A random sample of 9 boxes was taken and weighed. The weight in ounces is stored in the data frame capt_crisp. capt_crisp = data.frame(weight = c(15.5, 16.2, 16.1, 15.8, 15.6, 16.0, 15.8, 15.9, 16.2)) The company that makes Captain Crisp cereal claims that the average weight of a box is at least 16 ounces. We will assume the weight of cereal in a box is normally distributed and use a 0.05 level of significance to test the company’s claim. To test \\(H_{0}: \\mu \\geq 16\\) versus \\(H_{1}: \\mu &lt; 16\\), the test statistic is \\[ t = \\frac{\\bar{x} - \\mu_{0}}{s / \\sqrt{n}} \\] The sample mean \\(\\bar{x}\\) and the sample standard deviation \\(s\\) can be easily computed using R. We also create variables which store the hypothesized mean and the sample size. x_bar = mean(capt_crisp$weight) s = sd(capt_crisp$weight) mu_0 = 16 n = 9 We can then easily compute the test statistic. t = (x_bar - mu_0) / (s / sqrt(n)) t ## [1] -1.2 Under the null hypothesis, the test statistic has a \\(t\\) distribution with \\(n - 1\\) degrees of freedom, in this case 8. To complete the test, we need to obtain the p-value of the test. Since this is a one-sided test with a less-than alternative, we need the area to the left of -1.2 for a \\(t\\) distribution with 8 degrees of freedom. That is, \\[ P(t_{8} &lt; -1.2) \\] pt(t, df = n - 1) ## [1] 0.1322336 We now have the p-value of our test, which is greater than our significance level (0.05), so we fail to reject the null hypothesis. Alternatively, this entire process could have been completed using one line of R code. t.test(x = capt_crisp$weight, mu = 16, alternative = c(&quot;less&quot;), conf.level = 0.95) ## ## One Sample t-test ## ## data: capt_crisp$weight ## t = -1.2, df = 8, p-value = 0.1322 ## alternative hypothesis: true mean is less than 16 ## 95 percent confidence interval: ## -Inf 16.05496 ## sample estimates: ## mean of x ## 15.9 We supply R with the data, the hypothesized value of \\(\\mu\\), the alternative, and the confidence level. R then returns a wealth of information including: The value of the test statistic. The degrees of freedom of the distribution under the null hypothesis. The p-value of the test. The confidence interval which corresponds to the test. An estimate of \\(\\mu\\). Since the test was one-sided, R returned a one-sided confidence interval. If instead we wanted a two-sided interval for the mean weight of boxes of Captain Crisp cereal we could modify our code. capt_test_results = t.test(capt_crisp$weight, mu = 16, alternative = c(&quot;two.sided&quot;), conf.level = 0.95) This time we have stored the results. By doing so, we can directly access portions of the output from t.test(). To see what information is available we use the names() function. names(capt_test_results) ## [1] &quot;statistic&quot; &quot;parameter&quot; &quot;p.value&quot; &quot;conf.int&quot; &quot;estimate&quot; ## [6] &quot;null.value&quot; &quot;stderr&quot; &quot;alternative&quot; &quot;method&quot; &quot;data.name&quot; We are interested in the confidence interval which is stored in conf.int. capt_test_results$conf.int ## [1] 15.70783 16.09217 ## attr(,&quot;conf.level&quot;) ## [1] 0.95 Let’s check this interval “by hand.” The one piece of information we are missing is the critical value, \\(t_{\\alpha/2, n-1} = t_{8}(0.025)\\), which can be calculated in R using the qt() function. qt(0.975, df = 8) ## [1] 2.306004 So, the 95% CI for the mean weight of a cereal box is calculated by plugging into the formula, \\[ \\bar{x} \\pm t_{\\alpha/2, n-1} \\frac{s}{\\sqrt{n}} \\] c(mean(capt_crisp$weight) - qt(0.975, df = 8) * sd(capt_crisp$weight) / sqrt(9), mean(capt_crisp$weight) + qt(0.975, df = 8) * sd(capt_crisp$weight) / sqrt(9)) ## [1] 15.70783 16.09217 5.2.3 Two Sample t-Test: Review Suppose \\(x_{i} \\sim \\mathrm{N}(\\mu_{x}, \\sigma^{2})\\) and \\(y_{i} \\sim \\mathrm{N}(\\mu_{y}, \\sigma^{2}).\\) Want to test \\(H_{0}: \\mu_{x} - \\mu_{y} = \\mu_{0}\\) versus \\(H_{1}: \\mu_{x} - \\mu_{y} \\neq \\mu_{0}.\\) Assuming \\(\\sigma\\) is unknown, use the two-sample Student’s \\(t\\) test statistic: \\[ t = \\frac{(\\bar{x} - \\bar{y})-\\mu_{0}}{s_{p}\\sqrt{\\frac{1}{n}+\\frac{1}{m}}} \\sim t_{n+m-2}, \\] where \\(\\displaystyle\\bar{x}=\\frac{\\sum_{i=1}^{n}x_{i}}{n}\\), \\(\\displaystyle\\bar{y}=\\frac{\\sum_{i=1}^{m}y_{i}}{m}\\), and \\(s_p^2 = \\displaystyle\\frac{(n-1)s_x^2+(m-1)s_y^2}{n+m-2}\\). A \\(100(1-\\alpha)\\)% CI for \\(\\mu_{x}-\\mu_{y}\\) is given by \\[ (\\bar{x} - \\bar{y}) \\pm t_{\\alpha/2, n+m-2}\\left(s_{p}\\textstyle\\sqrt{\\frac{1}{n}+\\frac{1}{m}}\\right), \\] where \\(t_{\\alpha/2, n+m-2}\\) is the critical value such that \\(P\\left(t&gt;t_{\\alpha/2, n+m-2}\\right)=\\alpha/2\\). 5.2.4 Two Sample t-Test: Example Assume that the distributions of \\(X\\) and \\(Y\\) are \\(\\mathrm{N}(\\mu_{1},\\sigma^{2})\\) and \\(\\mathrm{N}(\\mu_{2},\\sigma^{2})\\), respectively. Given the \\(n = 6\\) observations of \\(X\\), x = c(70, 82, 78, 74, 94, 82) n = length(x) and the \\(m = 8\\) observations of \\(Y\\), y = c(64, 72, 60, 76, 72, 80, 84, 68) m = length(y) we will test \\(H_{0}: \\mu_{1} = \\mu_{2}\\) versus \\(H_{1}: \\mu_{1} &gt; \\mu_{2}\\). First, note that we can calculate the sample means and standard deviations. x_bar = mean(x) s_x = sd(x) y_bar = mean(y) s_y = sd(y) We can then calculate the pooled standard deviation. \\[ s_{p} = \\sqrt{\\frac{(n-1)s_{x}^{2}+(m-1)s_{y}^{2}}{n+m-2}} \\] s_p = sqrt(((n - 1) * s_x ^ 2 + (m - 1) * s_y ^ 2) / (n + m - 2)) Thus, the relevant \\(t\\) test statistic is given by \\[ t = \\frac{(\\bar{x}-\\bar{y})-\\mu_{0}}{s_{p}\\sqrt{\\frac{1}{n}+\\frac{1}{m}}}. \\] t = ((x_bar - y_bar) - 0) / (s_p * sqrt(1 / n + 1 / m)) t ## [1] 1.823369 Note that \\(t \\sim t_{n + m - 2} = t_{12}\\), so we can calculate the p-value, which is \\[ P(t_{12} &gt; 1.8233692). \\] 1 - pt(t, df = n + m - 2) ## [1] 0.04661961 But, then again, we could have simply performed this test in one line of R. t.test(x, y, alternative = c(&quot;greater&quot;), var.equal = TRUE) ## ## Two Sample t-test ## ## data: x and y ## t = 1.8234, df = 12, p-value = 0.04662 ## alternative hypothesis: true difference in means is greater than 0 ## 95 percent confidence interval: ## 0.1802451 Inf ## sample estimates: ## mean of x mean of y ## 80 72 Recall that a two-sample \\(t\\)-test can be done with or without an equal variance assumption. Here var.equal = TRUE tells R we would like to perform the test under the equal variance assumption. Above we carried out the analysis using two vectors x and y. In general, we will have a preference for using data frames. t_test_data = data.frame(values = c(x, y), group = c(rep(&quot;A&quot;, length(x)), rep(&quot;B&quot;, length(y)))) We now have the data stored in a single variables (values) and have created a second variable (group) which indicates which “sample” the value belongs to. t_test_data ## values group ## 1 70 A ## 2 82 A ## 3 78 A ## 4 74 A ## 5 94 A ## 6 82 A ## 7 64 B ## 8 72 B ## 9 60 B ## 10 76 B ## 11 72 B ## 12 80 B ## 13 84 B ## 14 68 B Now to perform the test, we still use the t.test() function but with the ~ syntax and a data argument. t.test(values ~ group, data = t_test_data, alternative = c(&quot;greater&quot;), var.equal = TRUE) ## ## Two Sample t-test ## ## data: values by group ## t = 1.8234, df = 12, p-value = 0.04662 ## alternative hypothesis: true difference in means between group A and group B is greater than 0 ## 95 percent confidence interval: ## 0.1802451 Inf ## sample estimates: ## mean in group A mean in group B ## 80 72 5.3 Simulation Simulation and model fitting are related but opposite processes. In simulation, the data generating process is known. We will know the form of the model as well as the value of each of the parameters. In particular, we will often control the distribution and parameters which define the randomness, or noise in the data. In model fitting, the data is known. We will then assume a certain form of model and find the best possible values of the parameters given the observed data. Essentially we are seeking to uncover the truth. Often we will attempt to fit many models, and we will learn metrics to assess which model fits best. Simulation vs Modeling Often we will simulate data according to a process we decide, then use a modeling method seen in class. We can then verify how well the method works, since we know the data generating process. One of the biggest strengths of R is its ability to carry out simulations using built-in functions for generating random samples from certain distributions. We’ll look at two very simple examples here, however simulation will be a topic we revisit several times throughout the course. 5.3.1 Paired Differences Consider the model: \\[ \\begin{split} X_{11}, X_{12}, \\ldots, X_{1n} \\sim N(\\mu_1,\\sigma^2)\\\\ X_{21}, X_{22}, \\ldots, X_{2n} \\sim N(\\mu_2,\\sigma^2) \\end{split} \\] Assume that \\(\\mu_1 = 6\\), \\(\\mu_2 = 5\\), \\(\\sigma^2 = 4\\) and \\(n = 25\\). Let \\[ \\begin{aligned} \\bar{X}_1 &amp;= \\displaystyle\\frac{1}{n}\\sum_{i=1}^{n}X_{1i}\\\\ \\bar{X}_2 &amp;= \\displaystyle\\frac{1}{n}\\sum_{i=1}^{n}X_{2i}\\\\ D &amp;= \\bar{X}_1 - \\bar{X}_2. \\end{aligned} \\] Suppose we would like to calculate \\(P(0 &lt; D &lt; 2)\\). First we will need to obtain the distribution of \\(D\\). Recall, \\[ \\bar{X}_1 \\sim N\\left(\\mu_1,\\frac{\\sigma^2}{n}\\right) \\] and \\[ \\bar{X}_2 \\sim N\\left(\\mu_2,\\frac{\\sigma^2}{n}\\right). \\] Then, \\[ D = \\bar{X}_1 - \\bar{X}_2 \\sim N\\left(\\mu_1-\\mu_2, \\frac{\\sigma^2}{n} + \\frac{\\sigma^2}{n}\\right) = N\\left(6-5, \\frac{4}{25} + \\frac{4}{25}\\right). \\] So, \\[ D \\sim N(\\mu = 1, \\sigma^2 = 0.32). \\] Thus, \\[ P(0 &lt; D &lt; 2) = P(D &lt; 2) - P(D &lt; 0). \\] This can then be calculated using R without a need to first standardize, or use a table. pnorm(2, mean = 1, sd = sqrt(0.32)) - pnorm(0, mean = 1, sd = sqrt(0.32)) ## [1] 0.9229001 An alternative approach, would be to simulate a large number of observations of \\(D\\) then use the empirical distribution to calculate the probability. Our strategy will be to repeatedly: Generate a sample of 25 random observations from \\(N(\\mu_1 = 6,\\sigma^2 = 4)\\). Call the mean of this sample \\(\\bar{x}_{1s}\\). Generate a sample of 25 random observations from \\(N(\\mu_1 = 5,\\sigma^2 = 4)\\). Call the mean of this sample \\(\\bar{x}_{2s}\\). Calculate the differences in the means, \\(d_s = \\bar{x}_{1s} - \\bar{x}_{2s}\\). We will repeat the process a large number of times. Then we will use the distribution of the simulated observations of \\(d_s\\) as an estimate for the true distribution of \\(D\\). set.seed(42) num_samples = 10000 differences = rep(0, num_samples) Before starting our for loop to perform the operation, we set a seed for reproducibility, create and set a variable num_samples which will define the number of repetitions, and lastly create a variables differences which will store the simulate values, \\(d_s\\). By using set.seed() we can reproduce the random results of rnorm() each time starting from that line. for (s in 1:num_samples) { x1 = rnorm(n = 25, mean = 6, sd = 2) x2 = rnorm(n = 25, mean = 5, sd = 2) differences[s] = mean(x1) - mean(x2) } To estimate \\(P(0 &lt; D &lt; 2)\\) we will find the proportion of values of \\(d_s\\) (among the 10^{4} values of \\(d_s\\) generated) that are between 0 and 2. mean(0 &lt; differences &amp; differences &lt; 2) ## [1] 0.9222 Recall that above we derived the distribution of \\(D\\) to be \\(N(\\mu = 1, \\sigma^2 = 0.32)\\) If we look at a histogram of the differences, we find that it looks very much like a normal distribution. hist(differences, breaks = 20, main = &quot;Empirical Distribution of D&quot;, xlab = &quot;Simulated Values of D&quot;, col = &quot;dodgerblue&quot;, border = &quot;darkorange&quot;) Also the sample mean and variance are very close to what we would expect. mean(differences) ## [1] 1.001423 var(differences) ## [1] 0.3230183 We could have also accomplished this task with a single line of more “idiomatic” R. set.seed(42) diffs = replicate(10000, mean(rnorm(25, 6, 2)) - mean(rnorm(25, 5, 2))) Use ?replicate to take a look at the documentation for the replicate function and see if you can understand how this line performs the same operations that our for loop above executed. mean(differences == diffs) ## [1] 1 We see that by setting the same seed for the randomization, we actually obtain identical results! 5.3.2 Distribution of a Sample Mean For another example of simulation, we will simulate observations from a Poisson distribution, and examine the empirical distribution of the sample mean of these observations. Recall, if \\[ X \\sim Pois(\\mu) \\] then \\[ E[X] = \\mu \\] and \\[ Var[X] = \\mu. \\] Also, recall that for a random variable \\(X\\) with finite mean \\(\\mu\\) and finite variance \\(\\sigma^2\\), the central limit theorem tells us that the mean, \\(\\bar{X}\\) of a random sample of size \\(n\\) is approximately normal for large values of \\(n\\). Specifically, as \\(n \\to \\infty\\), \\[ \\bar{X} \\overset{d}{\\to} N\\left(\\mu, \\frac{\\sigma^2}{n}\\right). \\] The following verifies this result for a Poisson distribution with \\(\\mu = 10\\) and a sample size of \\(n = 50\\). set.seed(1337) mu = 10 sample_size = 50 samples = 100000 x_bars = rep(0, samples) for(i in 1:samples){ x_bars[i] = mean(rpois(sample_size, lambda = mu)) } x_bar_hist = hist(x_bars, breaks = 50, main = &quot;Histogram of Sample Means&quot;, xlab = &quot;Sample Means&quot;) Now we will compare sample statistics from the empirical distribution with their known values based on the parent distribution. c(mean(x_bars), mu) ## [1] 10.00008 10.00000 c(var(x_bars), mu / sample_size) ## [1] 0.1989732 0.2000000 c(sd(x_bars), sqrt(mu) / sqrt(sample_size)) ## [1] 0.4460641 0.4472136 And here, we will calculate the proportion of sample means that are within 2 standard deviations of the population mean. mean(x_bars &gt; mu - 2 * sqrt(mu) / sqrt(sample_size) &amp; x_bars &lt; mu + 2 * sqrt(mu) / sqrt(sample_size)) ## [1] 0.95429 This last histogram uses a bit of a trick to approximately shade the bars that are within two standard deviations of the mean.) shading = ifelse(x_bar_hist$breaks &gt; mu - 2 * sqrt(mu) / sqrt(sample_size) &amp; x_bar_hist$breaks &lt; mu + 2 * sqrt(mu) / sqrt(sample_size), &quot;darkorange&quot;, &quot;dodgerblue&quot;) x_bar_hist = hist(x_bars, breaks = 50, col = shading, main = &quot;Histogram of Sample Means, Two Standard Deviations&quot;, xlab = &quot;Sample Means&quot;) "],["r-resources.html", "Chapter 6 R Resources 6.1 Beginner Tutorials and References 6.2 Intermediate References 6.3 Advanced References 6.4 Quick Comparisons to Other Languages 6.5 RStudio and RMarkdown Videos 6.6 RMarkdown Template", " Chapter 6 R Resources So far, we have seen a lot of R, and a lot of R quickly. Again, the preceding chapters were in no way meant to be a complete reference for the R language, but rather an introduction to many of the concepts we will need in this text. The following resources are not necessary for the remainder of this text, but you may find them useful if you would like a deeper understanding of R: 6.1 Beginner Tutorials and References Try R from Code School. An interactive introduction to the basics of R. Useful for getting up to speed on R’s syntax. Quick-R by Robert Kabacoff. A good reference for R basics. R Tutorial by Chi Yau. A combination reference and tutorial for R basics. R Programming for Data Science by Roger Peng A great text for R programming beginners. Discusses R from the ground up, highlighting programming details we might not discuss. 6.2 Intermediate References R for Data Science by Hadley Wickham and Garrett Grolemund. Similar to Advanced R, but focuses more on data analysis, while still introducing programming concepts. Especially useful for working in the tidyverse. The Art of R Programming by Norman Matloff. Gentle introduction to the programming side of R. (Whereas we will focus more on the data analysis side.) A free electronic version is available through the Illinois library. 6.3 Advanced References Advanced R by Hadley Wickham. From the author of several extremely popular R packages. Good follow-up to The Art of R Programming. (And more up-to-date material.) The R Inferno by Patrick Burns. Likens learning the tricks of R to descending through the levels of hell. Very advanced material, but may be important if R becomes a part of your everyday toolkit. Efficient R Programming by Colin Gillespie and Robin Lovelace Discusses both efficient R programs, as well as programming in R efficiently. 6.4 Quick Comparisons to Other Languages Those who are familiar with other languages may find the following “cheatsheets” helpful for transitioning to R. MATLAB, NumPy, Julia Python pandas comparison with R/R Libraries Stata SAS - Look for a resource still! Suggestions welcome. 6.5 RStudio and RMarkdown Videos The following video playlists were made as an introduction to R, RStudio, and RMarkdown for STAT 420 at UIUC. If you are currently using this text for a Coursera course, you can also find updated videos there. R and RStudio Data in R RMarkdown Note that RStudio and RMarkdown are constantly receiving excellent support and updates, so these videos may already contain some outdated information. RStudio provides their own tutorial for RMarkdown. They also have an excellent RStudio “cheatsheet” which visually identifies many of the features available in the IDE. You may also explore some additional “cheatsheets” on their website. 6.6 RMarkdown Template This .zip file contains the files necessary to produce this rendered document. This document is a more complete version of a template than what is seen in the above videos. "],["simple-linear-regression.html", "Chapter 7 Simple Linear Regression 7.1 Modeling 7.2 Least Squares Approach 7.3 Decomposition of Variation 7.4 The lm Function 7.5 Maximum Likelihood Estimation (MLE) Approach 7.6 Simulating SLR 7.7 History 7.8 R Markdown", " Chapter 7 Simple Linear Regression “All models are wrong, but some are useful.” — George E. P. Box After reading this chapter you will be able to: Understand the concept of a model. Describe two ways in which regression coefficients are derived. Estimate and visualize a regression model using R. Interpret regression coefficients and statistics in the context of real-world problems. Use a regression model to make predictions. 7.1 Modeling Let’s consider a simple example of how the speed of a car affects its stopping distance, that is, how far it travels before it comes to a stop. To examine this relationship, we will use the cars dataset, which is a default R dataset. Thus, we don’t need to load a package first; it is immediately available. To get a first look at the data you can use the View() function inside RStudio. View(cars) We could also take a look at the variable names, the dimension of the data frame, and some sample observations with str(). str(cars) ## &#39;data.frame&#39;: 50 obs. of 2 variables: ## $ speed: num 4 4 7 7 8 9 10 10 10 11 ... ## $ dist : num 2 10 4 22 16 10 18 26 34 17 ... As we have seen before with data frames, there are a number of additional functions to access some of this information directly. dim(cars) ## [1] 50 2 nrow(cars) ## [1] 50 ncol(cars) ## [1] 2 Other than the two variable names and the number of observations, this data is still just a bunch of numbers, so we should probably obtain some context. ?cars Reading the documentation we learn that this is data gathered during the 1920s about the speed of cars and the resulting distance it takes for the car to come to a stop. The interesting task here is to determine how far a car travels before stopping, when traveling at a certain speed. So, we will first plot the stopping distance against the speed. plot(dist ~ speed, data = cars, xlab = &quot;Speed (in Miles Per Hour)&quot;, ylab = &quot;Stopping Distance (in Feet)&quot;, main = &quot;Stopping Distance vs Speed&quot;, pch = 20, cex = 2, col = &quot;grey&quot;) Let’s now define some terminology. We have pairs of data, \\((x_i, y_i)\\), for \\(i = 1, 2, \\ldots n\\), where \\(n\\) is the sample size of the dataset. We use \\(i\\) as an index, simply for notation. We use \\(x_i\\) as the predictor (explanatory) variable. The predictor variable is used to help predict or explain the response (target, outcome) variable, \\(y_i\\). Other texts may use the term independent variable instead of predictor and dependent variable in place of response. However, those monikers imply mathematical characteristics that might not be true. While these other terms are not incorrect, independence is already a strictly defined concept in probability. For example, when trying to predict a person’s weight given their height, would it be accurate to say that height is independent of weight? Certainly not, but that is an unintended implication of saying “independent variable.” We prefer to stay away from this nomenclature. In the cars example, we are interested in using the predictor variable speed to predict and explain the response variable dist. Broadly speaking, we would like to model the relationship between \\(X\\) and \\(Y\\) using the form \\[ Y = f(X) + \\epsilon. \\] The function \\(f\\) describes the functional relationship between the two variables, and the \\(\\epsilon\\) term is used to account for error. This indicates that if we plug in a given value of \\(X\\) as input, our output is a value of \\(Y\\), within a certain range of error. You could think of this a number of ways: Response = Prediction + Error Response = Signal + Noise Response = Model + Unexplained Response = Deterministic + Random Response = Explainable + Unexplainable What sort of function should we use for \\(f(X)\\) for the cars data? We could try to model the data with a horizontal line. That is, the model for \\(y\\) does not depend on the value of \\(x\\). (Some function \\(f(X) = c\\).) In the plot below, we see this doesn’t seem to do a very good job. Many of the data points are very far from the orange line representing \\(c\\). This is an example of underfitting. The obvious fix is to make the function \\(f(X)\\) actually depend on \\(x\\). We could also try to model the data with a very “wiggly” function that tries to go through as many of the data points as possible. This also doesn’t seem to work very well. The stopping distance for a speed of 5 mph shouldn’t be off the chart! (Even in 1920.) This is an example of overfitting. (Note that in this example no function will go through every point, since there are some \\(x\\) values that have several possible \\(y\\) values in the data.) Lastly, we could try to model the data with a well-chosen line rather than one of the two extremes previously attempted. The line on the plot below seems to summarize the relationship between stopping distance and speed quite well. As speed increases, the distance required to come to a stop increases. There is still some variation about this line, but it seems to capture the overall trend. With this in mind, we would like to restrict our choice of \\(f(X)\\) to linear functions of \\(X\\). We will write our model using \\(\\beta_1\\) for the slope, and \\(\\beta_0\\) for the intercept, \\[ Y = \\beta_0 + \\beta_1 X + \\epsilon. \\] 7.1.1 Simple Linear Regression Model We now define what we will call the simple linear regression model, \\[ Y_i = \\beta_0 + \\beta_1 x_i + \\epsilon_i \\] where \\[ \\epsilon_i \\sim N(0, \\sigma^2). \\] That is, the \\(\\epsilon_i\\) are independent and identically distributed (iid) normal random variables with mean \\(0\\) and variance \\(\\sigma^2\\). This model has three parameters to be estimated: \\(\\beta_0\\), \\(\\beta_1\\), and \\(\\sigma^2\\), which are fixed, but unknown constants. We have slightly modified our notation here. We are now using \\(Y_i\\) and \\(x_i\\), since we will be fitting this model to a set of \\(n\\) data points, for \\(i = 1, 2, \\ldots n\\). Recall that we use capital \\(Y\\) to indicate a random variable, and lower case \\(y\\) to denote a potential value of the random variable. Since we will have \\(n\\) observations, we have \\(n\\) random variables \\(Y_i\\) and their possible values \\(y_i\\). In the simple linear regression model, the \\(x_i\\) are assumed to be fixed, known constants, and are thus notated with a lower case variable. The response \\(Y_i\\) remains a random variable because of the random behavior of the error variable, \\(\\epsilon_i\\). That is, each response \\(Y_i\\) is tied to an observable \\(x_i\\) and a random, unobservable, \\(\\epsilon_i\\). Essentially, we could explicitly think of the \\(Y_i\\) as having a different distribution for each \\(X_i\\). In other words, \\(Y_i\\) has a conditional distribution dependent on the value of \\(X_i\\), written \\(x_i\\). Doing so, we still make no distributional assumptions of the \\(X_i\\), since we are only interested in the distribution of the \\(Y_i\\) for a particular value \\(x_i\\). \\[ Y_i \\mid X_i \\sim N(\\beta_0 + \\beta_1 x_i, \\sigma^2) \\] The random \\(Y_i\\) are a function of \\(x_i\\), thus we can write its mean as a function of \\(x_i\\), \\[ \\text{E}[Y_i \\mid X_i = x_i] = \\beta_0 + \\beta_1 x_i. \\] However, its variance remains constant for each \\(x_i\\), \\[ \\text{Var}[Y_i \\mid X_i = x_i] = \\sigma^2. \\] This is visually displayed in the image below. We see that for any value \\(x\\), the expected value of \\(Y\\) is \\(\\beta_0 + \\beta_1 x\\). At each value of \\(x\\), \\(Y\\) has the same variance \\(\\sigma^2\\). Simple Linear Regression Model Introductory Statistics (Shafer and Zhang), UC Davis Stat Wiki Often, we directly talk about the assumptions that this model makes. They can be cleverly shortened to LINE. Linear. The relationship between \\(Y\\) and \\(x\\) is linear, of the form \\(\\beta_0 + \\beta_1 x\\). Independent. The errors \\(\\epsilon\\) are independent. Normal. The errors, \\(\\epsilon\\) are normally distributed. That is the “error” around the line follows a normal distribution. Equal Variance. At each value of \\(x\\), the variance of \\(Y\\) is the same, \\(\\sigma^2\\). We are also assuming that the values of \\(x\\) are fixed, that is, not random. We do not make a distributional assumption about the predictor variable. As a side note, we will often refer to simple linear regression as SLR. Some explanation of the name SLR: Simple refers to the fact that we are using a single predictor variable. Later we will use multiple predictor variables. Linear tells us that our model for \\(Y\\) is a linear combination of the predictors \\(X\\). (In this case just the one.) Right now, this always results in a model that is a line, but later we will see how this is not always the case. Regression simply means that we are attempting to measure the relationship between a response variable and (one or more) predictor variables. In the case of SLR, both the response and the predictor are numeric variables. So SLR models \\(Y\\) as a linear function of \\(X\\), but how do we actually define a good line? There are an infinite number of lines we could use, so we will attempt to find one with “small errors.” That is a line with as many points as close to it as possible. The question now becomes, how do we find such a line? There are many approaches we could take. We could find the line that has the smallest maximum distance from any of the points to the line. That is, \\[ \\underset{\\beta_0, \\beta_1}{\\mathrm{argmin}} \\max|y_i - (\\beta_0 + \\beta_1 x_i)|. \\] We could find the line that minimizes the sum of all the distances from the points to the line. That is, \\[ \\underset{\\beta_0, \\beta_1}{\\mathrm{argmin}} \\sum_{i = 1}^{n}|y_i - (\\beta_0 + \\beta_1 x_i)|. \\] We could find the line that minimizes the sum of all the squared distances from the points to the line. That is, \\[ \\underset{\\beta_0, \\beta_1}{\\mathrm{argmin}} \\sum_{i = 1}^{n}(y_i - (\\beta_0 + \\beta_1 x_i))^2. \\] This last option is called the method of least squares. It is essentially the de-facto method for fitting a line to data. (You may have even seen it before in a linear algebra course.) Its popularity is largely due to the fact that it is mathematically “easy.” (Which was important historically, as computers are a modern contraption.) It is also very popular because many relationships are well approximated by a linear function. 7.2 Least Squares Approach Given observations \\((x_i, y_i)\\), for \\(i = 1, 2, \\ldots n\\), we want to find values of \\(\\beta_0\\) and \\(\\beta_1\\) which minimize \\[ f(\\beta_0, \\beta_1) = \\sum_{i = 1}^{n}(y_i - (\\beta_0 + \\beta_1 x_i))^2 = \\sum_{i = 1}^{n}(y_i - \\beta_0 - \\beta_1 x_i)^2. \\] We will call these values \\(\\hat{\\beta}_0\\) and \\(\\hat{\\beta}_1\\). First, we take a partial derivative with respect to both \\(\\beta_0\\) and \\(\\beta_1\\). \\[ \\begin{aligned} \\frac{\\partial f}{\\partial \\beta_0} &amp;= -2 \\sum_{i = 1}^{n}(y_i - \\beta_0 - \\beta_1 x_i) \\\\ \\frac{\\partial f}{\\partial \\beta_1} &amp;= -2 \\sum_{i = 1}^{n}(x_i)(y_i - \\beta_0 - \\beta_1 x_i) \\end{aligned} \\] We then set each of the partial derivatives equal to zero and solve the resulting system of equations. \\[ \\begin{aligned} \\sum_{i = 1}^{n}(y_i - \\beta_0 - \\beta_1 x_i) &amp;= 0 \\\\ \\sum_{i = 1}^{n}(x_i)(y_i - \\beta_0 - \\beta_1 x_i) &amp;= 0 \\end{aligned} \\] While solving the system of equations, one common algebraic rearrangement results in the normal equations. \\[ \\begin{aligned} n \\beta_0 + \\beta_1 \\sum_{i = 1}^{n} x_i &amp;= \\sum_{i = 1}^{n} y_i\\\\ \\beta_0 \\sum_{i = 1}^{n} x_i + \\beta_1 \\sum_{i = 1}^{n} x_i^2 &amp;= \\sum_{i = 1}^{n} x_i y_i \\end{aligned} \\] Finally, we finish solving the system of equations. \\[ \\begin{aligned} \\hat{\\beta}_1 &amp;= \\frac{\\sum_{i = 1}^{n} x_i y_i - \\frac{(\\sum_{i = 1}^{n} x_i)(\\sum_{i = 1}^{n} y_i)}{n}}{\\sum_{i = 1}^{n} x_i^2 - \\frac{(\\sum_{i = 1}^{n} x_i)^2}{n}} = \\frac{S_{xy}}{S_{xx}}\\\\ \\hat{\\beta}_0 &amp;= \\bar{y} - \\hat{\\beta}_1 \\bar{x} \\end{aligned} \\] Here, we have defined some notation for the expression we’ve obtained. Note that they have alternative forms which are much easier to work with. (We won’t do it here, but you can try to prove the equalities below on your own, for “fun”). We use the capital letter \\(S\\) to denote “summation” which replaces the capital letter \\(\\Sigma\\) when we calculate these values based on observed data, \\((x_i ,y_i)\\). The subscripts such as \\(xy\\) denote over which variables the function \\((z - \\bar{z})\\) is applied. \\[ \\begin{aligned} S_{xy} &amp;= \\sum_{i = 1}^{n} x_i y_i - \\frac{(\\sum_{i = 1}^{n} x_i)(\\sum_{i = 1}^{n} y_i)}{n} = \\sum_{i = 1}^{n}(x_i - \\bar{x})(y_i - \\bar{y})\\\\ S_{xx} &amp;= \\sum_{i = 1}^{n} x_i^2 - \\frac{(\\sum_{i = 1}^{n} x_i)^2}{n} = \\sum_{i = 1}^{n}(x_i - \\bar{x})^2\\\\ S_{yy} &amp;= \\sum_{i = 1}^{n} y_i^2 - \\frac{(\\sum_{i = 1}^{n} y_i)^2}{n} = \\sum_{i = 1}^{n}(y_i - \\bar{y})^2 \\end{aligned} \\] Note that these summations \\(S\\) are not to be confused with sample standard deviation \\(s\\). By using the above alternative expressions for \\(S_{xy}\\) and \\(S_{xx}\\), we arrive at a cleaner, more useful expression for \\(\\hat{\\beta}_1\\). \\[ \\hat{\\beta}_1 = \\frac{S_{xy}}{S_{xx}} = \\frac{\\sum_{i = 1}^{n}(x_i - \\bar{x})(y_i - \\bar{y})}{\\sum_{i = 1}^{n}(x_i - \\bar{x})^2} \\] Traditionally we would now calculate \\(\\hat{\\beta}_0\\) and \\(\\hat{\\beta}_1\\) by hand for the cars dataset. However because we are living in the 21st century and are intelligent (or lazy or efficient, depending on your perspective), we will utilize R to do the number crunching for us. To keep some notation consistent with the above mathematics, we will store the response variable as y and the predictor variable as x. x = cars$speed y = cars$dist We then calculate the three sums of squares defined above. Sxy = sum((x - mean(x)) * (y - mean(y))) Sxx = sum((x - mean(x)) ^ 2) Syy = sum((y - mean(y)) ^ 2) c(Sxy, Sxx, Syy) ## [1] 5387.40 1370.00 32538.98 Then finally calculate \\(\\hat{\\beta}_0\\) and \\(\\hat{\\beta}_1\\). beta_1_hat = Sxy / Sxx beta_0_hat = mean(y) - beta_1_hat * mean(x) c(beta_0_hat, beta_1_hat) ## [1] -17.579095 3.932409 What do these values tell us about our dataset? The slope parameter \\(\\beta_1\\) tells us that for an increase in speed of one mile per hour, the mean stopping distance increases by \\(\\beta_1\\). It is important to specify that we are talking about the mean. Recall that \\(\\beta_0 + \\beta_1 x\\) is the mean of \\(Y\\), in this case stopping distance, for a particular value of \\(x\\). (In this case speed.) So \\(\\beta_1\\) tells us how the mean of \\(Y\\) is affected by a change in \\(x\\). Similarly, the estimate \\(\\hat{\\beta}_1 = 3.93\\) tells us that for an increase in speed of one mile per hour, the estimated mean stopping distance increases by \\(3.93\\) feet. Here we should be sure to specify we are discussing an estimated quantity. Recall that \\(\\hat{y}\\) is the estimated mean of \\(Y\\), so \\(\\hat{\\beta}_1\\) tells us how the estimated mean of \\(Y\\) is affected by changing \\(x\\). The intercept parameter \\(\\beta_0\\) tells us the mean stopping distance for a car traveling zero miles per hour. (Not moving.) The estimate \\(\\hat{\\beta}_0 = -17.58\\) tells us that the estimated mean stopping distance for a car traveling zero miles per hour is \\(-17.58\\) feet. So when you apply the brakes to a car that is not moving, it moves backwards? This doesn’t seem right. (Extrapolation, which we will see later, is the issue here.) 7.2.1 Making Predictions We can now write the fitted or estimated line, \\[ \\hat{y} = \\hat{\\beta}_0 + \\hat{\\beta}_1 x. \\] In this case, \\[ \\hat{y} = -17.58 + 3.93 x. \\] We can now use this line to make predictions. First, let’s see the possible \\(x\\) values in the cars dataset. Since some \\(x\\) values may appear more than once, we use the unique() to return each unique value only once. unique(cars$speed) ## [1] 4 7 8 9 10 11 12 13 14 15 16 17 18 19 20 22 23 24 25 Let’s make a prediction for the stopping distance of a car traveling at 8 miles per hour. \\[ \\hat{y} = -17.58 + 3.93 \\times 8 % = 13.88 \\] beta_0_hat + beta_1_hat * 8 ## [1] 13.88018 This tells us that the estimated mean stopping distance of a car traveling at 8 miles per hour is \\(13.88\\). Now let’s make a prediction for the stopping distance of a car traveling at 21 miles per hour. This is considered interpolation as 21 is not an observed value of \\(x\\). (But is in the data range.) We can use the special %in% operator to quickly verify this in R. 8 %in% unique(cars$speed) ## [1] TRUE 21 %in% unique(cars$speed) ## [1] FALSE min(cars$speed) &lt; 21 &amp; 21 &lt; max(cars$speed) ## [1] TRUE \\[ \\hat{y} = -17.58 + 3.93 \\times 21 % = 65 \\] beta_0_hat + beta_1_hat * 21 ## [1] 65.00149 Lastly, we can make a prediction for the stopping distance of a car traveling at 50 miles per hour. This is considered extrapolation as 50 is not an observed value of \\(x\\) and is outside data range. We should be less confident in predictions of this type. range(cars$speed) ## [1] 4 25 range(cars$speed)[1] &lt; 50 &amp; 50 &lt; range(cars$speed)[2] ## [1] FALSE \\[ \\hat{y} = -17.58 + 3.93 \\times 50 % = 179.04 \\] beta_0_hat + beta_1_hat * 50 ## [1] 179.0413 Cars travel 50 miles per hour rather easily today, but not in the 1920s! This is also an issue we saw when interpreting \\(\\hat{\\beta}_0 = -17.58\\), which is equivalent to making a prediction at \\(x = 0\\). We should not be confident in the estimated linear relationship outside of the range of data we have observed. 7.2.2 Residuals If we think of our model as “Response = Prediction + Error,” we can then write it as \\[ y = \\hat{y} + e. \\] We then define a residual to be the observed value minus the predicted value. \\[ e_i = y_i - \\hat{y}_i \\] Let’s calculate the residual for the prediction we made for a car traveling 8 miles per hour. First, we need to obtain the observed value of \\(y\\) for this \\(x\\) value. which(cars$speed == 8) ## [1] 5 cars[5, ] ## speed dist ## 5 8 16 cars[which(cars$speed == 8), ] ## speed dist ## 5 8 16 We can then calculate the residual. \\[ e = 16 - 13.88 = 2.12 \\] 16 - (beta_0_hat + beta_1_hat * 8) ## [1] 2.119825 The positive residual value indicates that the observed stopping distance is actually 2.12 feet more than what was predicted. 7.2.3 Variance Estimation We’ll now use the residuals for each of the points to create an estimate for the variance, \\(\\sigma^2\\). Recall that, \\[ \\text{E}[Y_i \\mid X_i = x_i] = \\beta_0 + \\beta_1 x_i. \\] So, \\[ \\hat{y}_i = \\hat{\\beta}_0 + \\hat{\\beta}_1 x_i \\] is a natural estimate for the mean of \\(Y_i\\) for a given value of \\(x_i\\). Also, recall that when we specified the model, we had three unknown parameters; \\(\\beta_0\\), \\(\\beta_1\\), and \\(\\sigma^2\\). The method of least squares gave us estimates for \\(\\beta_0\\) and \\(\\beta_1\\), however, we have yet to see an estimate for \\(\\sigma^2\\). We will now define \\(s_e^2\\) which will be an estimate for \\(\\sigma^2\\). \\[ \\begin{aligned} s_e^2 &amp;= \\frac{1}{n - 2} \\sum_{i = 1}^{n}(y_i - (\\hat{\\beta}_0 + \\hat{\\beta}_1 x_i))^2 \\\\ &amp;= \\frac{1}{n - 2} \\sum_{i = 1}^{n}(y_i - \\hat{y}_i)^2 \\\\ &amp;= \\frac{1}{n - 2} \\sum_{i = 1}^{n} e_i^2 \\end{aligned} \\] This probably seems like a natural estimate, aside from the use of \\(n - 2\\), which we will put off explaining until the next chapter. It should actually look rather similar to something we have seen before. \\[ s^2 = \\frac{1}{n - 1}\\sum_{i=1}^{n}(x_i - \\bar{x})^2 \\] Here, \\(s^2\\) is the estimate of \\(\\sigma^2\\) when we have a single random variable \\(X\\). In this case \\(\\bar{x}\\) is an estimate of \\(\\mu\\) which is assumed to be the same for each \\(x\\). Now, in the regression case, with \\(s_e^2\\) each \\(y\\) has a different mean because of the relationship with \\(x\\). Thus, for each \\(y_i\\), we use a different estimate of the mean, that is \\(\\hat{y}_i\\). y_hat = beta_0_hat + beta_1_hat * x e = y - y_hat n = length(e) s2_e = sum(e^2) / (n - 2) s2_e ## [1] 236.5317 Just as with the univariate measure of variance, this value of 236.53 doesn’t have a practical interpretation in terms of stopping distance. Taking the square root, however, computes the standard deviation of the residuals, also known as residual standard error. s_e = sqrt(s2_e) s_e ## [1] 15.37959 This tells us that our estimates of mean stopping distance are “typically” off by 15.38 feet. 7.3 Decomposition of Variation We can re-express \\(y_i - \\bar{y}\\), which measures the deviation of an observation from the sample mean, in the following way, \\[ y_i - \\bar{y} = (y_i - \\hat{y}_i) + (\\hat{y}_i - \\bar{y}). \\] This is the common mathematical trick of “adding zero.” In this case we both added and subtracted \\(\\hat{y}_i\\). Here, \\(y_i - \\hat{y}_i\\) measures the deviation of an observation from the fitted regression line and \\(\\hat{y}_i - \\bar{y}\\) measures the deviation of the fitted regression line from the sample mean. If we square then sum both sides of the equation above, we can obtain the following, \\[ \\sum_{i=1}^{n}(y_i - \\bar{y})^2 = \\sum_{i=1}^{n}(y_i - \\hat{y}_i)^2 + \\sum_{i=1}^{n}(\\hat{y}_i - \\bar{y})^2. \\] This should be somewhat alarming or amazing. How is this true? For now we will leave this question unanswered. (Think about this, and maybe try to prove it.) We will now define three of the quantities seen in this equation. Sum of Squares Total \\[ \\text{SST} = \\sum_{i=1}^{n}(y_i - \\bar{y})^2 \\] The quantity “Sum of Squares Total,” or \\(\\text{SST}\\), represents the total variation of the observed \\(y\\) values. This should be a familiar-looking expression. Note that, \\[ s ^ 2 = \\frac{1}{n - 1}\\sum_{i=1}^{n}(y_i - \\bar{y})^2 = \\frac{1}{n - 1} \\text{SST}. \\] Sum of Squares Regression \\[ \\text{SSReg} = \\sum_{i=1}^{n}(\\hat{y}_i - \\bar{y})^2 \\] The quantity “Sum of Squares Regression,” \\(\\text{SSReg}\\), represents the explained variation of the observed \\(y\\) values. Sum of Squares Error \\[ \\text{SSE} = \\text{RSS} = \\sum_{i=1}^{n}(y_i - \\hat{y}_i)^2 \\] The quantity “Sum of Squares Error,” \\(\\text{SSE}\\), represents the unexplained variation of the observed \\(y\\) values. You will often see \\(\\text{SSE}\\) written as \\(\\text{RSS}\\), or “Residual Sum of Squares.” SST = sum((y - mean(y)) ^ 2) SSReg = sum((y_hat - mean(y)) ^ 2) SSE = sum((y - y_hat) ^ 2) c(SST = SST, SSReg = SSReg, SSE = SSE) ## SST SSReg SSE ## 32538.98 21185.46 11353.52 Note that, \\[ s_e^2 = \\frac{\\text{SSE}}{n - 2}. \\] SSE / (n - 2) ## [1] 236.5317 We can use R to verify that this matches our previous calculation of \\(s_e^2\\). s2_e == SSE / (n - 2) ## [1] TRUE These three measures also do not have an important practical interpretation individually. But together, they’re about to reveal a new statistic to help measure the strength of an SLR model. 7.3.1 Coefficient of Determination The coefficient of determination, \\(R^2\\), is defined as \\[ \\begin{aligned} R^2 &amp;= \\frac{\\text{SSReg}}{\\text{SST}} = \\frac{\\sum_{i=1}^{n}(\\hat{y}_i - \\bar{y})^2}{\\sum_{i=1}^{n}(y_i - \\bar{y})^2} \\\\[2.5ex] &amp;= \\frac{\\text{SST} - \\text{SSE}}{\\text{SST}} = 1 - \\frac{\\text{SSE}}{\\text{SST}} \\\\[2.5ex] &amp;= 1 - \\frac{\\sum_{i=1}^{n}(y_i - \\hat{y}_i)^2}{\\sum_{i=1}^{n}(y_i - \\bar{y})^2} = 1 - \\frac{\\sum_{i = 1}^{n}e_i^2}{\\sum_{i=1}^{n}(y_i - \\bar{y})^2} \\end{aligned} \\] The coefficient of determination is interpreted as the proportion of observed variation in \\(y\\) that can be explained by the simple linear regression model. R2 = SSReg / SST R2 ## [1] 0.6510794 For the cars example, we calculate \\(R^2 = 0.65\\). We then say that \\(65\\%\\) of the observed variability in stopping distance is explained by the linear relationship with speed. The following plots visually demonstrate the three “sums of squares” for a simulated dataset which has \\(R^2 = 0.92\\) which is a somewhat high value. Notice in the final plot, that the orange arrows account for a larger proportion of the total arrow. The next plots again visually demonstrate the three “sums of squares,” this time for a simulated dataset which has \\(R^2 = 0.19\\). Notice in the final plot, that now the blue arrows account for a larger proportion of the total arrow. 7.4 The lm Function So far we have done regression by deriving the least squares estimates, then writing simple R commands to perform the necessary calculations. Since this is such a common task, this is functionality that is built directly into R via the lm() command. The lm() command is used to fit linear models which actually account for a broader class of models than simple linear regression, but we will use SLR as our first demonstration of lm(). The lm() function will be one of our most commonly used tools, so you may want to take a look at the documentation by using ?lm. You’ll notice there is a lot of information there, but we will start with just the very basics. This is documentation you will want to return to often. We’ll continue using the cars data, and essentially use the lm() function to check the work we had previously done. stop_dist_model = lm(dist ~ speed, data = cars) This line of code fits our very first linear model. The syntax should look somewhat familiar. We use the dist ~ speed syntax to tell R we would like to model the response variable dist as a linear function of the predictor variable speed. In general, you should think of the syntax as response ~ predictor. The data = cars argument then tells R that the dist and speed variables are from the dataset cars. We then store this result in a variable stop_dist_model. The variable stop_dist_model now contains a wealth of information, and we will now see how to extract and use that information. The first thing we will do is simply output whatever is stored immediately in the variable stop_dist_model. stop_dist_model ## ## Call: ## lm(formula = dist ~ speed, data = cars) ## ## Coefficients: ## (Intercept) speed ## -17.579 3.932 We see that it first tells us the formula we input into R, that is lm(formula = dist ~ speed, data = cars). We also see the coefficients of the model. We can check that these are what we had calculated previously. (Minus some rounding that R is doing when displaying the results. They are stored with full precision.) c(beta_0_hat, beta_1_hat) ## [1] -17.579095 3.932409 Next, it would be nice to add the fitted line to the scatterplot. To do so we will use the abline() function. plot(dist ~ speed, data = cars, xlab = &quot;Speed (in Miles Per Hour)&quot;, ylab = &quot;Stopping Distance (in Feet)&quot;, main = &quot;Stopping Distance vs Speed&quot;, pch = 20, cex = 2, col = &quot;grey&quot;) abline(stop_dist_model, lwd = 3, col = &quot;darkorange&quot;) The abline() function is used to add lines of the form \\(a + bx\\) to a plot. (Hence abline.) When we give it stop_dist_model as an argument, it automatically extracts the regression coefficient estimates (\\(\\hat{\\beta}_0\\) and \\(\\hat{\\beta}_1\\)) and uses them as the intercept and slope of the line. Here we also use lwd to modify the width of the line, as well as col to modify the color of the line. The “thing” that is returned by the lm() function is actually an object of class lm which is a list. The exact details of this are unimportant unless you are seriously interested in the inner-workings of R, but know that we can determine the names of the elements of the list using the names() command. names(stop_dist_model) ## [1] &quot;coefficients&quot; &quot;residuals&quot; &quot;effects&quot; &quot;rank&quot; ## [5] &quot;fitted.values&quot; &quot;assign&quot; &quot;qr&quot; &quot;df.residual&quot; ## [9] &quot;xlevels&quot; &quot;call&quot; &quot;terms&quot; &quot;model&quot; We can then use this information to, for example, access the residuals using the $ operator. stop_dist_model$residuals ## 1 2 3 4 5 6 7 ## 3.849460 11.849460 -5.947766 12.052234 2.119825 -7.812584 -3.744993 ## 8 9 10 11 12 13 14 ## 4.255007 12.255007 -8.677401 2.322599 -15.609810 -9.609810 -5.609810 ## 15 16 17 18 19 20 21 ## -1.609810 -7.542219 0.457781 0.457781 12.457781 -11.474628 -1.474628 ## 22 23 24 25 26 27 28 ## 22.525372 42.525372 -21.407036 -15.407036 12.592964 -13.339445 -5.339445 ## 29 30 31 32 33 34 35 ## -17.271854 -9.271854 0.728146 -11.204263 2.795737 22.795737 30.795737 ## 36 37 38 39 40 41 42 ## -21.136672 -11.136672 10.863328 -29.069080 -13.069080 -9.069080 -5.069080 ## 43 44 45 46 47 48 49 ## 2.930920 -2.933898 -18.866307 -6.798715 15.201285 16.201285 43.201285 ## 50 ## 4.268876 Another way to access stored information in stop_dist_model are the coef(), resid(), and fitted() functions. These return the coefficients, residuals, and fitted values, respectively. coef(stop_dist_model) ## (Intercept) speed ## -17.579095 3.932409 resid(stop_dist_model) ## 1 2 3 4 5 6 7 ## 3.849460 11.849460 -5.947766 12.052234 2.119825 -7.812584 -3.744993 ## 8 9 10 11 12 13 14 ## 4.255007 12.255007 -8.677401 2.322599 -15.609810 -9.609810 -5.609810 ## 15 16 17 18 19 20 21 ## -1.609810 -7.542219 0.457781 0.457781 12.457781 -11.474628 -1.474628 ## 22 23 24 25 26 27 28 ## 22.525372 42.525372 -21.407036 -15.407036 12.592964 -13.339445 -5.339445 ## 29 30 31 32 33 34 35 ## -17.271854 -9.271854 0.728146 -11.204263 2.795737 22.795737 30.795737 ## 36 37 38 39 40 41 42 ## -21.136672 -11.136672 10.863328 -29.069080 -13.069080 -9.069080 -5.069080 ## 43 44 45 46 47 48 49 ## 2.930920 -2.933898 -18.866307 -6.798715 15.201285 16.201285 43.201285 ## 50 ## 4.268876 fitted(stop_dist_model) ## 1 2 3 4 5 6 7 8 ## -1.849460 -1.849460 9.947766 9.947766 13.880175 17.812584 21.744993 21.744993 ## 9 10 11 12 13 14 15 16 ## 21.744993 25.677401 25.677401 29.609810 29.609810 29.609810 29.609810 33.542219 ## 17 18 19 20 21 22 23 24 ## 33.542219 33.542219 33.542219 37.474628 37.474628 37.474628 37.474628 41.407036 ## 25 26 27 28 29 30 31 32 ## 41.407036 41.407036 45.339445 45.339445 49.271854 49.271854 49.271854 53.204263 ## 33 34 35 36 37 38 39 40 ## 53.204263 53.204263 53.204263 57.136672 57.136672 57.136672 61.069080 61.069080 ## 41 42 43 44 45 46 47 48 ## 61.069080 61.069080 61.069080 68.933898 72.866307 76.798715 76.798715 76.798715 ## 49 50 ## 76.798715 80.731124 An R function that is useful in many situations is summary(). We see that when it is called on our model, it returns a good deal of information. By the end of the course, you will know what every value here is used for. For now, you should immediately notice the coefficient estimates, and you may recognize the \\(R^2\\) value we saw earlier. summary(stop_dist_model) ## ## Call: ## lm(formula = dist ~ speed, data = cars) ## ## Residuals: ## Min 1Q Median 3Q Max ## -29.069 -9.525 -2.272 9.215 43.201 ## ## Coefficients: ## Estimate Std. Error t value Pr(&gt;|t|) ## (Intercept) -17.5791 6.7584 -2.601 0.0123 * ## speed 3.9324 0.4155 9.464 1.49e-12 *** ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 ## ## Residual standard error: 15.38 on 48 degrees of freedom ## Multiple R-squared: 0.6511, Adjusted R-squared: 0.6438 ## F-statistic: 89.57 on 1 and 48 DF, p-value: 1.49e-12 The summary() command also returns a list, and we can again use names() to learn what about the elements of this list. names(summary(stop_dist_model)) ## [1] &quot;call&quot; &quot;terms&quot; &quot;residuals&quot; &quot;coefficients&quot; ## [5] &quot;aliased&quot; &quot;sigma&quot; &quot;df&quot; &quot;r.squared&quot; ## [9] &quot;adj.r.squared&quot; &quot;fstatistic&quot; &quot;cov.unscaled&quot; So, for example, if we wanted to directly access the value of \\(R^2\\), instead of copy and pasting it out of the printed statement from summary(), we could do so. summary(stop_dist_model)$r.squared ## [1] 0.6510794 Another value we may want to access is \\(s_e\\), which R calls sigma. summary(stop_dist_model)$sigma ## [1] 15.37959 Note that this is the same result seen earlier as s_e. You may also notice that this value was displayed above as a result of the summary() command, which R labeled the “Residual Standard Error.” \\[ s_e = \\text{RSE} = \\sqrt{\\frac{1}{n - 2}\\sum_{i = 1}^n e_i^2} \\] Often it is useful to talk about \\(s_e\\) (or RSE) instead of \\(s_e^2\\) because of their units. The unit of \\(s_e\\) in the cars example is feet, while the unit of \\(s_e^2\\) is feet-squared. Another useful function, which we will use almost as often as lm() is the predict() function. predict(stop_dist_model, newdata = data.frame(speed = 8)) ## 1 ## 13.88018 The above code reads “predict the stopping distance of a car traveling 8 miles per hour using the stop_dist_model.” Importantly, the second argument to predict() is a data frame that we make in place. We do this so that we can specify that 8 is a value of speed, so that predict knows how to use it with the model stored in stop_dist_model. We see that this result is what we had calculated “by hand” previously. We could also predict multiple values at once. predict(stop_dist_model, newdata = data.frame(speed = c(8, 21, 50))) ## 1 2 3 ## 13.88018 65.00149 179.04134 \\[ \\begin{aligned} \\hat{y} &amp;= -17.58 + 3.93 \\times 8 = 13.88 \\\\ \\hat{y} &amp;= -17.58 + 3.93 \\times 21 = 65 \\\\ \\hat{y} &amp;= -17.58 + 3.93 \\times 50 = 179.04 \\end{aligned} \\] Or we could calculate the fitted value for each of the original data points. We can simply supply the original data frame, cars, since it contains a variables called speed which has the values we would like to predict at. predict(stop_dist_model, newdata = cars) ## 1 2 3 4 5 6 7 8 ## -1.849460 -1.849460 9.947766 9.947766 13.880175 17.812584 21.744993 21.744993 ## 9 10 11 12 13 14 15 16 ## 21.744993 25.677401 25.677401 29.609810 29.609810 29.609810 29.609810 33.542219 ## 17 18 19 20 21 22 23 24 ## 33.542219 33.542219 33.542219 37.474628 37.474628 37.474628 37.474628 41.407036 ## 25 26 27 28 29 30 31 32 ## 41.407036 41.407036 45.339445 45.339445 49.271854 49.271854 49.271854 53.204263 ## 33 34 35 36 37 38 39 40 ## 53.204263 53.204263 53.204263 57.136672 57.136672 57.136672 61.069080 61.069080 ## 41 42 43 44 45 46 47 48 ## 61.069080 61.069080 61.069080 68.933898 72.866307 76.798715 76.798715 76.798715 ## 49 50 ## 76.798715 80.731124 # predict(stop_dist_model, newdata = data.frame(speed = cars$speed)) This is actually equivalent to simply calling predict() on stop_dist_model without a second argument. predict(stop_dist_model) ## 1 2 3 4 5 6 7 8 ## -1.849460 -1.849460 9.947766 9.947766 13.880175 17.812584 21.744993 21.744993 ## 9 10 11 12 13 14 15 16 ## 21.744993 25.677401 25.677401 29.609810 29.609810 29.609810 29.609810 33.542219 ## 17 18 19 20 21 22 23 24 ## 33.542219 33.542219 33.542219 37.474628 37.474628 37.474628 37.474628 41.407036 ## 25 26 27 28 29 30 31 32 ## 41.407036 41.407036 45.339445 45.339445 49.271854 49.271854 49.271854 53.204263 ## 33 34 35 36 37 38 39 40 ## 53.204263 53.204263 53.204263 57.136672 57.136672 57.136672 61.069080 61.069080 ## 41 42 43 44 45 46 47 48 ## 61.069080 61.069080 61.069080 68.933898 72.866307 76.798715 76.798715 76.798715 ## 49 50 ## 76.798715 80.731124 Note that then in this case, this is the same as using fitted(). fitted(stop_dist_model) ## 1 2 3 4 5 6 7 8 ## -1.849460 -1.849460 9.947766 9.947766 13.880175 17.812584 21.744993 21.744993 ## 9 10 11 12 13 14 15 16 ## 21.744993 25.677401 25.677401 29.609810 29.609810 29.609810 29.609810 33.542219 ## 17 18 19 20 21 22 23 24 ## 33.542219 33.542219 33.542219 37.474628 37.474628 37.474628 37.474628 41.407036 ## 25 26 27 28 29 30 31 32 ## 41.407036 41.407036 45.339445 45.339445 49.271854 49.271854 49.271854 53.204263 ## 33 34 35 36 37 38 39 40 ## 53.204263 53.204263 53.204263 57.136672 57.136672 57.136672 61.069080 61.069080 ## 41 42 43 44 45 46 47 48 ## 61.069080 61.069080 61.069080 68.933898 72.866307 76.798715 76.798715 76.798715 ## 49 50 ## 76.798715 80.731124 7.5 Maximum Likelihood Estimation (MLE) Approach Recall the model, \\[ Y_i = \\beta_0 + \\beta_1 x_i + \\epsilon_i \\] where \\(\\epsilon_i \\sim N(0, \\sigma^2)\\). Then we can find the mean and variance of each \\(Y_i\\). \\[ \\text{E}[Y_i \\mid X_i = x_i] = \\beta_0 + \\beta_1 x_i \\] and \\[ \\text{Var}[Y_i \\mid X_i = x_i] = \\sigma^2. \\] Additionally, the \\(Y_i\\) follow a normal distribution conditioned on the \\(x_i\\). \\[ Y_i \\mid X_i \\sim N(\\beta_0 + \\beta_1 x_i, \\sigma^2) \\] Recall that the pdf of a random variable \\(X \\sim N(\\mu, \\sigma^2)\\) is given by \\[ f_{X}(x; \\mu, \\sigma^2) = \\frac{1}{\\sqrt{2\\pi\\sigma^2}} \\exp{\\left[-\\frac{1}{2}\\left(\\frac{x - \\mu}{\\sigma}\\right)^2\\right]}. \\] Then we can write the pdf of each of the \\(Y_i\\) as \\[ f_{Y_i}(y_i; x_i, \\beta_0, \\beta_1, \\sigma^2) = \\frac{1}{\\sqrt{2\\pi\\sigma^2}} \\exp{\\left[-\\frac{1}{2}\\left(\\frac{y_i - (\\beta_0 + \\beta_1 x_i)}{\\sigma}\\right)^2\\right]}. \\] Given \\(n\\) data points \\((x_i, y_i)\\) we can write the likelihood, which is a function of the three parameters \\(\\beta_0\\), \\(\\beta_1\\), and \\(\\sigma^2\\). Since the data have been observed, we use lower case \\(y_i\\) to denote that these values are no longer random. \\[ L(\\beta_0, \\beta_1, \\sigma^2) = \\prod_{i = 1}^{n} \\frac{1}{\\sqrt{2\\pi\\sigma^2}} \\exp{\\left[-\\frac{1}{2}\\left(\\frac{y_i - \\beta_0 - \\beta_1 x_i}{\\sigma}\\right)^2\\right]} \\] Our goal is to find values of \\(\\beta_0\\), \\(\\beta_1\\), and \\(\\sigma^2\\) which maximize this function, which is a straightforward multivariate calculus problem. We’ll start by doing a bit of rearranging to make our task easier. \\[ L(\\beta_0, \\beta_1, \\sigma^2) = \\left(\\frac{1}{\\sqrt{2 \\pi \\sigma^2}}\\right)^n \\exp{\\left[-\\frac{1}{2 \\sigma^2} \\sum_{i = 1}^{n} (y_i - \\beta_0 - \\beta_1 x_i)^2\\right]} \\] Then, as is often the case when finding MLEs, for mathematical convenience we will take the natural logarithm of the likelihood function since log is a monotonically increasing function. Then we will proceed to maximize the log-likelihood, and the resulting estimates will be the same as if we had not taken the log. \\[ \\log L(\\beta_0, \\beta_1, \\sigma^2) = -\\frac{n}{2}\\log(2 \\pi) - \\frac{n}{2}\\log(\\sigma^2) - \\frac{1}{2 \\sigma^2} \\sum_{i = 1}^{n} (y_i - \\beta_0 - \\beta_1 x_i)^2 \\] Note that we use \\(\\log\\) to mean the natural logarithm. We now take a partial derivative with respect to each of the parameters. \\[ \\begin{aligned} \\frac{\\partial \\log L(\\beta_0, \\beta_1, \\sigma^2)}{\\partial \\beta_0} &amp;= \\frac{1}{\\sigma^2} \\sum_{i = 1}^{n} (y_i - \\beta_0 - \\beta_1 x_i)\\\\ \\frac{\\partial \\log L(\\beta_0, \\beta_1, \\sigma^2)}{\\partial \\beta_1} &amp;= \\frac{1}{\\sigma^2} \\sum_{i = 1}^{n}(x_i)(y_i - \\beta_0 - \\beta_1 x_i) \\\\ \\frac{\\partial \\log L(\\beta_0, \\beta_1, \\sigma^2)}{\\partial \\sigma^2} &amp;= -\\frac{n}{2 \\sigma^2} + \\frac{1}{2(\\sigma^2)^2} \\sum_{i = 1}^{n} (y_i - \\beta_0 - \\beta_1 x_i)^2 \\end{aligned} \\] We then set each of the partial derivatives equal to zero and solve the resulting system of equations. \\[ \\begin{aligned} \\sum_{i = 1}^{n} (y_i - \\beta_0 - \\beta_1 x_i) &amp;= 0\\\\ \\sum_{i = 1}^{n}(x_i)(y_i - \\beta_0 - \\beta_1 x_i) &amp;= 0\\\\ -\\frac{n}{2 \\sigma^2} + \\frac{1}{2(\\sigma^2)^2} \\sum_{i = 1}^{n} (y_i - \\beta_0 - \\beta_1 x_i)^2 &amp;= 0 \\end{aligned} \\] You may notice that the first two equations also appear in the least squares approach. Then, skipping the issue of actually checking if we have found a maximum, we then arrive at our estimates. We call these estimates the maximum likelihood estimates. \\[ \\begin{aligned} \\hat{\\beta}_1 &amp;= \\frac{\\sum_{i = 1}^{n} x_i y_i - \\frac{(\\sum_{i = 1}^{n} x_i)(\\sum_{i = 1}^{n} y_i)}{n}}{\\sum_{i = 1}^{n} x_i^2 - \\frac{(\\sum_{i = 1}^{n} x_i)^2}{n}} = \\frac{S_{xy}}{S_{xx}}\\\\ \\hat{\\beta}_0 &amp;= \\bar{y} - \\hat{\\beta}_1 \\bar{x}\\\\ \\hat{\\sigma}^2 &amp;= \\frac{1}{n} \\sum_{i = 1}^{n}(y_i - \\hat{y}_i)^2 \\end{aligned} \\] Note that \\(\\hat{\\beta}_0\\) and \\(\\hat{\\beta}_1\\) are the same as the least squares estimates. However we now have a new estimate of \\(\\sigma^2\\), that is \\(\\hat{\\sigma}^2\\). So we now have two different estimates of \\(\\sigma^2\\). \\[ \\begin{aligned} s_e^2 &amp;= \\frac{1}{n - 2} \\sum_{i = 1}^{n}(y_i - \\hat{y}_i)^2 = \\frac{1}{n - 2} \\sum_{i = 1}^{n}e_i^2 &amp; \\text{Least Squares}\\\\ \\hat{\\sigma}^2 &amp;= \\frac{1}{n} \\sum_{i = 1}^{n}(y_i - \\hat{y}_i)^2 = \\frac{1}{n} \\sum_{i = 1}^{n}e_i^2 &amp; \\text{MLE} \\end{aligned} \\] In the next chapter, we will discuss in detail the difference between these two estimates, which involves biasedness. 7.6 Simulating SLR We return again to more examples of simulation. This will be a common theme! In practice you will almost never have a true model, and you will use data to attempt to recover information about the unknown true model. With simulation, we decide the true model and simulate data from it. Then, we apply a method to the data, in this case least squares. Now, since we know the true model, we can assess how well it did. For this example, we will simulate \\(n = 21\\) observations from the model \\[ Y = 5 - 2 x + \\epsilon. \\] That is \\(\\beta_0 = 5\\), \\(\\beta_1 = -2\\), and let \\(\\epsilon \\sim N(\\mu = 0, \\sigma^2 = 9)\\). Or, even more succinctly we could write \\[ Y \\mid X \\sim N(\\mu = 5 - 2 x, \\sigma^ 2 = 9). \\] We first set the true parameters of the model to be simulated. num_obs = 21 beta_0 = 5 beta_1 = -2 sigma = 3 Next, we obtain simulated values of \\(\\epsilon_i\\) after setting a seed for reproducibility. set.seed(1) epsilon = rnorm(n = num_obs, mean = 0, sd = sigma) Now, since the \\(x_i\\) values in SLR are considered fixed and known, we simply specify 21 values. Another common practice is to generate them from a uniform distribution, and then use them for the remainder of the analysis. x_vals = seq(from = 0, to = 10, length.out = num_obs) # set.seed(1) # x_vals = runif(num_obs, 0, 10) We then generate the \\(y\\) values according to the specified functional relationship. y_vals = beta_0 + beta_1 * x_vals + epsilon The data, \\((x_i, y_i)\\), represent a possible sample from the true distribution. Now to check how well the method of least squares works, we use lm() to fit the model to our simulated data, then take a look at the estimated coefficients. sim_fit = lm(y_vals ~ x_vals) coef(sim_fit) ## (Intercept) x_vals ## 4.832639 -1.831401 And look at that, they aren’t too far from the true parameters we specified! plot(y_vals ~ x_vals) abline(sim_fit) We should say here, that we’re being sort of lazy, and not the good kind of lazy that could be considered efficient. Any time you simulate data, you should consider doing two things: writing a function, and storing the data in a data frame. The function below, sim_slr(), can be used for the same task as above, but is much more flexible. Notice that we provide x to the function, instead of generating x inside the function. In the SLR model, the \\(x_i\\) are considered known values. That is, they are not random, so we do not assume a distribution for the \\(x_i\\). Because of this, we will repeatedly use the same x values across all simulations. sim_slr = function(x, beta_0 = 10, beta_1 = 5, sigma = 1) { n = length(x) epsilon = rnorm(n, mean = 0, sd = sigma) y = beta_0 + beta_1 * x + epsilon data.frame(predictor = x, response = y) } Here, we use the function to repeat the analysis above. set.seed(1) sim_data = sim_slr(x = x_vals, beta_0 = 5, beta_1 = -2, sigma = 3) This time, the simulated observations are stored in a data frame. head(sim_data) ## predictor response ## 1 0.0 3.1206386 ## 2 0.5 4.5509300 ## 3 1.0 0.4931142 ## 4 1.5 6.7858424 ## 5 2.0 1.9885233 ## 6 2.5 -2.4614052 Now when we fit the model with lm() we can use a data argument, a very good practice. sim_fit = lm(response ~ predictor, data = sim_data) coef(sim_fit) ## (Intercept) predictor ## 4.832639 -1.831401 And this time, we’ll make the plot look a lot nicer. plot(response ~ predictor, data = sim_data, xlab = &quot;Simulated Predictor Variable&quot;, ylab = &quot;Simulated Response Variable&quot;, main = &quot;Simulated Regression Data&quot;, pch = 20, cex = 2, col = &quot;grey&quot;) abline(sim_fit, lwd = 3, lty = 1, col = &quot;darkorange&quot;) abline(beta_0, beta_1, lwd = 3, lty = 2, col = &quot;dodgerblue&quot;) legend(&quot;topright&quot;, c(&quot;Estimate&quot;, &quot;Truth&quot;), lty = c(1, 2), lwd = 2, col = c(&quot;darkorange&quot;, &quot;dodgerblue&quot;)) 7.7 History For some brief background on the history of linear regression, see “Galton, Pearson, and the Peas: A Brief History of Linear Regression for Statistics Instructors” from the Journal of Statistics Education as well as the Wikipedia page on the history of regression analysis and lastly the article for regression toward the mean which details the origins of the term “regression.” 7.8 R Markdown The R Markdown file for this chapter can be found here: slr.Rmd The file was created using R version 4.3.2. "],["inference-for-simple-linear-regression.html", "Chapter 8 Inference for Simple Linear Regression 8.1 Gauss–Markov Theorem 8.2 Sampling Distributions 8.3 Standard Errors 8.4 Confidence Intervals for Slope and Intercept 8.5 Hypothesis Tests 8.6 cars Example 8.7 Confidence Interval for Mean Response 8.8 Prediction Interval for New Observations 8.9 Confidence and Prediction Bands 8.10 Significance of Regression, F-Test 8.11 R Markdown", " Chapter 8 Inference for Simple Linear Regression “There are three types of lies: lies, damn lies, and statistics.” — Benjamin Disraeli After reading this chapter you will be able to: Understand the distributions of regression estimates. Create interval estimates for regression parameters, mean response, and predictions. Test for significance of regression. Last chapter we defined the simple linear regression model, \\[ Y_i = \\beta_0 + \\beta_1 x_i + \\epsilon_i \\] where \\(\\epsilon_i \\sim N(0, \\sigma^2)\\). We then used observations \\((x_i, y_i)\\), for \\(i = 1, 2, \\ldots n\\), to find values of \\(\\beta_0\\) and \\(\\beta_1\\) which minimized \\[ f(\\beta_0, \\beta_1) = \\sum_{i = 1}^{n}(y_i - (\\beta_0 + \\beta_1 x_i))^2. \\] We called these values \\(\\hat{\\beta}_0\\) and \\(\\hat{\\beta}_1\\), which we found to be \\[ \\begin{aligned} \\hat{\\beta}_1 &amp;= \\frac{S_{xy}}{S_{xx}} = \\frac{\\sum_{i = 1}^{n}(x_i - \\bar{x})(y_i - \\bar{y})}{\\sum_{i = 1}^{n}(x_i - \\bar{x})^2}\\\\ \\hat{\\beta}_0 &amp;= \\bar{y} - \\hat{\\beta}_1 \\bar{x}. \\end{aligned} \\] We also estimated \\(\\sigma ^2\\) using \\(s_e^2\\). In other words, we found that \\(s_e\\) is an estimate of \\(\\sigma\\), where; \\[ s_e = \\text{RSE} = \\sqrt{\\frac{1}{n - 2}\\sum_{i = 1}^n e_i^2} \\] which we also called \\(\\text{RSE}\\), for “Residual Standard Error.” When applied to the cars data, we obtained the following results: stop_dist_model = lm(dist ~ speed, data = cars) summary(stop_dist_model) ## ## Call: ## lm(formula = dist ~ speed, data = cars) ## ## Residuals: ## Min 1Q Median 3Q Max ## -29.069 -9.525 -2.272 9.215 43.201 ## ## Coefficients: ## Estimate Std. Error t value Pr(&gt;|t|) ## (Intercept) -17.5791 6.7584 -2.601 0.0123 * ## speed 3.9324 0.4155 9.464 1.49e-12 *** ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 ## ## Residual standard error: 15.38 on 48 degrees of freedom ## Multiple R-squared: 0.6511, Adjusted R-squared: 0.6438 ## F-statistic: 89.57 on 1 and 48 DF, p-value: 1.49e-12 Last chapter, we only discussed the Estimate, Residual standard error, and Multiple R-squared values. In this chapter, we will discuss all of the information under Coefficients as well as F-statistic. plot(dist ~ speed, data = cars, xlab = &quot;Speed (in Miles Per Hour)&quot;, ylab = &quot;Stopping Distance (in Feet)&quot;, main = &quot;Stopping Distance vs Speed&quot;, pch = 20, cex = 2, col = &quot;grey&quot;) abline(stop_dist_model, lwd = 5, col = &quot;darkorange&quot;) To get started, we’ll note that there is another equivalent expression for \\(S_{xy}\\) which we did not see last chapter, \\[ S_{xy}= \\sum_{i = 1}^{n}(x_i - \\bar{x})(y_i - \\bar{y}) = \\sum_{i = 1}^{n}(x_i - \\bar{x}) y_i. \\] This may be a surprising equivalence. (Maybe try to prove it.) However, it will be useful for illustrating concepts in this chapter. Note that \\(\\hat{\\beta}_1\\) is a sample statistic when calculated with observed data as written above, as is \\(\\hat{\\beta}_0\\). However, in this chapter it will often be convenient to use both \\(\\hat{\\beta}_1\\) and \\(\\hat{\\beta}_0\\) as random variables, that is, we have not yet observed the values for each \\(Y_i\\). When this is the case, we will use a slightly different notation, substituting in capital \\(Y_i\\) for lower case \\(y_i\\). \\[ \\begin{aligned} \\hat{\\beta}_1 &amp;= \\frac{\\sum_{i = 1}^{n}(x_i - \\bar{x}) Y_i}{\\sum_{i = 1}^{n}(x_i - \\bar{x})^2} \\\\ \\hat{\\beta}_0 &amp;= \\bar{Y} - \\hat{\\beta}_1 \\bar{x} \\end{aligned} \\] Last chapter we argued that these estimates of unknown model parameters \\(\\beta_0\\) and \\(\\beta_1\\) were good because we obtained them by minimizing errors. We will now discuss the Gauss–Markov theorem, which takes this idea further, showing that these estimates are actually the “best” estimates, from a certain point of view. 8.1 Gauss–Markov Theorem The Gauss–Markov theorem tells us that when estimating the parameters of the simple linear regression model \\(\\beta_0\\) and \\(\\beta_1\\), the \\(\\hat{\\beta}_0\\) and \\(\\hat{\\beta}_1\\) which we derived are the best linear unbiased estimates, or BLUE for short. (The actual conditions for the Gauss–Markov theorem are more relaxed than the SLR model.) We will now discuss linear, unbiased, and best as they relate to these estimates. Linear Recall, in the SLR setup that the \\(x_i\\) values are considered fixed and known quantities. Then a linear estimate is one which can be written as a linear combination of the \\(Y_i\\). In the case of \\(\\hat{\\beta}_1\\) we see \\[ \\hat{\\beta}_1 = \\frac{\\sum_{i = 1}^{n}(x_i - \\bar{x}) Y_i}{\\sum_{i = 1}^{n}(x_i - \\bar{x})^2} = \\sum_{i = 1}^n k_i Y_i = k_1 Y_1 + k_2 Y_2 + \\cdots k_n Y_n \\] where \\(k_i = \\displaystyle\\frac{(x_i - \\bar{x})}{\\sum_{i = 1}^{n}(x_i - \\bar{x})^2}\\). In a similar fashion, we could show that \\(\\hat{\\beta}_0\\) can be written as a linear combination of the \\(Y_i\\). Thus both \\(\\hat{\\beta}_0\\) and \\(\\hat{\\beta}_1\\) are linear estimators. Unbiased Now that we know our estimates are linear, how good are these estimates? One measure of the “goodness” of an estimate is its bias. Specifically, we prefer estimates that are unbiased, meaning their expected value is the parameter being estimated. In the case of the regression estimates, we have, \\[ \\begin{aligned} \\text{E}[\\hat{\\beta}_0] &amp;= \\beta_0 \\\\ \\text{E}[\\hat{\\beta}_1] &amp;= \\beta_1. \\end{aligned} \\] This tells us that, when the conditions of the SLR model are met, on average our estimates will be correct. However, as we saw last chapter when simulating from the SLR model, that does not mean that each individual estimate will be correct. Only that, if we repeated the process an infinite number of times, on average the estimate would be correct. Best Now, if we restrict ourselves to both linear and unbiased estimates, how do we define the best estimate? The estimate with the minimum variance. First note that it is very easy to create an estimate for \\(\\beta_1\\) that has very low variance, but is not unbiased. For example, define: \\[ \\hat{\\theta}_{BAD} = 5. \\] Then, since \\(\\hat{\\theta}_{BAD}\\) is a constant value, \\[ \\text{Var}[\\hat{\\theta}_{BAD}] = 0. \\] However since, \\[ \\text{E}[\\hat{\\theta}_{BAD}] = 5 \\] we say that \\(\\hat{\\theta}_{BAD}\\) is a biased estimator unless \\(\\beta_1 = 5\\), which we would not know ahead of time. For this reason, it is a terrible estimate (unless by chance \\(\\beta_1 = 5\\)) even though it has the smallest possible variance. This is part of the reason we restrict ourselves to unbiased estimates. What good is an estimate, if it estimates the wrong quantity? So now, the natural question is, what are the variances of \\(\\hat{\\beta}_0\\) and \\(\\hat{\\beta}_1\\)? They are, \\[ \\begin{aligned} \\text{Var}[\\hat{\\beta}_0] &amp;= \\sigma^2 \\left(\\frac{1}{n} + \\frac{\\bar{x}^2}{S_{xx}}\\right) \\\\ \\text{Var}[\\hat{\\beta}_1] &amp;= \\frac{\\sigma^2}{S_{xx}}. \\end{aligned} \\] These quantify the variability of the estimates due to random chance during sampling. Are these “the best”? Are these variances as small as we can possibly get? You’ll just have to take our word for it that they are because showing that this is true is beyond the scope of this course. 8.2 Sampling Distributions Now that we have “redefined” the estimates for \\(\\hat{\\beta}_0\\) and \\(\\hat{\\beta}_1\\) as random variables, we can discuss their sampling distribution, which is the distribution when a statistic is considered a random variable. Since both \\(\\hat{\\beta}_0\\) and \\(\\hat{\\beta}_1\\) are a linear combination of the \\(Y_i\\) and each \\(Y_i\\) is normally distributed, then both \\(\\hat{\\beta}_0\\) and \\(\\hat{\\beta}_1\\) also follow a normal distribution. Then, putting all of the above together, we arrive at the distributions of \\(\\hat{\\beta}_0\\) and \\(\\hat{\\beta}_1\\). For \\(\\hat{\\beta}_1\\) we say, \\[ \\hat{\\beta}_1 = \\frac{S_{xy}}{S_{xx}} = \\frac{\\sum_{i = 1}^{n}(x_i - \\bar{x}) Y_i}{\\sum_{i = 1}^{n}(x_i - \\bar{x})^2} \\sim N\\left( \\beta_1, \\ \\frac{\\sigma^2}{\\sum_{i = 1}^{n}(x_i - \\bar{x})^2} \\right). \\] Or more succinctly, \\[ \\hat{\\beta}_1 \\sim N\\left( \\beta_1, \\frac{\\sigma^2}{S_{xx}} \\right). \\] And for \\(\\hat{\\beta}_0\\), \\[ \\hat{\\beta}_0 = \\bar{Y} - \\hat{\\beta}_1 \\bar{x} \\sim N\\left( \\beta_0, \\ \\frac{\\sigma^2 \\sum_{i = 1}^{n}x_i^2}{n \\sum_{i = 1}^{n}(x_i - \\bar{x})^2} \\right). \\] Or more succinctly, \\[ \\hat{\\beta}_0 \\sim N\\left( \\beta_0, \\sigma^2 \\left(\\frac{1}{n} + \\frac{\\bar{x}^2}{S_{xx}}\\right) \\right) \\] At this point we have neglected to prove a number of these results. Instead of working through the tedious derivations of these sampling distributions, we will instead justify these results to ourselves using simulation. A note to current readers: These derivations and proofs may be added to an appendix at a later time. You can also find these results in nearly any standard linear regression textbook. At UIUC, these results will likely be presented in both STAT 424 and STAT 425. However, since you will not be asked to perform derivations of this type in this course, they are for now omitted. 8.2.1 Simulating Sampling Distributions To verify the above results, we will simulate samples of size \\(n = 100\\) from the model \\[ Y_i = \\beta_0 + \\beta_1 x_i + \\epsilon_i \\] where \\(\\epsilon_i \\sim N(0, \\sigma^2).\\) In this case, the parameters are known to be: \\(\\beta_0 = 3\\) \\(\\beta_1 = 6\\) \\(\\sigma^2 = 4\\) Then, based on the above, we should find that \\[ \\hat{\\beta}_1 \\sim N\\left( \\beta_1, \\frac{\\sigma^2}{S_{xx}} \\right) \\] and \\[ \\hat{\\beta}_0 \\sim N\\left( \\beta_0, \\sigma^2 \\left(\\frac{1}{n} + \\frac{\\bar{x}^2}{S_{xx}}\\right) \\right). \\] First we need to decide ahead of time what our \\(x\\) values will be for this simulation, since the \\(x\\) values in SLR are also considered known quantities. The choice of \\(x\\) values is arbitrary. Here we also set a seed for randomization, and calculate \\(S_{xx}\\) which we will need going forward. set.seed(42) sample_size = 100 # this is n x = seq(-1, 1, length = sample_size) Sxx = sum((x - mean(x)) ^ 2) We also fix our parameter values. beta_0 = 3 beta_1 = 6 sigma = 2 With this information, we know the sampling distributions should be: (var_beta_1_hat = sigma ^ 2 / Sxx) ## [1] 0.1176238 (var_beta_0_hat = sigma ^ 2 * (1 / sample_size + mean(x) ^ 2 / Sxx)) ## [1] 0.04 \\[ \\hat{\\beta}_1 \\sim N( 6, 0.1176238) \\] and \\[ \\hat{\\beta}_0 \\sim N( 3, 0.04). \\] That is, \\[ \\begin{aligned} \\text{E}[\\hat{\\beta}_1] &amp;= 6 \\\\ \\text{Var}[\\hat{\\beta}_1] &amp;= 0.1176238 \\end{aligned} \\] and \\[ \\begin{aligned} \\text{E}[\\hat{\\beta}_0] &amp;= 3 \\\\ \\text{Var}[\\hat{\\beta}_0] &amp;= 0.04. \\end{aligned} \\] We now simulate data from this model 10,000 times. Note this may not be the most R way of doing the simulation. We perform the simulation in this manner in an attempt at clarity. For example, we could have used the sim_slr() function from the previous chapter. We also simply store variables in the global environment instead of creating a data frame for each new simulated dataset. num_samples = 10000 beta_0_hats = rep(0, num_samples) beta_1_hats = rep(0, num_samples) for (i in 1:num_samples) { eps = rnorm(sample_size, mean = 0, sd = sigma) y = beta_0 + beta_1 * x + eps sim_model = lm(y ~ x) beta_0_hats[i] = coef(sim_model)[1] beta_1_hats[i] = coef(sim_model)[2] } Each time we simulated the data, we obtained values of the estimated coefficients. The variables beta_0_hats and beta_1_hats now store 10,000 simulated values of \\(\\hat{\\beta}_0\\) and \\(\\hat{\\beta}_1\\) respectively. We first verify the distribution of \\(\\hat{\\beta}_1\\). mean(beta_1_hats) # empirical mean ## [1] 6.001998 beta_1 # true mean ## [1] 6 var(beta_1_hats) # empirical variance ## [1] 0.11899 var_beta_1_hat # true variance ## [1] 0.1176238 We see that the empirical and true means and variances are very similar. We also verify that the empirical distribution is normal. To do so, we plot a histogram of the beta_1_hats, and add the curve for the true distribution of \\(\\hat{\\beta}_1\\). We use prob = TRUE to put the histogram on the same scale as the normal curve. # note need to use prob = TRUE hist(beta_1_hats, prob = TRUE, breaks = 20, xlab = expression(hat(beta)[1]), main = &quot;&quot;, border = &quot;dodgerblue&quot;) curve(dnorm(x, mean = beta_1, sd = sqrt(var_beta_1_hat)), col = &quot;darkorange&quot;, add = TRUE, lwd = 3) We then repeat the process for \\(\\hat{\\beta}_0\\). mean(beta_0_hats) # empirical mean ## [1] 3.001147 beta_0 # true mean ## [1] 3 var(beta_0_hats) # empirical variance ## [1] 0.04017924 var_beta_0_hat # true variance ## [1] 0.04 hist(beta_0_hats, prob = TRUE, breaks = 25, xlab = expression(hat(beta)[0]), main = &quot;&quot;, border = &quot;dodgerblue&quot;) curve(dnorm(x, mean = beta_0, sd = sqrt(var_beta_0_hat)), col = &quot;darkorange&quot;, add = TRUE, lwd = 3) In this simulation study, we have only simulated a finite number of samples. To truly verify the distributional results, we would need to observe an infinite number of samples. However, the following plot should make it clear that if we continued simulating, the empirical results would get closer and closer to what we should expect. par(mar = c(5, 5, 1, 1)) # adjusted plot margins, otherwise the &quot;hat&quot; does not display plot(cumsum(beta_1_hats) / (1:length(beta_1_hats)), type = &quot;l&quot;, ylim = c(5.95, 6.05), xlab = &quot;Number of Simulations&quot;, ylab = expression(&quot;Empirical Mean of &quot; ~ hat(beta)[1]), col = &quot;dodgerblue&quot;) abline(h = 6, col = &quot;darkorange&quot;, lwd = 2) par(mar = c(5, 5, 1, 1)) # adjusted plot margins, otherwise the &quot;hat&quot; does not display plot(cumsum(beta_0_hats) / (1:length(beta_0_hats)), type = &quot;l&quot;, ylim = c(2.95, 3.05), xlab = &quot;Number of Simulations&quot;, ylab = expression(&quot;Empirical Mean of &quot; ~ hat(beta)[0]), col = &quot;dodgerblue&quot;) abline(h = 3, col = &quot;darkorange&quot;, lwd = 2) 8.3 Standard Errors So now we believe the two distributional results, \\[ \\begin{aligned} \\hat{\\beta}_0 &amp;\\sim N\\left( \\beta_0, \\sigma^2 \\left(\\frac{1}{n} + \\frac{\\bar{x}^2}{S_{xx}}\\right) \\right) \\\\ \\hat{\\beta}_1 &amp;\\sim N\\left( \\beta_1, \\frac{\\sigma^2}{S_{xx}} \\right). \\end{aligned} \\] Then by standardizing these results we find that \\[ \\frac{\\hat{\\beta}_0 - \\beta_0}{\\text{SD}[\\hat{\\beta}_0]} \\sim N(0, 1) \\] and \\[ \\frac{\\hat{\\beta}_1 - \\beta_1}{\\text{SD}[\\hat{\\beta}_1]} \\sim N(0, 1) \\] where \\[ \\text{SD}[\\hat{\\beta}_0] = \\sigma\\sqrt{\\frac{1}{n} + \\frac{\\bar{x}^2}{S_{xx}}} \\] and \\[ \\text{SD}[\\hat{\\beta}_1] = \\frac{\\sigma}{\\sqrt{S_{xx}}}. \\] Since we don’t know \\(\\sigma\\) in practice, we will have to estimate it using \\(s_e\\), which we plug into our existing expression for the standard deviations of our estimates. These two new expressions are called standard errors which are the estimated standard deviations of the sampling distributions. \\[ \\text{SE}[\\hat{\\beta}_0] = s_e\\sqrt{\\frac{1}{n} + \\frac{\\bar{x}^2}{S_{xx}}} \\] \\[ \\text{SE}[\\hat{\\beta}_1] = \\frac{s_e}{\\sqrt{S_{xx}}} \\] Now if we divide by the standard error, instead of the standard deviation, we obtain the following results which will allow us to make confidence intervals and perform hypothesis testing. \\[ \\frac{\\hat{\\beta}_0 - \\beta_0}{\\text{SE}[\\hat{\\beta}_0]} \\sim t_{n-2} \\] \\[ \\frac{\\hat{\\beta}_1 - \\beta_1}{\\text{SE}[\\hat{\\beta}_1]} \\sim t_{n-2} \\] To see this, first note that, \\[ \\frac{\\text{RSS}}{\\sigma^2} = \\frac{(n-2)s_e^2}{\\sigma^2} \\sim \\chi_{n-2}^2. \\] Also recall that a random variable \\(T\\) defined as, \\[ T = \\frac{Z}{\\sqrt{\\frac{\\chi_{d}^2}{d}}} \\] follows a \\(t\\) distribution with \\(d\\) degrees of freedom, where \\(\\chi_{d}^2\\) is a \\(\\chi^2\\) random variable with \\(d\\) degrees of freedom. We write, \\[ T \\sim t_d \\] to say that the random variable \\(T\\) follows a \\(t\\) distribution with \\(d\\) degrees of freedom. Then we use the classic trick of “multiply by 1” and some rearranging to arrive at \\[ \\begin{aligned} \\frac{\\hat{\\beta}_1 - \\beta_1}{\\text{SE}[\\hat{\\beta}_1]} &amp;= \\frac{\\hat{\\beta}_1 - \\beta_1}{s_e / \\sqrt{S_{xx}}} \\\\ &amp;= \\frac{\\hat{\\beta}_1 - \\beta_1}{s_e / \\sqrt{S_{xx}}} \\cdot \\frac{\\sigma / \\sqrt{S_{xx}}}{\\sigma / \\sqrt{S_{xx}}} \\\\ &amp;= \\frac{\\hat{\\beta}_1 - \\beta_1}{\\sigma / \\sqrt{S_{xx}}} \\cdot \\frac{\\sigma / \\sqrt{S_{xx}}}{s_e / \\sqrt{S_{xx}}} \\\\ &amp;= \\frac{\\hat{\\beta}_1 - \\beta_1}{\\sigma / \\sqrt{S_{xx}}} \\bigg/ \\sqrt{\\frac{s_e^2}{\\sigma^2}} \\\\ &amp;= \\frac{\\hat{\\beta}_1 - \\beta_1}{\\text{SD}[\\hat{\\beta}_1]} \\bigg/ \\sqrt{\\frac{\\frac{(n - 2)s_e^2}{\\sigma^2}}{n - 2}} \\sim \\frac{Z}{\\sqrt{\\frac{\\chi_{n-2}^2}{n-2}}} \\sim t_{n-2} \\end{aligned} \\] where \\(Z \\sim N(0,1)\\). Recall that a \\(t\\) distribution is similar to a standard normal, but with heavier tails. As the degrees of freedom increases, the \\(t\\) distribution becomes more and more like a standard normal. Below we plot a standard normal distribution as well as two examples of a \\(t\\) distribution with different degrees of freedom. Notice how the \\(t\\) distribution with the larger degrees of freedom is more similar to the standard normal curve. # define grid of x values x = seq(-4, 4, length = 100) # plot curve for standard normal plot(x, dnorm(x), type = &quot;l&quot;, lty = 1, lwd = 2, xlab = &quot;x&quot;, ylab = &quot;Density&quot;, main = &quot;Normal vs t Distributions&quot;) # add curves for t distributions lines(x, dt(x, df = 1), lty = 3, lwd = 2, col = &quot;darkorange&quot;) lines(x, dt(x, df = 10), lty = 2, lwd = 2, col = &quot;dodgerblue&quot;) # add legend legend(&quot;topright&quot;, title = &quot;Distributions&quot;, legend = c(&quot;t, df = 1&quot;, &quot;t, df = 10&quot;, &quot;Standard Normal&quot;), lwd = 2, lty = c(3, 2, 1), col = c(&quot;darkorange&quot;, &quot;dodgerblue&quot;, &quot;black&quot;)) 8.4 Confidence Intervals for Slope and Intercept Recall that confidence intervals for means often take the form: \\[ \\text{EST} \\pm \\text{CRIT} \\cdot \\text{SE} \\] or \\[ \\text{EST} \\pm \\text{MARGIN} \\] where \\(\\text{EST}\\) is an estimate for the parameter of interest, \\(\\text{SE}\\) is the standard error of the estimate, and \\(\\text{MARGIN} = \\text{CRIT} \\cdot \\text{SE}\\). Then, for \\(\\beta_0\\) and \\(\\beta_1\\) we can create confidence intervals using \\[ \\hat{\\beta}_0 \\pm t_{\\alpha/2, n - 2} \\cdot \\text{SE}[\\hat{\\beta}_0] \\quad \\quad \\quad \\hat{\\beta}_0 \\pm t_{\\alpha/2, n - 2} \\cdot s_e\\sqrt{\\frac{1}{n}+\\frac{\\bar{x}^2}{S_{xx}}} \\] and \\[ \\hat{\\beta}_1 \\pm t_{\\alpha/2, n - 2} \\cdot \\text{SE}[\\hat{\\beta}_1] \\quad \\quad \\quad \\hat{\\beta}_1 \\pm t_{\\alpha/2, n - 2} \\cdot \\frac{s_e}{\\sqrt{S_{xx}}} \\] where \\(t_{\\alpha/2, n - 2}\\) is the critical value such that \\(P(t_{n-2} &gt; t_{\\alpha/2, n - 2}) = \\alpha/2\\). 8.5 Hypothesis Tests “We may speak of this hypothesis as the ‘null hypothesis,’ and it should be noted that the null hypothesis is never proved or established, but is possibly disproved, in the course of experimentation.” — Ronald Aylmer Fisher Recall that a test statistic (\\(\\text{TS}\\)) for testing means often takes the form: \\[ \\text{TS} = \\frac{\\text{EST} - \\text{HYP}}{\\text{SE}} \\] where \\(\\text{EST}\\) is an estimate for the parameter of interest, \\(\\text{HYP}\\) is a hypothesized value of the parameter, and \\(\\text{SE}\\) is the standard error of the estimate. So, to test \\[ H_0: \\beta_0 = \\beta_{00} \\quad \\text{vs} \\quad H_1: \\beta_0 \\neq \\beta_{00} \\] we use the test statistic \\[ t = \\frac{\\hat{\\beta}_0 - \\beta_{00}}{\\text{SE}[\\hat{\\beta}_0]} = \\frac{\\hat{\\beta}_0-\\beta_{00}}{s_e\\sqrt{\\frac{1}{n} + \\frac{\\bar{x}^2}{S_{xx}}}} \\] which, under the null hypothesis, follows a \\(t\\) distribution with \\(n - 2\\) degrees of freedom. We use \\(\\beta_{00}\\) to denote the hypothesized value of \\(\\beta_0\\). Similarly, to test \\[ H_0: \\beta_1 = \\beta_{10} \\quad \\text{vs} \\quad H_1: \\beta_1 \\neq \\beta_{10} \\] we use the test statistic \\[ t = \\frac{\\hat{\\beta}_1-\\beta_{10}}{\\text{SE}[\\hat{\\beta}_1]} = \\frac{\\hat{\\beta}_1-\\beta_{10}}{s_e / \\sqrt{S_{xx}}} \\] which again, under the null hypothesis, follows a \\(t\\) distribution with \\(n - 2\\) degrees of freedom. We now use \\(\\beta_{10}\\) to denote the hypothesized value of \\(\\beta_1\\). 8.6 cars Example We now return to the cars example from last chapter to illustrate these concepts. We first fit the model using lm() then use summary() to view the results in greater detail. stop_dist_model = lm(dist ~ speed, data = cars) summary(stop_dist_model) ## ## Call: ## lm(formula = dist ~ speed, data = cars) ## ## Residuals: ## Min 1Q Median 3Q Max ## -29.069 -9.525 -2.272 9.215 43.201 ## ## Coefficients: ## Estimate Std. Error t value Pr(&gt;|t|) ## (Intercept) -17.5791 6.7584 -2.601 0.0123 * ## speed 3.9324 0.4155 9.464 1.49e-12 *** ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 ## ## Residual standard error: 15.38 on 48 degrees of freedom ## Multiple R-squared: 0.6511, Adjusted R-squared: 0.6438 ## F-statistic: 89.57 on 1 and 48 DF, p-value: 1.49e-12 8.6.1 Tests in R We will now discuss the results displayed called Coefficients. First recall that we can extract this information directly. names(summary(stop_dist_model)) ## [1] &quot;call&quot; &quot;terms&quot; &quot;residuals&quot; &quot;coefficients&quot; ## [5] &quot;aliased&quot; &quot;sigma&quot; &quot;df&quot; &quot;r.squared&quot; ## [9] &quot;adj.r.squared&quot; &quot;fstatistic&quot; &quot;cov.unscaled&quot; summary(stop_dist_model)$coefficients ## Estimate Std. Error t value Pr(&gt;|t|) ## (Intercept) -17.579095 6.7584402 -2.601058 1.231882e-02 ## speed 3.932409 0.4155128 9.463990 1.489836e-12 The names() function tells us what information is available, and then we use the $ operator and coefficients to extract the information we are interested in. Two values here should be immediately familiar. \\[ \\hat{\\beta}_0 = -17.5790949 \\] and \\[ \\hat{\\beta}_1 = 3.9324088 \\] which are our estimates for the model parameters \\(\\beta_0\\) and \\(\\beta_1\\). Let’s now focus on the second row of output, which is relevant to \\(\\beta_1\\). summary(stop_dist_model)$coefficients[2,] ## Estimate Std. Error t value Pr(&gt;|t|) ## 3.932409e+00 4.155128e-01 9.463990e+00 1.489836e-12 Again, the first value, Estimate is \\[ \\hat{\\beta}_1 = 3.9324088. \\] The second value, Std. Error, is the standard error of \\(\\hat{\\beta}_1\\), \\[ \\text{SE}[\\hat{\\beta}_1] = \\frac{s_e}{\\sqrt{S_{xx}}} = 0.4155128. \\] The third value, t value, is the value of the test statistic for testing \\(H_0: \\beta_1 = 0\\) vs \\(H_1: \\beta_1 \\neq 0\\), \\[ t = \\frac{\\hat{\\beta}_1-0}{\\text{SE}[\\hat{\\beta}_1]} = \\frac{\\hat{\\beta}_1-0}{s_e / \\sqrt{S_{xx}}} = 9.46399. \\] Lastly, Pr(&gt;|t|), gives us the p-value of that test. \\[ \\text{p-value} = 1.4898365\\times 10^{-12} \\] Note here, we are specifically testing whether or not \\(\\beta_1 = 0\\). The first row of output reports the same values, but for \\(\\beta_0\\). summary(stop_dist_model)$coefficients[1,] ## Estimate Std. Error t value Pr(&gt;|t|) ## -17.57909489 6.75844017 -2.60105800 0.01231882 In summary, the following code stores the information of summary(stop_dist_model)$coefficients in a new variable stop_dist_model_test_info, then extracts each element into a new variable which describes the information it contains. stop_dist_model_test_info = summary(stop_dist_model)$coefficients beta_0_hat = stop_dist_model_test_info[1, 1] # Estimate beta_0_hat_se = stop_dist_model_test_info[1, 2] # Std. Error beta_0_hat_t = stop_dist_model_test_info[1, 3] # t value beta_0_hat_pval = stop_dist_model_test_info[1, 4] # Pr(&gt;|t|) beta_1_hat = stop_dist_model_test_info[2, 1] # Estimate beta_1_hat_se = stop_dist_model_test_info[2, 2] # Std. Error beta_1_hat_t = stop_dist_model_test_info[2, 3] # t value beta_1_hat_pval = stop_dist_model_test_info[2, 4] # Pr(&gt;|t|) We can then verify some equivalent expressions: the \\(t\\) test statistic for \\(\\hat{\\beta}_1\\) and the two-sided p-value associated with that test statistic. (beta_1_hat - 0) / beta_1_hat_se ## [1] 9.46399 beta_1_hat_t ## [1] 9.46399 2 * pt(abs(beta_1_hat_t), df = length(resid(stop_dist_model)) - 2, lower.tail = FALSE) ## [1] 1.489836e-12 beta_1_hat_pval ## [1] 1.489836e-12 8.6.2 Significance of Regression, t-Test We pause to discuss the significance of regression test. First, note that based on the above distributional results, we could test \\(\\beta_0\\) and \\(\\beta_1\\) against any particular value, and perform both one and two-sided tests. However, one very specific test, \\[ H_0: \\beta_1 = 0 \\quad \\text{vs} \\quad H_1: \\beta_1 \\neq 0 \\] is used most often. Let’s think about this test in terms of the simple linear regression model, \\[ Y_i = \\beta_0 + \\beta_1 x_i + \\epsilon_i. \\] If we assume the null hypothesis is true, then \\(\\beta_1 = 0\\) and we have the model, \\[ Y_i = \\beta_0 + \\epsilon_i. \\] In this model, the response does not depend on the predictor. So then we could think of this test in the following way, Under \\(H_0\\) there is not a significant linear relationship between \\(x\\) and \\(y\\). Under \\(H_1\\) there is a significant linear relationship between \\(x\\) and \\(y\\). For the cars example, Under \\(H_0\\) there is not a significant linear relationship between speed and stopping distance. Under \\(H_1\\) there is a significant linear relationship between speed and stopping distance. Again, that test is seen in the output from summary(), \\[ \\text{p-value} = 1.4898365\\times 10^{-12}. \\] With this extremely low p-value, we would reject the null hypothesis at any reasonable \\(\\alpha\\) level, say for example \\(\\alpha = 0.01\\). So we say there is a significant linear relationship between speed and stopping distance. Notice that we emphasize linear. In this plot of simulated data, we see a clear relationship between \\(x\\) and \\(y\\), however it is not a linear relationship. If we fit a line to this data, it is very flat. The resulting test for \\(H_0: \\beta_1 = 0\\) vs \\(H_1: \\beta_1 \\neq 0\\) gives a large p-value, in this case \\(0.7564548\\), so we would fail to reject and say that there is no significant linear relationship between \\(x\\) and \\(y\\). We will see later how to fit a curve to this data using a “linear” model, but for now, realize that testing \\(H_0: \\beta_1 = 0\\) vs \\(H_1: \\beta_1 \\neq 0\\) can only detect straight line relationships. 8.6.3 Confidence Intervals in R Using R we can very easily obtain the confidence intervals for \\(\\beta_0\\) and \\(\\beta_1\\). confint(stop_dist_model, level = 0.99) ## 0.5 % 99.5 % ## (Intercept) -35.706610 0.5484205 ## speed 2.817919 5.0468988 This automatically calculates 99% confidence intervals for both \\(\\beta_0\\) and \\(\\beta_1\\), the first row for \\(\\beta_0\\), the second row for \\(\\beta_1\\). For the cars example when interpreting these intervals, we say, we are 99% confident that for an increase in speed of 1 mile per hour, the average increase in stopping distance is between 2.8179187 and 5.0468988 feet, which is the interval for \\(\\beta_1\\). Note that this 99% confidence interval does not contain the hypothesized value of 0. Since it does not contain 0, it is equivalent to rejecting the test of \\(H_0: \\beta_1 = 0\\) vs \\(H_1: \\beta_1 \\neq 0\\) at \\(\\alpha = 0.01\\), which we had seen previously. You should be somewhat suspicious of the confidence interval for \\(\\beta_0\\), as it covers negative values, which correspond to negative stopping distances. Technically the interpretation would be that we are 99% confident that the average stopping distance of a car traveling 0 miles per hour is between -35.7066103 and 0.5484205 feet, but we don’t really believe that, since we are actually certain that it would be non-negative. Note, we can extract specific values from this output a number of ways. This code is not run, and instead, you should check how it relates to the output of the code above. confint(stop_dist_model, level = 0.99)[1,] confint(stop_dist_model, level = 0.99)[1, 1] confint(stop_dist_model, level = 0.99)[1, 2] confint(stop_dist_model, parm = &quot;(Intercept)&quot;, level = 0.99) confint(stop_dist_model, level = 0.99)[2,] confint(stop_dist_model, level = 0.99)[2, 1] confint(stop_dist_model, level = 0.99)[2, 2] confint(stop_dist_model, parm = &quot;speed&quot;, level = 0.99) We can also verify that calculations that R is performing for the \\(\\beta_1\\) interval. # store estimate beta_1_hat = coef(stop_dist_model)[2] # store standard error beta_1_hat_se = summary(stop_dist_model)$coefficients[2, 2] # calculate critical value for two-sided 99% CI crit = qt(0.995, df = length(resid(stop_dist_model)) - 2) # est - margin, est + margin c(beta_1_hat - crit * beta_1_hat_se, beta_1_hat + crit * beta_1_hat_se) ## speed speed ## 2.817919 5.046899 8.7 Confidence Interval for Mean Response In addition to confidence intervals for \\(\\beta_0\\) and \\(\\beta_1\\), there are two other common interval estimates used with regression. The first is called a confidence interval for the mean response. Often, we would like an interval estimate for the mean, \\(E[Y \\mid X = x]\\) for a particular value of \\(x\\). In this situation we use \\(\\hat{y}(x)\\) as our estimate of \\(E[Y \\mid X = x]\\). We modify our notation slightly to make it clear that the predicted value is a function of the \\(x\\) value. \\[ \\hat{y}(x) = \\hat{\\beta}_0 + \\hat{\\beta}_1 x \\] Recall that, \\[ \\text{E}[Y \\mid X = x] = \\beta_0 + \\beta_1 x. \\] Thus, \\(\\hat{y}(x)\\) is a good estimate since it is unbiased: \\[ \\text{E}[\\hat{y}(x)] = \\beta_0 + \\beta_1 x. \\] We could then derive, \\[ \\text{Var}[\\hat{y}(x)] = \\sigma^2 \\left(\\frac{1}{n}+\\frac{(x-\\bar{x})^2}{S_{xx}}\\right). \\] Like the other estimates we have seen, \\(\\hat{y}(x)\\) also follows a normal distribution. Since \\(\\hat{\\beta}_0\\) and \\(\\hat{\\beta}_1\\) are linear combinations of normal random variables, \\(\\hat{y}(x)\\) is as well. \\[ \\hat{y}(x) \\sim N \\left(\\beta_0 + \\beta_1 x, \\sigma^2 \\left(\\frac{1}{n}+\\frac{(x-\\bar{x})^2}{S_{xx}}\\right) \\right) \\] And lastly, since we need to estimate this variance, we arrive at the standard error of our estimate, \\[ \\text{SE}[\\hat{y}(x)] = s_e \\sqrt{\\frac{1}{n}+\\frac{(x-\\bar{x})^2}{S_{xx}}}. \\] We can then use this to find the confidence interval for the mean response, \\[ \\hat{y}(x) \\pm t_{\\alpha/2, n - 2} \\cdot s_e\\sqrt{\\frac{1}{n}+\\frac{(x-\\bar{x})^2}{S_{xx}}} \\] To find confidence intervals for the mean response using R, we use the predict() function. We give the function our fitted model as well as new data, stored as a data frame. (This is important, so that R knows the name of the predictor variable.) Here, we are finding the confidence interval for the mean stopping distance when a car is travelling 5 miles per hour and when a car is travelling 21 miles per hour. new_speeds = data.frame(speed = c(5, 21)) predict(stop_dist_model, newdata = new_speeds, interval = c(&quot;confidence&quot;), level = 0.99) ## fit lwr upr ## 1 2.082949 -10.89309 15.05898 ## 2 65.001489 56.45836 73.54462 8.8 Prediction Interval for New Observations Sometimes we would like an interval estimate for a new observation, \\(Y\\), for a particular value of \\(x\\). This is very similar to an interval for the mean response, \\(\\text{E}[Y \\mid X = x]\\), but different in one very important way. Our best guess for a new observation is still \\(\\hat{y}(x)\\). The estimated mean is still the best prediction we can make. The difference is in the amount of variability. We know that observations will vary about the true regression line according to a \\(N(0, \\sigma^2)\\) distribution. Because of this we add an extra factor of \\(\\sigma^2\\) to our estimate’s variability in order to account for the variability of observations about the regression line. \\[ \\begin{aligned} \\text{Var}[\\hat{y}(x) + \\epsilon] &amp;= \\text{Var}[\\hat{y}(x)] + \\text{Var}[\\epsilon] \\\\[2ex] &amp;= \\sigma^2 \\left(\\frac{1}{n}+\\frac{(x-\\bar{x})^2}{S_{xx}}\\right) + \\sigma^2 \\\\[2ex] &amp;= \\sigma^2 \\left(1 + \\frac{1}{n}+\\frac{(x-\\bar{x})^2}{S_{xx}}\\right) \\end{aligned} \\] \\[ \\hat{y}(x) + \\epsilon \\sim N \\left(\\beta_0 + \\beta_1 x, \\ \\sigma^2 \\left(1 + \\frac{1}{n}+\\frac{(x-\\bar{x})^2}{S_{xx}}\\right) \\right) \\] \\[ \\text{SE}[\\hat{y}(x) + \\epsilon] = s_e \\sqrt{1 + \\frac{1}{n}+\\frac{(x-\\bar{x})^2}{S_{xx}}} \\] We can then find a prediction interval using, \\[ \\hat{y}(x) \\pm t_{\\alpha/2, n - 2} \\cdot s_e\\sqrt{1 + \\frac{1}{n}+\\frac{(x-\\bar{x})^2}{S_{xx}}}. \\] To calculate this for a set of points in R notice there is only a minor change in syntax from finding a confidence interval for the mean response. predict(stop_dist_model, newdata = new_speeds, interval = c(&quot;prediction&quot;), level = 0.99) ## fit lwr upr ## 1 2.082949 -41.16099 45.32689 ## 2 65.001489 22.87494 107.12803 Also notice that these two intervals are wider than the corresponding confidence intervals for the mean response. 8.9 Confidence and Prediction Bands Often we will like to plot both confidence intervals for the mean response and prediction intervals for all possible values of \\(x\\). We calls these confidence and prediction bands. speed_grid = seq(min(cars$speed), max(cars$speed), by = 0.01) dist_ci_band = predict(stop_dist_model, newdata = data.frame(speed = speed_grid), interval = &quot;confidence&quot;, level = 0.99) dist_pi_band = predict(stop_dist_model, newdata = data.frame(speed = speed_grid), interval = &quot;prediction&quot;, level = 0.99) plot(dist ~ speed, data = cars, xlab = &quot;Speed (in Miles Per Hour)&quot;, ylab = &quot;Stopping Distance (in Feet)&quot;, main = &quot;Stopping Distance vs Speed&quot;, pch = 20, cex = 2, col = &quot;grey&quot;, ylim = c(min(dist_pi_band), max(dist_pi_band))) abline(stop_dist_model, lwd = 5, col = &quot;darkorange&quot;) lines(speed_grid, dist_ci_band[,&quot;lwr&quot;], col = &quot;dodgerblue&quot;, lwd = 3, lty = 2) lines(speed_grid, dist_ci_band[,&quot;upr&quot;], col = &quot;dodgerblue&quot;, lwd = 3, lty = 2) lines(speed_grid, dist_pi_band[,&quot;lwr&quot;], col = &quot;dodgerblue&quot;, lwd = 3, lty = 3) lines(speed_grid, dist_pi_band[,&quot;upr&quot;], col = &quot;dodgerblue&quot;, lwd = 3, lty = 3) points(mean(cars$speed), mean(cars$dist), pch = &quot;+&quot;, cex = 3) Some things to notice: We use the ylim argument to stretch the \\(y\\)-axis of the plot, since the bands extend further than the points. We add a point at the point \\((\\bar{x}, \\bar{y})\\). This is a point that the regression line will always pass through. (Think about why.) This is the point where both the confidence and prediction bands are the narrowest. Look at the standard errors of both to understand why. The prediction bands (dotted blue) are less curved than the confidence bands (dashed blue). This is a result of the extra factor of \\(\\sigma^2\\) added to the variance at any value of \\(x\\). 8.10 Significance of Regression, F-Test In the case of simple linear regression, the \\(t\\) test for the significance of the regression is equivalent to another test, the \\(F\\) test for the significance of the regression. This equivalence will only be true for simple linear regression, and in the next chapter we will only use the \\(F\\) test for the significance of the regression. Recall from last chapter the decomposition of variance we saw before calculating \\(R^2\\), \\[ \\sum_{i=1}^{n}(y_i - \\bar{y})^2 = \\sum_{i=1}^{n}(y_i - \\hat{y}_i)^2 + \\sum_{i=1}^{n}(\\hat{y}_i - \\bar{y})^2, \\] or, in short, \\[ \\text{SST} = \\text{SSE} + \\text{SSReg}. \\] To develop the \\(F\\) test, we will arrange this information in an ANOVA table, Source Sum of Squares Degrees of Freedom Mean Square \\(F\\) Regression \\(\\sum_{i=1}^{n}(\\hat{y}_i - \\bar{y})^2\\) \\(1\\) \\(\\text{SSReg} / 1\\) \\(\\text{MSReg} / \\text{MSE}\\) Error \\(\\sum_{i=1}^{n}(y_i - \\hat{y}_i)^2\\) \\(n - 2\\) \\(\\text{SSE} / (n - 2)\\) Total \\(\\sum_{i=1}^{n}(y_i - \\bar{y})^2\\) \\(n - 1\\) ANOVA, or Analysis of Variance will be a concept we return to often in this course. For now, we will focus on the results of the table, which is the \\(F\\) statistic, \\[ F = \\frac{\\sum_{i=1}^{n}(\\hat{y}_i - \\bar{y})^2 / 1}{\\sum_{i=1}^{n}(y_i - \\hat{y}_i)^2 / (n - 2)} \\sim F_{1, n - 2} \\] which follows an \\(F\\) distribution with degrees of freedom \\(1\\) and \\(n - 2\\) under the null hypothesis. An \\(F\\) distribution is a continuous distribution which takes only positive values and has two parameters, which are the two degrees of freedom. Recall, in the significance of the regression test, \\(Y\\) does not depend on \\(x\\) in the null hypothesis. \\[ H_0: \\beta_1 = 0 \\quad \\quad Y_i = \\beta_0 + \\epsilon_i \\] While in the alternative hypothesis \\(Y\\) may depend on \\(x\\). \\[ H_1: \\beta_1 \\neq 0 \\quad \\quad Y_i = \\beta_0 + \\beta_1 x_i + \\epsilon_i \\] We can use the \\(F\\) statistic to perform this test. \\[ F = \\frac{\\sum_{i=1}^{n}(\\hat{y}_i - \\bar{y})^2 / 1}{\\sum_{i=1}^{n}(y_i - \\hat{y}_i)^2 / (n - 2)} \\] In particular, we will reject the null when the \\(F\\) statistic is large, that is, when there is a low probability that the observations could have come from the null model by chance. We will let R calculate the p-value for us. To perform the \\(F\\) test in R you can look at the last row of the output from summary() called F-statistic which gives the value of the test statistic, the relevant degrees of freedom, as well as the p-value of the test. summary(stop_dist_model) ## ## Call: ## lm(formula = dist ~ speed, data = cars) ## ## Residuals: ## Min 1Q Median 3Q Max ## -29.069 -9.525 -2.272 9.215 43.201 ## ## Coefficients: ## Estimate Std. Error t value Pr(&gt;|t|) ## (Intercept) -17.5791 6.7584 -2.601 0.0123 * ## speed 3.9324 0.4155 9.464 1.49e-12 *** ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 ## ## Residual standard error: 15.38 on 48 degrees of freedom ## Multiple R-squared: 0.6511, Adjusted R-squared: 0.6438 ## F-statistic: 89.57 on 1 and 48 DF, p-value: 1.49e-12 Additionally, you can use the anova() function to display the information in an ANOVA table. anova(stop_dist_model) ## Analysis of Variance Table ## ## Response: dist ## Df Sum Sq Mean Sq F value Pr(&gt;F) ## speed 1 21186 21185.5 89.567 1.49e-12 *** ## Residuals 48 11354 236.5 ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 This also gives a p-value for the test. You should notice that the p-value from the \\(t\\) test was the same. You might also notice that the value of the test statistic for the \\(t\\) test, \\(9.46399\\), can be squared to obtain the value of the \\(F\\) statistic, \\(89.5671065\\). Note that there is another equivalent way to do this in R, which we will return to often to compare two models. anova(lm(dist ~ 1, data = cars), lm(dist ~ speed, data = cars)) ## Analysis of Variance Table ## ## Model 1: dist ~ 1 ## Model 2: dist ~ speed ## Res.Df RSS Df Sum of Sq F Pr(&gt;F) ## 1 49 32539 ## 2 48 11354 1 21186 89.567 1.49e-12 *** ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 The model statement lm(dist ~ 1, data = cars) applies the model \\(Y_i = \\beta_0 + \\epsilon_i\\) to the cars data. Note that \\(\\hat{y} = \\bar{y}\\) when \\(Y_i = \\beta_0 + \\epsilon_i\\). The model statement lm(dist ~ speed, data = cars) applies the model \\(Y_i = \\beta_0 + \\beta_1 x_i + \\epsilon_i\\). We can then think of this usage of anova() as directly comparing the two models. (Notice we get the same p-value again.) 8.11 R Markdown The R Markdown file for this chapter can be found here: slr-inf.Rmd The file was created using R version 4.3.2. "],["multiple-linear-regression.html", "Chapter 9 Multiple Linear Regression 9.1 Matrix Approach to Regression 9.2 Sampling Distribution 9.3 Significance of Regression 9.4 Nested Models 9.5 Simulation 9.6 R Markdown", " Chapter 9 Multiple Linear Regression “Life is really simple, but we insist on making it complicated.” — Confucius After reading this chapter you will be able to: Construct and interpret linear regression models with more than one predictor. Understand how regression models are derived using matrices. Create interval estimates and perform hypothesis tests for multiple regression parameters. Formulate and interpret interval estimates for the mean response under various conditions. Compare nested models using an ANOVA F-Test. The last two chapters we saw how to fit a model that assumed a linear relationship between a response variable and a single predictor variable. Specifically, we defined the simple linear regression model, \\[ Y_i = \\beta_0 + \\beta_1 x_i + \\epsilon_i \\] where \\(\\epsilon_i \\sim N(0, \\sigma^2)\\). However, it is rarely the case that a dataset will have a single predictor variable. It is also rarely the case that a response variable will only depend on a single variable. So in this chapter, we will extend our current linear model to allow a response to depend on multiple predictors. # read the data from the web autompg = read.table( &quot;http://archive.ics.uci.edu/ml/machine-learning-databases/auto-mpg/auto-mpg.data&quot;, quote = &quot;\\&quot;&quot;, comment.char = &quot;&quot;, stringsAsFactors = FALSE) # give the dataframe headers colnames(autompg) = c(&quot;mpg&quot;, &quot;cyl&quot;, &quot;disp&quot;, &quot;hp&quot;, &quot;wt&quot;, &quot;acc&quot;, &quot;year&quot;, &quot;origin&quot;, &quot;name&quot;) # remove missing data, which is stored as &quot;?&quot; autompg = subset(autompg, autompg$hp != &quot;?&quot;) # remove the plymouth reliant, as it causes some issues autompg = subset(autompg, autompg$name != &quot;plymouth reliant&quot;) # give the dataset row names, based on the engine, year and name rownames(autompg) = paste(autompg$cyl, &quot;cylinder&quot;, autompg$year, autompg$name) # remove the variable for name, as well as origin autompg = subset(autompg, select = c(&quot;mpg&quot;, &quot;cyl&quot;, &quot;disp&quot;, &quot;hp&quot;, &quot;wt&quot;, &quot;acc&quot;, &quot;year&quot;)) # change horsepower from character to numeric autompg$hp = as.numeric(autompg$hp) # check final structure of data str(autompg) ## &#39;data.frame&#39;: 390 obs. of 7 variables: ## $ mpg : num 18 15 18 16 17 15 14 14 14 15 ... ## $ cyl : int 8 8 8 8 8 8 8 8 8 8 ... ## $ disp: num 307 350 318 304 302 429 454 440 455 390 ... ## $ hp : num 130 165 150 150 140 198 220 215 225 190 ... ## $ wt : num 3504 3693 3436 3433 3449 ... ## $ acc : num 12 11.5 11 12 10.5 10 9 8.5 10 8.5 ... ## $ year: int 70 70 70 70 70 70 70 70 70 70 ... We will once again discuss a dataset with information about cars. This dataset, which can be found at the UCI Machine Learning Repository contains a response variable mpg which stores the city fuel efficiency of cars, as well as several predictor variables for the attributes of the vehicles. We load the data, and perform some basic tidying before moving on to analysis. For now we will focus on using two variables, wt and year, as predictor variables. That is, we would like to model the fuel efficiency (mpg) of a car as a function of its weight (wt) and model year (year). To do so, we will define the following linear model, \\[ Y_i = \\beta_0 + \\beta_1 x_{i1} + \\beta_2 x_{i2} + \\epsilon_i, \\qquad i = 1, 2, \\ldots, n \\] where \\(\\epsilon_i \\sim N(0, \\sigma^2)\\). In this notation we will define: \\(x_{i1}\\) as the weight (wt) of the \\(i\\)th car. \\(x_{i2}\\) as the model year (year) of the \\(i\\)th car. The picture below will visualize what we would like to accomplish. The data points \\((x_{i1}, x_{i2}, y_i)\\) now exist in 3-dimensional space, so instead of fitting a line to the data, we will fit a plane. (We’ll soon move to higher dimensions, so this will be the last example that is easy to visualize and think about this way.) How do we find such a plane? Well, we would like a plane that is as close as possible to the data points. That is, we would like it to minimize the errors it is making. How will we define these errors? Squared distance of course! So, we would like to minimize \\[ f(\\beta_0, \\beta_1, \\beta_2) = \\sum_{i = 1}^{n}(y_i - (\\beta_0 + \\beta_1 x_{i1} + \\beta_2 x_{i2}))^2 \\] with respect to \\(\\beta_0\\), \\(\\beta_1\\), and \\(\\beta_2\\). How do we do so? It is another straightforward multivariate calculus problem. All we have done is add an extra variable since we did this last time. So again, we take a derivative with respect to each of \\(\\beta_0\\), \\(\\beta_1\\), and \\(\\beta_2\\) and set them equal to zero, then solve the resulting system of equations. That is, \\[ \\begin{aligned} \\frac{\\partial f}{\\partial \\beta_0} &amp;= 0 \\\\ \\frac{\\partial f}{\\partial \\beta_1} &amp;= 0 \\\\ \\frac{\\partial f}{\\partial \\beta_2} &amp;= 0 \\end{aligned} \\] After doing so, we will once again obtain the normal equations. \\[ \\begin{aligned} n \\beta_0 + \\beta_1 \\sum_{i = 1}^{n} x_{i1} + \\beta_2 \\sum_{i = 1}^{n} x_{i2} &amp;= \\sum_{i = 1}^{n} y_i \\\\ \\beta_0 \\sum_{i = 1}^{n} x_{i1} + \\beta_1 \\sum_{i = 1}^{n} x_{i1}^2 + \\beta_2 \\sum_{i = 1}^{n} x_{i1}x_{i2} &amp;= \\sum_{i = 1}^{n} x_{i1}y_i \\\\ \\beta_0 \\sum_{i = 1}^{n} x_{i2} + \\beta_1 \\sum_{i = 1}^{n} x_{i1}x_{i2} + \\beta_2 \\sum_{i = 1}^{n} x_{i2}^2 &amp;= \\sum_{i = 1}^{n} x_{i2}y_i \\end{aligned} \\] We now have three equations and three variables, which we could solve, or we could simply let R solve for us. mpg_model = lm(mpg ~ wt + year, data = autompg) coef(mpg_model) ## (Intercept) wt year ## -14.637641945 -0.006634876 0.761401955 \\[ \\hat{y} = -14.6376419 + -0.0066349 x_1 + 0.761402 x_2 \\] Here we have once again fit our model using lm(), however we have introduced a new syntactical element. The formula mpg ~ wt + year now reads: “model the response variable mpg as a linear function of wt and year.” That is, it will estimate an intercept, as well as slope coefficients for wt and year. We then extract these as we have done before using coef(). In the multiple linear regression setting, some of the interpretations of the coefficients change slightly. Here, \\(\\hat{\\beta}_0 = -14.6376419\\) is our estimate for \\(\\beta_0\\), the mean miles per gallon for a car that weighs 0 pounds and was built in 1900. We see our estimate here is negative, which is a physical impossibility. However, this isn’t unexpected, as we shouldn’t expect our model to be accurate for cars from 1900 which weigh 0 pounds. (Because they never existed!) This isn’t much of a change from SLR. That is, \\(\\beta_0\\) is still simply the mean when all of the predictors are 0. The interpretation of the coefficients in front of our predictors is slightly different than before. For example \\(\\hat{\\beta}_1 = -0.0066349\\) is our estimate for \\(\\beta_1\\), the average change in miles per gallon for an increase in weight (\\(x_{1}\\)) of one-pound for a car of a certain model year, that is, for a fixed value of \\(x_{2}\\). Note that this coefficient is actually the same for any given value of \\(x_{2}\\). Later, we will look at models that allow for a different change in mean response for different values of \\(x_{2}\\). Also note that this estimate is negative, which we would expect since, in general, fuel efficiency decreases for larger vehicles. Recall that in the multiple linear regression setting, this interpretation is dependent on a fixed value for \\(x_{2}\\), that is, “for a car of a certain model year.” It is possible that the indirect relationship between fuel efficiency and weight does not hold when an additional factor, say year, is included, and thus we could have the sign of our coefficient flipped. Lastly, \\(\\hat{\\beta}_2 = 0.761402\\) is our estimate for \\(\\beta_2\\), the average change in miles per gallon for a one-year increase in model year (\\(x_{2}\\)) for a car of a certain weight, that is, for a fixed value of \\(x_{1}\\). It is not surprising that the estimate is positive. We expect that as time passes and the years march on, technology would improve so that a car of a specific weight would get better mileage now as compared to their predecessors. And yet, the coefficient could have been negative because we are also including weight as variable, and not strictly as a fixed value. 9.1 Matrix Approach to Regression In our above example we used two predictor variables, but it will only take a little more work to allow for an arbitrary number of predictor variables and derive their coefficient estimates. We can consider the model, \\[ Y_i = \\beta_0 + \\beta_1 x_{i1} + \\beta_2 x_{i2} + \\cdots + \\beta_{p-1} x_{i(p-1)} + \\epsilon_i, \\qquad i = 1, 2, \\ldots, n \\] where \\(\\epsilon_i \\sim N(0, \\sigma^2)\\). In this model, there are \\(p - 1\\) predictor variables, \\(x_1, x_2, \\cdots, x_{p-1}\\). There are a total of \\(p\\) \\(\\beta\\)-parameters and a single parameter \\(\\sigma^2\\) for the variance of the errors. (It should be noted that almost as often, authors will use \\(p\\) as the number of predictors, making the total number of \\(\\beta\\) parameters \\(p+1\\). This is always something you should be aware of when reading about multiple regression. There is not a standard that is used most often.) If we were to stack together the \\(n\\) linear equations that represent each \\(Y_i\\) into a column vector, we get the following. \\[ \\begin{bmatrix} Y_1 \\\\ Y_2 \\\\ \\vdots\\\\ Y_n \\\\ \\end{bmatrix} = \\begin{bmatrix} 1 &amp; x_{11} &amp; x_{12} &amp; \\cdots &amp; x_{1(p-1)} \\\\ 1 &amp; x_{21} &amp; x_{22} &amp; \\cdots &amp; x_{2(p-1)} \\\\ \\vdots &amp; \\vdots &amp; \\vdots &amp; &amp; \\vdots \\\\ 1 &amp; x_{n1} &amp; x_{n2} &amp; \\cdots &amp; x_{n(p-1)} \\\\ \\end{bmatrix} \\begin{bmatrix} \\beta_0 \\\\ \\beta_1 \\\\ \\beta_2 \\\\ \\vdots \\\\ \\beta_{p-1} \\\\ \\end{bmatrix} + \\begin{bmatrix} \\epsilon_1 \\\\ \\epsilon_2 \\\\ \\vdots\\\\ \\epsilon_n \\\\ \\end{bmatrix} \\] \\[ Y = X \\beta + \\epsilon \\] \\[ Y = \\begin{bmatrix} Y_1 \\\\ Y_2 \\\\ \\vdots\\\\ Y_n \\end{bmatrix}, \\quad X = \\begin{bmatrix} 1 &amp; x_{11} &amp; x_{12} &amp; \\cdots &amp; x_{1(p-1)} \\\\ 1 &amp; x_{21} &amp; x_{22} &amp; \\cdots &amp; x_{2(p-1)} \\\\ \\vdots &amp; \\vdots &amp; \\vdots &amp; &amp; \\vdots \\\\ 1 &amp; x_{n1} &amp; x_{n2} &amp; \\cdots &amp; x_{n(p-1)} \\\\ \\end{bmatrix}, \\quad \\beta = \\begin{bmatrix} \\beta_0 \\\\ \\beta_1 \\\\ \\beta_2 \\\\ \\vdots \\\\ \\beta_{p-1} \\\\ \\end{bmatrix}, \\quad \\epsilon = \\begin{bmatrix} \\epsilon_1 \\\\ \\epsilon_2 \\\\ \\vdots\\\\ \\epsilon_n \\end{bmatrix} \\] So now with data, \\[ y = \\begin{bmatrix} y_1 \\\\ y_2 \\\\ \\vdots\\\\ y_n \\end{bmatrix} \\] Just as before, we can estimate \\(\\beta\\) by minimizing, \\[ f(\\beta_0, \\beta_1, \\beta_2, \\cdots, \\beta_{p-1}) = \\sum_{i = 1}^{n}(y_i - (\\beta_0 + \\beta_1 x_{i1} + \\beta_2 x_{i2} + \\cdots + \\beta_{p-1} x_{i(p-1)}))^2, \\] which would require taking \\(p\\) derivatives, which result in the following normal equations. \\[ \\begin{bmatrix} n &amp; \\sum_{i = 1}^{n} x_{i1} &amp; \\sum_{i = 1}^{n} x_{i2} &amp; \\cdots &amp; \\sum_{i = 1}^{n} x_{i(p-1)} \\\\ \\sum_{i = 1}^{n} x_{i1} &amp; \\sum_{i = 1}^{n} x_{i1}^2 &amp; \\sum_{i = 1}^{n} x_{i1}x_{i2} &amp; \\cdots &amp; \\sum_{i = 1}^{n} x_{i1}x_{i(p-1)} \\\\ \\vdots &amp; \\vdots &amp; \\vdots &amp; &amp; \\vdots \\\\ \\sum_{i = 1}^{n} x_{i(p-1)} &amp; \\sum_{i = 1}^{n} x_{i(p-1)}x_{i1} &amp; \\sum_{i = 1}^{n} x_{i(p-1)}x_{i2} &amp; \\cdots &amp; \\sum_{i = 1}^{n} x_{i(p-1)}^2 \\\\ \\end{bmatrix} \\begin{bmatrix} \\beta_0 \\\\ \\beta_1 \\\\ \\vdots \\\\ \\beta_{p-1} \\\\ \\end{bmatrix} = \\begin{bmatrix} \\sum_{i = 1}^{n} y_i \\\\ \\sum_{i = 1}^{n} x_{i1}y_i \\\\ \\vdots \\\\ \\sum_{i = 1}^{n} x_{i(p-1)}y_i \\\\ \\end{bmatrix} \\] The normal equations can be written much more succinctly in matrix notation, \\[ X^\\top X \\beta = X^\\top y. \\] We can then solve this expression by multiplying both sides by the inverse of \\(X^\\top X\\), which exists, provided the columns of \\(X\\) are linearly independent. Then as always, we denote our solution with a hat. \\[ \\hat{\\beta} = \\left( X^\\top X \\right)^{-1}X^\\top y \\] To verify that this is what R has done for us in the case of two predictors, we create an \\(X\\) matrix. Note that the first column is all 1s, and the remaining columns contain the data. n = nrow(autompg) p = length(coef(mpg_model)) X = cbind(rep(1, n), autompg$wt, autompg$year) y = autompg$mpg (beta_hat = solve(t(X) %*% X) %*% t(X) %*% y) ## [,1] ## [1,] -14.637641945 ## [2,] -0.006634876 ## [3,] 0.761401955 coef(mpg_model) ## (Intercept) wt year ## -14.637641945 -0.006634876 0.761401955 \\[ \\hat{\\beta} = \\begin{bmatrix} -14.6376419 \\\\ -0.0066349 \\\\ 0.761402 \\\\ \\end{bmatrix} \\] In our new notation, the fitted values can be written \\[ \\hat{y} = X \\hat{\\beta}. \\] \\[ \\hat{y} = \\begin{bmatrix} \\hat{y}_1 \\\\ \\hat{y}_2 \\\\ \\vdots\\\\ \\hat{y}_n \\end{bmatrix} \\] Then, we can create a vector for the residual values, \\[ e = \\begin{bmatrix} e_1 \\\\ e_2 \\\\ \\vdots\\\\ e_n \\end{bmatrix} = \\begin{bmatrix} y_1 \\\\ y_2 \\\\ \\vdots\\\\ y_n \\end{bmatrix} - \\begin{bmatrix} \\hat{y}_1 \\\\ \\hat{y}_2 \\\\ \\vdots\\\\ \\hat{y}_n \\end{bmatrix}. \\] And lastly, we can update our estimate for \\(\\sigma^2\\). \\[ s_e^2 = \\frac{\\sum_{i=1}^n (y_i - \\hat{y}_i)^2}{n - p} = \\frac{e^\\top e}{n-p} \\] Recall, we like this estimate because it is unbiased, that is, \\[ \\text{E}[s_e^2] = \\sigma^2 \\] Note that the change from the SLR estimate to now is in the denominator. Specifically we now divide by \\(n - p\\) instead of \\(n - 2\\). Or actually, we should note that in the case of SLR, there are two \\(\\beta\\) parameters and thus \\(p = 2\\). Also note that if we fit the model \\(Y_i = \\beta + \\epsilon_i\\) that \\(\\hat{y} = \\bar{y}\\) and \\(p = 1\\) and \\(s_e^2\\) would become \\[ s_e^2 = \\frac{\\sum_{i=1}^n (y_i - \\bar{y})^2}{n - 1} \\] which is likely the very first sample variance you saw in a mathematical statistics class. The same reason for \\(n - 1\\) in this case, that we estimated one parameter, so we lose one degree of freedom. Now, in general, we are estimating \\(p\\) parameters, the \\(\\beta\\) parameters, so we lose \\(p\\) degrees of freedom. Also, recall that most often we will be interested in \\(s_e\\), the residual standard error as R calls it, \\[ s_e = \\sqrt{\\frac{\\sum_{i=1}^n (y_i - \\hat{y}_i)^2}{n - p}}. \\] In R, we could directly access \\(s_e\\) for a fitted model, as we have seen before. summary(mpg_model)$sigma ## [1] 3.431367 And we can now verify that our math above is indeed calculating the same quantities. y_hat = X %*% solve(t(X) %*% X) %*% t(X) %*% y e = y - y_hat sqrt(t(e) %*% e / (n - p)) ## [,1] ## [1,] 3.431367 sqrt(sum((y - y_hat) ^ 2) / (n - p)) ## [1] 3.431367 9.2 Sampling Distribution As we can see in the output below, the results of calling summary() are similar to SLR, but there are some differences, most obviously a new row for the added predictor variable. summary(mpg_model) ## ## Call: ## lm(formula = mpg ~ wt + year, data = autompg) ## ## Residuals: ## Min 1Q Median 3Q Max ## -8.852 -2.292 -0.100 2.039 14.325 ## ## Coefficients: ## Estimate Std. Error t value Pr(&gt;|t|) ## (Intercept) -1.464e+01 4.023e+00 -3.638 0.000312 *** ## wt -6.635e-03 2.149e-04 -30.881 &lt; 2e-16 *** ## year 7.614e-01 4.973e-02 15.312 &lt; 2e-16 *** ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 ## ## Residual standard error: 3.431 on 387 degrees of freedom ## Multiple R-squared: 0.8082, Adjusted R-squared: 0.8072 ## F-statistic: 815.6 on 2 and 387 DF, p-value: &lt; 2.2e-16 To understand these differences in detail, we will need to first obtain the sampling distribution of \\(\\hat{\\beta}\\). The derivation of the sampling distribution of \\(\\hat{\\beta}\\) involves the multivariate normal distribution. These brief notes from semesters past give a basic overview. These are simply for your information, as we will not present the derivation in full here. Our goal now is to obtain the distribution of the \\(\\hat{\\beta}\\) vector, \\[ \\hat{\\beta} = \\begin{bmatrix} \\hat{\\beta}_0 \\\\ \\hat{\\beta}_1 \\\\ \\hat{\\beta}_2 \\\\ \\vdots \\\\ \\hat{\\beta}_{p-1} \\end{bmatrix} \\] Recall from last time that when discussing sampling distributions, we now consider \\(\\hat{\\beta}\\) to be a random vector, thus we use \\(Y\\) instead of the data vector \\(y\\). \\[ \\hat{\\beta} = \\left( X^\\top X \\right)^{-1}X^\\top Y \\] Then it is a consequence of the multivariate normal distribution that, \\[ \\hat{\\beta} \\sim N\\left(\\beta, \\sigma^2 \\left(X^\\top X\\right)^{-1} \\right). \\] We then have \\[ \\text{E}[\\hat{\\beta}] = \\beta \\] and for any \\(\\hat{\\beta}_j\\) we have \\[ \\text{E}[\\hat{\\beta}_j] = \\beta_j. \\] We also have \\[ \\text{Var}[\\hat{\\beta}] = \\sigma^2 \\left( X^\\top X \\right)^{-1} \\] and for any \\(\\hat{\\beta}_j\\) we have \\[ \\text{Var}[\\hat{\\beta}_j] = \\sigma^2 C_{jj} \\] where \\[ C = \\left(X^\\top X\\right)^{-1} \\] and the elements of \\(C\\) are denoted \\[ C = \\begin{bmatrix} C_{00} &amp; C_{01} &amp; C_{02} &amp; \\cdots &amp; C_{0(p-1)} \\\\ C_{10} &amp; C_{11} &amp; C_{12} &amp; \\cdots &amp; C_{1(p-1)} \\\\ C_{20} &amp; C_{21} &amp; C_{22} &amp; \\cdots &amp; C_{2(p-1)} \\\\ \\vdots &amp; \\vdots &amp; \\vdots &amp; &amp; \\vdots \\\\ C_{(p-1)0} &amp; C_{(p-1)1} &amp; C_{(p-1)2} &amp; \\cdots &amp; C_{(p-1)(p-1)} \\\\ \\end{bmatrix}. \\] Essentially, the diagonal elements correspond to the \\(\\beta\\) vector. Then the standard error for the \\(\\hat{\\beta}\\) vector is given by \\[ \\text{SE}[\\hat{\\beta}] = s_e \\sqrt{\\left( X^\\top X \\right)^{-1}} \\] and for a particular \\(\\hat{\\beta}_j\\) \\[ \\text{SE}[\\hat{\\beta}_j] = s_e \\sqrt{C_{jj}}. \\] Lastly, each of the \\(\\hat{\\beta}_j\\) follows a normal distribution, \\[ \\hat{\\beta}_j \\sim N\\left(\\beta_j, \\sigma^2 C_{jj} \\right). \\] thus \\[ \\frac{\\hat{\\beta}_j - \\beta_j}{s_e \\sqrt{C_{jj}}} \\sim t_{n-p}. \\] Now that we have the necessary distributional results, we can move on to perform tests and make interval estimates. 9.2.1 Single Parameter Tests The first test we will see is a test for a single \\(\\beta_j\\). \\[ H_0: \\beta_j = 0 \\quad \\text{vs} \\quad H_1: \\beta_j \\neq 0 \\] Again, the test statistic takes the form \\[ \\text{TS} = \\frac{\\text{EST} - \\text{HYP}}{\\text{SE}}. \\] In particular, \\[ t = \\frac{\\hat{\\beta}_j - \\beta_j}{\\text{SE}[\\hat{\\beta}_j]} = \\frac{\\hat{\\beta}_j-0}{s_e\\sqrt{C_{jj}}}, \\] which, under the null hypothesis, follows a \\(t\\) distribution with \\(n - p\\) degrees of freedom. Recall our model for mpg, \\[ Y_i = \\beta_0 + \\beta_1 x_{i1} + \\beta_2 x_{i2} + \\epsilon_i, \\qquad i = 1, 2, \\ldots, n \\] where \\(\\epsilon_i \\sim N(0, \\sigma^2)\\). \\(x_{i1}\\) as the weight (wt) of the \\(i\\)th car. \\(x_{i2}\\) as the model year (year) of the \\(i\\)th car. Then the test \\[ H_0: \\beta_1 = 0 \\quad \\text{vs} \\quad H_1: \\beta_1 \\neq 0 \\] can be found in the summary() output, in particular: summary(mpg_model)$coef ## Estimate Std. Error t value Pr(&gt;|t|) ## (Intercept) -14.637641945 4.0233913563 -3.638135 3.118311e-04 ## wt -0.006634876 0.0002148504 -30.881372 1.850466e-106 ## year 0.761401955 0.0497265950 15.311765 1.036597e-41 The estimate (Estimate), standard error (Std. Error), test statistic (t value), and p-value (Pr(&gt;|t|)) for this test are displayed in the second row, labeled wt. Remember that the p-value given here is specifically for a two-sided test, where the hypothesized value is 0. Also note in this case, by hypothesizing that \\(\\beta_1 = 0\\) the null and alternative essentially specify two different models: \\(H_0\\): \\(Y = \\beta_0 + \\beta_2 x_{2} + \\epsilon\\) \\(H_1\\): \\(Y = \\beta_0 + \\beta_1 x_{1} + \\beta_2 x_{2} + \\epsilon\\) This is important. We are not simply testing whether or not there is a relationship between weight and fuel efficiency. We are testing if there is a relationship between weight and fuel efficiency, given that a term for year is in the model. (Note, we dropped some indexing here, for readability.) 9.2.2 Confidence Intervals Since \\(\\hat{\\beta}_j\\) is our estimate for \\(\\beta_j\\) and we have \\[ \\text{E}[\\hat{\\beta}_j] = \\beta_j \\] as well as the standard error, \\[ \\text{SE}[\\hat{\\beta}_j] = s_e\\sqrt{C_{jj}} \\] and the sampling distribution of \\(\\hat{\\beta}_j\\) is Normal, then we can easily construct confidence intervals for each of the \\(\\hat{\\beta}_j\\). \\[ \\hat{\\beta}_j \\pm t_{\\alpha/2, n - p} \\cdot s_e\\sqrt{C_{jj}} \\] We can find these in R using the same method as before. Now there will simply be additional rows for the additional \\(\\beta\\). confint(mpg_model, level = 0.99) ## 0.5 % 99.5 % ## (Intercept) -25.052563681 -4.222720208 ## wt -0.007191036 -0.006078716 ## year 0.632680051 0.890123859 9.2.3 Confidence Intervals for Mean Response As we saw in SLR, we can create confidence intervals for the mean response, that is, an interval estimate for \\(\\text{E}[Y \\mid X = x]\\). In SLR, the mean of \\(Y\\) was only dependent on a single value \\(x\\). Now, in multiple regression, \\(\\text{E}[Y \\mid X = x]\\) is dependent on the value of each of the predictors, so we define the vector \\(x_0\\) to be, \\[ x_{0} = \\begin{bmatrix} 1 \\\\ x_{01} \\\\ x_{02} \\\\ \\vdots \\\\ x_{0(p-1)} \\\\ \\end{bmatrix}. \\] Then our estimate of \\(\\text{E}[Y \\mid X = x_0]\\) for a set of values \\(x_0\\) is given by \\[ \\begin{aligned} \\hat{y}(x_0) &amp;= x_{0}^\\top\\hat{\\beta} \\\\ &amp;= \\hat{\\beta}_0 + \\hat{\\beta}_1 x_{01} + \\hat{\\beta}_2 x_{02} + \\cdots + \\hat{\\beta}_{p-1} x_{0(p-1)}. \\end{aligned} \\] As with SLR, this is an unbiased estimate. \\[ \\begin{aligned} \\text{E}[\\hat{y}(x_0)] &amp;= x_{0}^\\top\\beta \\\\ &amp;= \\beta_0 + \\beta_1 x_{01} + \\beta_2 x_{02} + \\cdots + \\beta_{p-1} x_{0(p-1)} \\end{aligned} \\] To make an interval estimate, we will also need its standard error. \\[ \\text{SE}[\\hat{y}(x_0)] = s_e \\sqrt{x_{0}^\\top\\left(X^\\top X\\right)^{-1}x_{0}} \\] Putting it all together, we obtain a confidence interval for the mean response. \\[ \\hat{y}(x_0) \\pm t_{\\alpha/2, n - p} \\cdot s_e \\sqrt{x_{0}^\\top\\left(X^\\top X\\right)^{-1}x_{0}} \\] The math has changed a bit, but the process in R remains almost identical. Here, we create a data frame for two additional cars. One car that weighs 3500 pounds produced in 1976, as well as a second car that weighs 5000 pounds which was produced in 1981. new_cars = data.frame(wt = c(3500, 5000), year = c(76, 81)) new_cars ## wt year ## 1 3500 76 ## 2 5000 81 We can then use the predict() function with interval = \"confidence\" to obtain intervals for the mean fuel efficiency for both new cars. Again, it is important to make the data passed to newdata a data frame, so that R knows which values are for which variables. predict(mpg_model, newdata = new_cars, interval = &quot;confidence&quot;, level = 0.99) ## fit lwr upr ## 1 20.00684 19.4712 20.54248 ## 2 13.86154 12.3341 15.38898 R then reports the estimate \\(\\hat{y}(x_0)\\) (fit) for each, as well as the lower (lwr) and upper (upr) bounds for the interval at a desired level (99%). A word of caution here: one of these estimates is good while one is suspect. new_cars$wt ## [1] 3500 5000 range(autompg$wt) ## [1] 1613 5140 Note that both of the weights of the new cars are within the range of observed values. new_cars$year ## [1] 76 81 range(autompg$year) ## [1] 70 82 As are the years of each of the new cars. plot(year ~ wt, data = autompg, pch = 20, col = &quot;dodgerblue&quot;, cex = 1.5) points(new_cars, col = &quot;darkorange&quot;, cex = 3, pch = &quot;X&quot;) However, we have to consider weight and year together now. And based on the above plot, one of the new cars is within the “blob” of observed values, while the other, the car from 1981 weighing 5000 pounds, is noticeably outside of the observed values. This is a hidden extrapolation which you should be aware of when using multiple regression. Shifting gears back to the new data pair that can be reasonably estimated, we do a quick verification of some of the mathematics in R. x0 = c(1, 3500, 76) x0 %*% beta_hat ## [,1] ## [1,] 20.00684 \\[ x_{0} = \\begin{bmatrix} 1 \\\\ 3500 \\\\ 76 \\\\ \\end{bmatrix} \\] \\[ \\hat{\\beta} = \\begin{bmatrix} -14.6376419 \\\\ -0.0066349 \\\\ 0.761402 \\\\ \\end{bmatrix} \\] \\[ \\hat{y}(x_0) = x_{0}^\\top\\hat{\\beta} = \\begin{bmatrix} 1 &amp; 3500 &amp; 76 \\\\ \\end{bmatrix} \\begin{bmatrix} -14.6376419 \\\\ -0.0066349 \\\\ 0.761402 \\\\ \\end{bmatrix}= 20.0068411 \\] Also note that, using a particular value for \\(x_0\\), we can essentially extract certain \\(\\hat{\\beta}_j\\) values. beta_hat ## [,1] ## [1,] -14.637641945 ## [2,] -0.006634876 ## [3,] 0.761401955 x0 = c(0, 0, 1) x0 %*% beta_hat ## [,1] ## [1,] 0.761402 With this in mind, confidence intervals for the individual \\(\\hat{\\beta}_j\\) are actually a special case of a confidence interval for mean response. 9.2.4 Prediction Intervals As with SLR, creating prediction intervals involves one slight change to the standard error to account for the fact that we are now considering an observation, instead of a mean. Here we use \\(\\hat{y}(x_0)\\) to estimate \\(Y_0\\), a new observation of \\(Y\\) at the predictor vector \\(x_0\\). \\[ \\begin{aligned} \\hat{y}(x_0) &amp;= x_{0}^\\top\\hat{\\beta} \\\\ &amp;= \\hat{\\beta}_0 + \\hat{\\beta}_1 x_{01} + \\hat{\\beta}_2 x_{02} + \\cdots + \\hat{\\beta}_{p-1} x_{0(p-1)} \\end{aligned} \\] \\[ \\begin{aligned} \\text{E}[\\hat{y}(x_0)] &amp;= x_{0}^\\top\\beta \\\\ &amp;= \\beta_0 + \\beta_1 x_{01} + \\beta_2 x_{02} + \\cdots + \\beta_{p-1} x_{0(p-1)} \\end{aligned} \\] As we did with SLR, we need to account for the additional variability of an observation about its mean. \\[ \\text{SE}[\\hat{y}(x_0) + \\epsilon] = s_e \\sqrt{1 + x_{0}^\\top\\left(X^\\top X\\right)^{-1}x_{0}} \\] Then we arrive at our updated prediction interval for MLR. \\[ \\hat{y}(x_0) \\pm t_{\\alpha/2, n - p} \\cdot s_e \\sqrt{1 + x_{0}^\\top\\left(X^\\top X\\right)^{-1}x_{0}} \\] new_cars ## wt year ## 1 3500 76 ## 2 5000 81 predict(mpg_model, newdata = new_cars, interval = &quot;prediction&quot;, level = 0.99) ## fit lwr upr ## 1 20.00684 11.108294 28.90539 ## 2 13.86154 4.848751 22.87432 9.3 Significance of Regression The decomposition of variation that we had seen in SLR still holds for MLR. \\[ \\sum_{i=1}^{n}(y_i - \\bar{y})^2 = \\sum_{i=1}^{n}(y_i - \\hat{y}_i)^2 + \\sum_{i=1}^{n}(\\hat{y}_i - \\bar{y})^2 \\] That is, \\[ \\text{SST} = \\text{SSE} + \\text{SSReg}. \\] This means that we can still calculate \\(R^2\\) in the same manner as before, which R continues to do automatically. summary(mpg_model)$r.squared ## [1] 0.8082355 The interpretation changes slightly as compared to SLR. In this MLR case, we say that \\(80.82\\%\\) of the observed variation in miles per gallon is explained by the linear relationship with the two predictor variables, weight and year. In multiple regression, the significance of regression test is \\[ H_0: \\beta_1 = \\beta_2 = \\cdots = \\beta_{p - 1} = 0. \\] Here, we see that the null hypothesis sets all of the \\(\\beta_j\\) equal to 0, except the intercept, \\(\\beta_0\\). We could then say that the null model, or “model under the null hypothesis” is \\[ Y_i = \\beta_0 + \\epsilon_i. \\] This is a model where the regression is insignificant. None of the predictors have a significant linear relationship with the response. Notationally, we will denote the fitted values of this model as \\(\\hat{y}_{0i}\\), which in this case happens to be: \\[ \\hat{y}_{0i} = \\bar{y}. \\] The alternative hypothesis here is that at least one of the \\(\\beta_j\\) from the null hypothesis is not 0. \\[ H_1: \\text{At least one of } \\beta_j \\neq 0, j = 1, 2, \\cdots, (p-1) \\] We could then say that the full model, or “model under the alternative hypothesis” is \\[ Y_i = \\beta_0 + \\beta_1 x_{i1} + \\beta_2 x_{i2} + \\cdots + \\beta_{(p-1)} x_{i(p-1)} + \\epsilon_i \\] This is a model where the regression is significant. At least one of the predictors has a significant linear relationship with the response. There is some linear relationship between \\(y\\) and the predictors, \\(x_1, x_2, \\ldots, x_{p - 1}\\). We will denote the fitted values of this model as \\(\\hat{y}_{1i}\\). To develop the \\(F\\) test for the significance of the regression, we will arrange the variance decomposition into an ANOVA table. Source Sum of Squares Degrees of Freedom Mean Square \\(F\\) Regression \\(\\sum_{i=1}^{n}(\\hat{y}_{1i} - \\bar{y})^2\\) \\(p - 1\\) \\(\\text{SSReg} / (p - 1)\\) \\(\\text{MSReg} / \\text{MSE}\\) Error \\(\\sum_{i=1}^{n}(y_i - \\hat{y}_{1i})^2\\) \\(n - p\\) \\(\\text{SSE} / (n - p)\\) Total \\(\\sum_{i=1}^{n}(y_i - \\bar{y})^2\\) \\(n - 1\\) In summary, the \\(F\\) statistic is \\[ F = \\frac{\\sum_{i=1}^{n}(\\hat{y}_{1i} - \\bar{y})^2 / (p - 1)}{\\sum_{i=1}^{n}(y_i - \\hat{y}_{1i})^2 / (n - p)}, \\] and the p-value is calculated as \\[ P(F_{p-1, n-p} &gt; F) \\] since we reject for large values of \\(F\\). A large value of the statistic corresponds to a large portion of the variance being explained by the regression. Here \\(F_{p-1, n-p}\\) represents a random variable which follows an \\(F\\) distribution with \\(p - 1\\) and \\(n - p\\) degrees of freedom. To perform this test in R, we first explicitly specify the two models in R and save the results in different variables. We then use anova() to compare the two models, giving anova() the null model first and the alternative (full) model second. (Specifying the full model first will result in the same p-value, but some nonsensical intermediate values.) In this case, \\(H_0\\): \\(Y_i = \\beta_0 + \\epsilon_i\\) \\(H_1\\): \\(Y_i = \\beta_0 + \\beta_1 x_{i1} + \\beta_2 x_{i2} + \\epsilon_i\\) That is, in the null model, we use neither of the predictors, whereas in the full (alternative) model, at least one of the predictors is useful. null_mpg_model = lm(mpg ~ 1, data = autompg) full_mpg_model = lm(mpg ~ wt + year, data = autompg) anova(null_mpg_model, full_mpg_model) ## Analysis of Variance Table ## ## Model 1: mpg ~ 1 ## Model 2: mpg ~ wt + year ## Res.Df RSS Df Sum of Sq F Pr(&gt;F) ## 1 389 23761.7 ## 2 387 4556.6 2 19205 815.55 &lt; 2.2e-16 *** ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 First, notice that R does not display the results in the same manner as the table above. More important than the layout of the table are its contents. We see that the value of the \\(F\\) statistic is 815.55, and the p-value is extremely low, so we reject the null hypothesis at any reasonable \\(\\alpha\\) and say that the regression is significant. At least one of wt or year has a useful linear relationship with mpg. summary(mpg_model) ## ## Call: ## lm(formula = mpg ~ wt + year, data = autompg) ## ## Residuals: ## Min 1Q Median 3Q Max ## -8.852 -2.292 -0.100 2.039 14.325 ## ## Coefficients: ## Estimate Std. Error t value Pr(&gt;|t|) ## (Intercept) -1.464e+01 4.023e+00 -3.638 0.000312 *** ## wt -6.635e-03 2.149e-04 -30.881 &lt; 2e-16 *** ## year 7.614e-01 4.973e-02 15.312 &lt; 2e-16 *** ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 ## ## Residual standard error: 3.431 on 387 degrees of freedom ## Multiple R-squared: 0.8082, Adjusted R-squared: 0.8072 ## F-statistic: 815.6 on 2 and 387 DF, p-value: &lt; 2.2e-16 Notice that the value reported in the row for F-statistic is indeed the \\(F\\) test statistic for the significance of regression test, and additionally it reports the two relevant degrees of freedom. Also, note that none of the individual \\(t\\)-tests are equivalent to the \\(F\\)-test as they were in SLR. This equivalence only holds for SLR because the individual test for \\(\\beta_1\\) is the same as testing for all non-intercept parameters, since there is only one. We can also verify the sums of squares and degrees of freedom directly in R. You should match these to the table from R and use this to match R’s output to the written table above. # SSReg sum((fitted(full_mpg_model) - fitted(null_mpg_model)) ^ 2) ## [1] 19205.03 # SSE sum(resid(full_mpg_model) ^ 2) ## [1] 4556.646 # SST sum(resid(null_mpg_model) ^ 2) ## [1] 23761.67 # Degrees of Freedom: Regression length(coef(full_mpg_model)) - length(coef(null_mpg_model)) ## [1] 2 # Degrees of Freedom: Error length(resid(full_mpg_model)) - length(coef(full_mpg_model)) ## [1] 387 # Degrees of Freedom: Total length(resid(null_mpg_model)) - length(coef(null_mpg_model)) ## [1] 389 9.4 Nested Models The significance of regression test is actually a special case of testing what we will call nested models. More generally we can compare two models, where one model is “nested” inside the other, meaning one model contains a subset of the predictors from only the larger model. Consider the following full model, \\[ Y_i = \\beta_0 + \\beta_1 x_{i1} + \\beta_2 x_{i2} + \\cdots + \\beta_{(p-1)} x_{i(p-1)} + \\epsilon_i \\] This model has \\(p - 1\\) predictors, for a total of \\(p\\) \\(\\beta\\)-parameters. We will denote the fitted values of this model as \\(\\hat{y}_{1i}\\). Let the null model be \\[ Y_i = \\beta_0 + \\beta_1 x_{i1} + \\beta_2 x_{i2} + \\cdots + \\beta_{(q-1)} x_{i(q-1)} + \\epsilon_i \\] where \\(q &lt; p\\). This model has \\(q - 1\\) predictors, for a total of \\(q\\) \\(\\beta\\)-parameters. We will denote the fitted values of this model as \\(\\hat{y}_{0i}\\). The difference between these two models can be codified by the null hypothesis of a test. \\[ H_0: \\beta_q = \\beta_{q+1} = \\cdots = \\beta_{p - 1} = 0. \\] Specifically, the \\(\\beta\\)-parameters from the full model that are not in the null model are zero. The resulting model, which is nested, is the null model. We can then perform this test using an \\(F\\)-test, which is the result of the following ANOVA table. Source Sum of Squares Degrees of Freedom Mean Square \\(F\\) Diff \\(\\sum_{i=1}^{n}(\\hat{y}_{1i} - \\hat{y}_{0i})^2\\) \\(p - q\\) \\(\\text{SSD} / (p - q)\\) \\(\\text{MSD} / \\text{MSE}\\) Full \\(\\sum_{i=1}^{n}(y_i - \\hat{y}_{1i})^2\\) \\(n - p\\) \\(\\text{SSE} / (n - p)\\) Null \\(\\sum_{i=1}^{n}(y_i - \\hat{y}_{0i})^2\\) \\(n - q\\) \\[ F = \\frac{\\sum_{i=1}^{n}(\\hat{y}_{1i} - \\hat{y}_{0i})^2 / (p - q)}{\\sum_{i=1}^{n}(y_i - \\hat{y}_{1i})^2 / (n - p)}. \\] Notice that the row for “Diff” compares the sum of the squared differences of the fitted values. The degrees of freedom is then the difference of the number of \\(\\beta\\)-parameters estimated between the two models. For example, the autompg dataset has a number of additional variables that we have yet to use. names(autompg) ## [1] &quot;mpg&quot; &quot;cyl&quot; &quot;disp&quot; &quot;hp&quot; &quot;wt&quot; &quot;acc&quot; &quot;year&quot; We’ll continue to use mpg as the response, but now we will consider two different models. Full: mpg ~ wt + year + cyl + disp + hp + acc Null: mpg ~ wt + year Note that these are nested models, as the null model contains a subset of the predictors from the full model, and no additional predictors. Both models have an intercept \\(\\beta_0\\) as well as a coefficient in front of each of the predictors. We could then write the null hypothesis for comparing these two models as, \\[ H_0: \\beta_{\\texttt{cyl}} = \\beta_{\\texttt{disp}} = \\beta_{\\texttt{hp}} = \\beta_{\\texttt{acc}} = 0 \\] The alternative is simply that at least one of the \\(\\beta_{j}\\) from the null is not 0. To perform this test in R we first define both models, then give them to the anova() commands. null_mpg_model = lm(mpg ~ wt + year, data = autompg) #full_mpg_model = lm(mpg ~ wt + year + cyl + disp + hp + acc, data = autompg) full_mpg_model = lm(mpg ~ ., data = autompg) anova(null_mpg_model, full_mpg_model) ## Analysis of Variance Table ## ## Model 1: mpg ~ wt + year ## Model 2: mpg ~ cyl + disp + hp + wt + acc + year ## Res.Df RSS Df Sum of Sq F Pr(&gt;F) ## 1 387 4556.6 ## 2 383 4530.5 4 26.18 0.5533 0.6967 Here we have used the formula mpg ~ . to define to full model. This is the same as the commented out line. Specifically, this is a common shortcut in R which reads, “model mpg as the response with each of the remaining variables in the data frame as predictors.” Here we see that the value of the \\(F\\) statistic is 0.553, and the p-value is very large, so we fail to reject the null hypothesis at any reasonable \\(\\alpha\\) and say that none of cyl, disp, hp, and acc are significant with wt and year already in the model. Again, we verify the sums of squares and degrees of freedom directly in R. You should match these to the table from R, and use this to match R’s output to the written table above. # SSDiff sum((fitted(full_mpg_model) - fitted(null_mpg_model)) ^ 2) ## [1] 26.17981 # SSE (For Full) sum(resid(full_mpg_model) ^ 2) ## [1] 4530.466 # SST (For Null) sum(resid(null_mpg_model) ^ 2) ## [1] 4556.646 # Degrees of Freedom: Diff length(coef(full_mpg_model)) - length(coef(null_mpg_model)) ## [1] 4 # Degrees of Freedom: Full length(resid(full_mpg_model)) - length(coef(full_mpg_model)) ## [1] 383 # Degrees of Freedom: Null length(resid(null_mpg_model)) - length(coef(null_mpg_model)) ## [1] 387 9.5 Simulation Since we ignored the derivation of certain results, we will again use simulation to convince ourselves of some of the above results. In particular, we will simulate samples of size n = 100 from the model \\[ Y_i = 5 + -2 x_{i1} + 6 x_{i2} + \\epsilon_i, \\qquad i = 1, 2, \\ldots, n \\] where \\(\\epsilon_i \\sim N(0, \\sigma^2 = 16)\\). Here we have two predictors, so \\(p = 3\\). set.seed(1337) n = 100 # sample size p = 3 beta_0 = 5 beta_1 = -2 beta_2 = 6 sigma = 4 As is the norm with regression, the \\(x\\) values are considered fixed and known quantities, so we will simulate those first, and they remain the same for the rest of the simulation study. Also note we create an x0 which is all 1, which we need to create our X matrix. If you look at the matrix formulation of regression, this unit vector of all 1s is a “predictor” that puts the intercept into the model. We also calculate the C matrix for later use. x0 = rep(1, n) x1 = sample(seq(1, 10, length = n)) x2 = sample(seq(1, 10, length = n)) X = cbind(x0, x1, x2) C = solve(t(X) %*% X) We then simulate the response according the model above. Lastly, we place the two predictors and response into a data frame. Note that we do not place x0 in the data frame. This is a result of R adding an intercept by default. eps = rnorm(n, mean = 0, sd = sigma) y = beta_0 + beta_1 * x1 + beta_2 * x2 + eps sim_data = data.frame(x1, x2, y) Plotting this data and fitting the regression produces the following plot. We then calculate \\[ \\hat{\\beta} = \\left( X^\\top X \\right)^{-1}X^\\top y. \\] (beta_hat = C %*% t(X) %*% y) ## [,1] ## x0 7.290735 ## x1 -2.282176 ## x2 5.843424 Notice that these values are the same as the coefficients found using lm() in R. coef(lm(y ~ x1 + x2, data = sim_data)) ## (Intercept) x1 x2 ## 7.290735 -2.282176 5.843424 Also, these values are close to what we would expect. c(beta_0, beta_1, beta_2) ## [1] 5 -2 6 We then calculated the fitted values in order to calculate \\(s_e\\), which we see is the same as the sigma which is returned by summary(). y_hat = X %*% beta_hat (s_e = sqrt(sum((y - y_hat) ^ 2) / (n - p))) ## [1] 4.294307 summary(lm(y ~ x1 + x2, data = sim_data))$sigma ## [1] 4.294307 So far so good. Everything checks out. Now we will finally simulate from this model repeatedly in order to obtain an empirical distribution of \\(\\hat{\\beta}_2\\). We expect \\(\\hat{\\beta}_2\\) to follow a normal distribution, \\[ \\hat{\\beta}_2 \\sim N\\left(\\beta_2, \\sigma^2 C_{22} \\right). \\] In this case, \\[ \\hat{\\beta}_2 \\sim N\\left(\\mu = 6, \\sigma^2 = 16 \\times 0.0014534 = 0.0232549 \\right). \\] \\[ \\hat{\\beta}_2 \\sim N\\left(\\mu = 6, \\sigma^2 = 0.0232549 \\right). \\] Note that \\(C_{22}\\) corresponds to the element in the third row and third column since \\(\\beta_2\\) is the third parameter in the model and because R is indexed starting at 1. However, we index the \\(C\\) matrix starting at 0 to match the diagonal elements to the corresponding \\(\\beta_j\\). C[3, 3] ## [1] 0.00145343 C[2 + 1, 2 + 1] ## [1] 0.00145343 sigma ^ 2 * C[2 + 1, 2 + 1] ## [1] 0.02325487 We now perform the simulation a large number of times. Each time, we update the y variable in the data frame, leaving the x variables the same. We then fit a model, and store \\(\\hat{\\beta}_2\\). num_sims = 10000 beta_hat_2 = rep(0, num_sims) for(i in 1:num_sims) { eps = rnorm(n, mean = 0 , sd = sigma) sim_data$y = beta_0 * x0 + beta_1 * x1 + beta_2 * x2 + eps fit = lm(y ~ x1 + x2, data = sim_data) beta_hat_2[i] = coef(fit)[3] } We then see that the mean of the simulated values is close to the true value of \\(\\beta_2\\). mean(beta_hat_2) ## [1] 5.999723 beta_2 ## [1] 6 We also see that the variance of the simulated values is close to the true variance of \\(\\hat{\\beta}_2\\). \\[ \\text{Var}[\\hat{\\beta}_2] = \\sigma^2 \\cdot C_{22} = 16 \\times 0.0014534 = 0.0232549 \\] var(beta_hat_2) ## [1] 0.02343408 sigma ^ 2 * C[2 + 1, 2 + 1] ## [1] 0.02325487 The standard deviations found from the simulated data and the parent population are also very close. sd(beta_hat_2) ## [1] 0.1530819 sqrt(sigma ^ 2 * C[2 + 1, 2 + 1]) ## [1] 0.1524955 Lastly, we plot a histogram of the simulated values, and overlay the true distribution. hist(beta_hat_2, prob = TRUE, breaks = 20, xlab = expression(hat(beta)[2]), main = &quot;&quot;, border = &quot;dodgerblue&quot;) curve(dnorm(x, mean = beta_2, sd = sqrt(sigma ^ 2 * C[2 + 1, 2 + 1])), col = &quot;darkorange&quot;, add = TRUE, lwd = 3) This looks good! The simulation-based histogram appears to be Normal with mean 6 and spread of about 0.15 as you measure from center to inflection point. That matches really well with the sampling distribution of \\(\\hat{\\beta}_2 \\sim N\\left(\\mu = 6, \\sigma^2 = 0.0232549 \\right)\\). One last check, we verify the \\(68 - 95 - 99.7\\) rule. sd_bh2 = sqrt(sigma ^ 2 * C[2 + 1, 2 + 1]) # We expect these to be: 0.68, 0.95, 0.997 mean(beta_2 - 1 * sd_bh2 &lt; beta_hat_2 &amp; beta_hat_2 &lt; beta_2 + 1 * sd_bh2) ## [1] 0.6807 mean(beta_2 - 2 * sd_bh2 &lt; beta_hat_2 &amp; beta_hat_2 &lt; beta_2 + 2 * sd_bh2) ## [1] 0.9529 mean(beta_2 - 3 * sd_bh2 &lt; beta_hat_2 &amp; beta_hat_2 &lt; beta_2 + 3 * sd_bh2) ## [1] 0.9967 9.6 R Markdown The R Markdown file for this chapter can be found here: mlr.Rmd The file was created using R version 4.3.2. "],["model-building.html", "Chapter 10 Model Building 10.1 Family, Form, and Fit 10.2 Explanation versus Prediction 10.3 Summary 10.4 R Markdown", " Chapter 10 Model Building “Statisticians, like artists, have the bad habit of falling in love with their models.” — George Box Let’s take a step back and consider the process of finding a model for data at a higher level. We are attempting to find a model for a response variable \\(y\\) based on a number of predictors \\(x_1, x_2, x_3, \\ldots, x_{p-1}\\). Essentially, we are trying to discover the functional relationship between \\(y\\) and the predictors. In the previous chapter we were fitting models for a car’s fuel efficiency (mpg) as a function of its attributes (wt, year, cyl, disp, hp, acc). We also consider \\(y\\) to be a function of some noise. Rarely if ever do we expect there to be an exact functional relationship between the predictors and the response. \\[ y = f(x_1, x_2, x_3, \\ldots, x_{p-1}) + \\epsilon \\] We can think of this as \\[ \\text{response} = \\text{signal} + \\text{noise}. \\] We could consider all sorts of complicated functions for \\(f\\). You will likely encounter several ways of doing this in future machine learning courses. So far in this course we have focused on (multiple) linear regression. That is \\[ \\begin{aligned} y &amp;= f(x_1, x_2, x_3, \\ldots, x_{p-1}) + \\epsilon \\\\ &amp;= \\beta_0 + \\beta_1 x_{1} + \\beta_2 x_{2} + \\cdots + \\beta_{p-1} x_{p-1} + \\epsilon \\end{aligned} \\] In the big picture of possible models that we could fit to this data, this is a rather restrictive model. What do we mean by a restrictive model? 10.1 Family, Form, and Fit When modeling data, there are a number of choices that need to be made. What family of models will be considered? What form of the model will be used? How will the model be fit? Let’s work backwards and discuss each of these. 10.1.1 Fit Consider one of the simplest models we could fit to data, simple linear regression. \\[ y = f(x_1, x_2, x_3, \\ldots, x_{p-1}) + \\epsilon = \\beta_0 + \\beta_1 x_{1} + \\epsilon \\] So here, despite having multiple predictors, we chose to use only one. How is this model fit? We will almost exclusively use the method of least squares, but recall, we had seen alternative methods of fitting this model. \\[ \\underset{\\beta_0, \\beta_1}{\\mathrm{argmin}} \\max|y_i - (\\beta_0 + \\beta_1 x_i)| \\] \\[ \\underset{\\beta_0, \\beta_1}{\\mathrm{argmin}} \\sum_{i = 1}^{n}|y_i - (\\beta_0 + \\beta_1 x_i)| \\] \\[ \\underset{\\beta_0, \\beta_1}{\\mathrm{argmin}} \\sum_{i = 1}^{n}(y_i - (\\beta_0 + \\beta_1 x_i))^2 \\] Any of these methods (we will always use the last, least squares) will obtain estimates of the unknown parameters \\(\\beta_0\\) and \\(\\beta_1\\). Since those are the only unknowns of the specified model, we have then fit the model. The fitted model is then \\[ \\hat{y} = \\hat{f}(x_1, x_2, x_3, \\ldots, x_{p-1}) = \\hat{\\beta}_0 + \\hat{\\beta}_1 x_{1} \\] Note that, now we have dropped the term for the noise. We don’t make any effort to model the noise, only the signal. 10.1.2 Form What are the different forms a model could take? Currently, for the linear models we have considered, the only method for altering the form of the model is to control the predictors used. For example, one form of the multiple linear regression model is simple linear regression. \\[ y = f(x_1, x_2, x_3, \\ldots, x_{p-1}) + \\epsilon = \\beta_0 + \\beta_1 x_{1} + \\epsilon \\] We could also consider a SLR model with a different predictor, thus altering the form of the model. \\[ y = f(x_1, x_2, x_3, \\ldots, x_{p-1}) + \\epsilon = \\beta_0 + \\beta_2 x_{2} + \\epsilon \\] Often, we’ll use multiple predictors in our model. Very often, we will at least try a model with all possible predictors. \\[ \\begin{aligned} y &amp;= f(x_1, x_2, x_3, \\ldots, x_{p-1}) + \\epsilon \\\\ &amp;= \\beta_0 + \\beta_1 x_{1} + \\beta_2 x_{2} + \\cdots + \\beta_{p-1} x_{p-1} + \\epsilon \\end{aligned} \\] We could also use some, but not all of the predictors. \\[ \\begin{aligned} y &amp;= f(x_1, x_2, x_3, \\ldots, x_{p-1}) + \\epsilon \\\\ &amp;= \\beta_0 + \\beta_1 x_{1} + \\beta_3 x_{3} + \\beta_5 x_{5} + \\epsilon \\end{aligned} \\] These forms are restrictive in two senses. First, they only allow for linear relationships between the response and the predictors. This seems like an obvious restriction of linear models, but in fact, we will soon see how to use linear models for non-linear relationships. (It will involve transforming variables.) Second, how one variable affects the response is the same for any values of the other predictors. Soon we will see how to create models where the effect of \\(x_{1}\\) can be different for different values of \\(x_{2}\\). We will discuss the concept of interaction. 10.1.3 Family A family of models is a broader grouping of many possible forms of a model. For example, above we saw several forms of models from the family of linear models. We will only ever concern ourselves with linear models, which model a response as a linear combination of predictors. There are certainly other families of models. For example, there are several families of non-parametric regression. Smoothing is a broad family of models. As are regression trees. In linear regression, we specified models with parameters, \\(\\beta_j\\) and fit the model by finding the best values of these parameters. This is a parametric approach. A non-parametric approach skips the step of specifying a model with parameters, and is often described as more of an algorithm. Non-parametric models are often used in machine learning. Here, SLR (parametric) is used on the left, while smoothing (non-parametric) is used on the right. SLR finds the best slope and intercept. Smoothing produces the fitted \\(y\\) value at a particular \\(x\\) value by considering the \\(y\\) values of the data in a neighborhood of the \\(x\\) value considered. (Local smoothing.) Why the focus on linear models? Two big reasons: Linear models are the go-to model. Linear models have been around for a long time, and are computationally easy. A linear model may not be the final model you use, but often, it should be the first model you try. The ideas behind linear models can be easily transferred to other modeling techniques. 10.1.4 Assumed Model, Fitted Model When searching for a model, we often need to make assumptions. These assumptions are codified in the family and form of the model. For example \\[ y = \\beta_0 + \\beta_1 x_{1} + \\beta_3 x_{3} + \\beta_5 x_{5} + \\epsilon \\] assumes that \\(y\\) is a linear combination of \\(x_{1}\\), \\(x_{3}\\), and \\(x_{5}\\) as well as some noise. This assumes that the effect of \\(x_{1}\\) on \\(y\\) is \\(\\beta_1\\), which is the same for all values of \\(x_{3}\\) and \\(x_{5}\\). That is, we are using the family of linear models with a particular form. Suppose we then fit this model to some data and obtain the fitted model. For example, in R we would use fit = lm(y ~ x1 + x3 + x5, data = some_data) This is R’s way of saying the family is linear and specifying the form from above. An additive model with the specified predictors as well as an intercept. We then obtain \\[ \\hat{y} = 1.5 + 0.9 x_{1} + 1.1 x_{3} + 2.3 x_{5}. \\] This is our best guess for the function \\(f\\) in \\[ y = f(x_1, x_2, x_3, \\ldots, x_{p-1}) + \\epsilon \\] for the assumed family and form. Fitting a model only gives us the best fit for the family and form that we specify. So the natural question is; how do we choose the correct family and form? We’ll focus on form since we are focusing on the family of linear models. 10.2 Explanation versus Prediction What is the purpose of fitting a model to data? Usually it is to accomplish one of two goals. We can use a model to explain the relationship between the response and the predictors. Models can also be used to predict the response based on the predictors. Often, a good model will do both, but we’ll discuss both goals separately since the processes of finding models for explaining and predicting have some differences. For our purposes, since we are only considering linear models, searching for a good model is essentially searching for a good form of a model. 10.2.1 Explanation If the goal of a model is to explain the relationship between the response and the predictors, we are looking for a model that is small and interpretable, but still fits the data well. When discussing linear models, the size of a model is essentially the number of \\(\\beta\\) parameters used. Suppose we would like to find a model that explains fuel efficiency (mpg) based on a car’s attributes (wt, year, cyl, disp, hp, acc). Perhaps we are a car manufacturer trying to engineer a fuel-efficient vehicle. If this is the case, we are interested in both which predictor variables are useful for explaining the car’s fuel efficiency, as well as how those variables affect fuel efficiency. By understanding this relationship, we can use this knowledge to our advantage when designing a car. To explain a relationship, we are interested in keeping models as small as possible, since smaller models are easy to interpret. The fewer predictors the fewer considerations we need to make in our design process. Note that linear models of any size are rather interpretable to begin with. Later in your data analysis careers, you will see more complicated models that may fit data better, but are much harder, if not impossible to interpret. These models aren’t nearly as useful for explaining a relationship. This is another reason to always attempt a linear model. If it fits as well as more complicated methods, it will be the easiest to understand. To find small and interpretable models, we will eventually use selection procedures, which search among many possible forms of a model. For now we will do this in a more ad-hoc manner using inference techniques we have already encountered. To use inference as we have seen it, we need an additional assumption in addition to the family and form of the model. \\[ y = \\beta_0 + \\beta_1 x_{1} + \\beta_3 x_{3} + \\beta_5 x_{5} + \\epsilon \\] Our additional assumption is about the error term. \\[ \\epsilon \\sim N(0, \\sigma^2) \\] This assumption that the errors are normally distributed with some common variance is the key to all of the inference we have done so far. We will discuss this in great detail later. So with our inference tools (ANOVA and \\(t\\)-test) we have two potential strategies. Start with a very small model (no predictors) and attempt to add predictors. Or, start with a big model (all predictors) and attempt to remove predictors. 10.2.1.1 Correlation and Causation A word of caution when using a model to explain a relationship. There are two terms often used to describe a relationship between two variables: causation and correlation. Correlation is often also referred to as association. Just because two variables are correlated does not necessarily mean that one causes the other. For example, consider modeling mpg as only a function of hp. plot(mpg ~ hp, data = autompg, col = &quot;dodgerblue&quot;, pch = 20, cex = 1.5) Does an increase in horsepower cause a drop in fuel efficiency? Or, perhaps the causality is reversed and an increase in fuel efficiency causes a decrease in horsepower. Or, perhaps there is a third variable that explains both! The issue here is that we have observational data. With observational data, we can only detect associations. To speak with confidence about causality, we would need to run experiments. Often, this decision is made for us, before we ever see data, so we can only modify our interpretation. This is a concept that you should encounter often in your statistics education. For some further reading, and some related fallacies, see: Wikipedia: Correlation does not imply causation. We’ll discuss this further when we discuss experimental design and traditional ANOVA techniques. (All of which has recently been re-branded as A/B testing.) 10.2.2 Prediction If the goal of a model is to predict the response, then the only consideration is how well the model fits the data. For this, we will need a metric. In regression problems, this is most often RMSE. \\[ \\text{RMSE}(\\text{model, data}) = \\sqrt{\\frac{1}{n} \\sum_{i = 1}^{n}(y_i - \\hat{y}_i)^2} \\] where \\(y_i\\) are the actual values of the response for the given data \\(\\hat{y}_i\\) are the predicted values using the fitted model and the predictors from the data Correlation and causation are not an issue here. If a predictor is correlated with the response, it is useful for prediction. For example, in elementary school aged children their shoe size certainly doesn’t cause them to read at a higher level, however we could very easily use shoe size to make a prediction about a child’s reading ability. The larger their shoe size, the better they read. There’s a lurking variable here though, their age! (Don’t send your kids to school with size 14 shoes, it won’t make them read better!) Also, since we are not performing inference, the extra assumption about the errors is not needed. The only thing we care about is how close the fitted model is to the data. Least squares is least squares. For a specified model, it will find the values of the parameters which will minimize the squared error loss. Your results might be largely uninterpretable and useless for inference, but for prediction none of that matters. Suppose instead of the manufacturer who would like to build a car, we are a consumer who wishes to purchase a new car. However this particular car is so new, it has not been rigorously tested, so we are unsure of what fuel efficiency to expect. (And, as skeptics, we don’t trust what the manufacturer is telling us.) In this case, we would like to use the model to help predict the fuel efficiency of this car based on its attributes, which are the predictors of the model. The smaller the errors the model makes, the more confident we are in its prediction. 10.2.2.1 Test-Train Split The trouble with using RMSE to identify how well a model fits data is that RMSE is always (equal or) lower for a larger model. This would suggest that we should always use the largest model possible when looking for a model that predicts well. The problem with this is the potential to overfit to the data. So, we want a model that fits well, but does not overfit. To understand overfitting, we need to think about applying a model to seen and unseen data. Suppose we fit a model using all data available and we evaluate RMSE on this fitted model and all of the seen data. We will call this data the training data, and this RMSE the train RMSE. Now, suppose we magically encounter some additional data. To truly assess how well the model predicts, we should evaluate how well our model predicts the response of this data. We will call this data the test data and this RMSE the test RMSE. Train RMSE: model fit on seen data, evaluated on seen data Test RMSE: model fit on seen data, evaluated on unseen data Below, we simulate some data and fit two models. We will call the solid blue line the “simple” model. The dashed orange line will be called the “complex” model, which was fit with methods we do not yet know. The left panel shows the data that was used to fit the two models. Clearly the “complex” model fits the data much better. The right panel shows additional data that was simulated in the same manner as the original data. Here we see that the “simple” model fits much better. The dashed orange line almost seems random. Model Train RMSE Test RMSE Simple 1.71 1.45 Complex 1.41 2.07 The more “complex,” wiggly, model fits the training data much better as it has a much lower train RMSE. However, we see that the “simple” model fits the test data much better, with a much lower test RMSE. This means that the complex model has overfit to the data, and we prefer the simple model. When choosing a model for prediction, we prefer a model that predicts unseen data. In practice, you can’t simply generate more data to evaluate your models. Instead we split existing data into data used to fit the model (train) and data used to evaluate the model (test). Never fit a model with test data. 10.3 Summary Models can be used to explain relationships and predict observations. When using model to, explain; we prefer small and interpretable models. predict; we prefer models that make the smallest errors possible, without overfitting. Linear models can accomplish both these goals. Later, we will see that often a linear model that accomplishes one of these goals, usually accomplishes the other. 10.4 R Markdown The R Markdown file for this chapter can be found here: model-building.Rmd The file was created using R version 4.3.2. "],["categorical-predictors-and-interactions.html", "Chapter 11 Categorical Predictors and Interactions 11.1 Dummy Variables 11.2 Interactions 11.3 Factor Variables 11.4 Parameterization 11.5 Building Larger Models 11.6 R Markdown", " Chapter 11 Categorical Predictors and Interactions “The greatest value of a picture is when it forces us to notice what we never expected to see.” — John Tukey After reading this chapter you will be able to: Include and interpret categorical variables in a linear regression model by way of dummy variables. Understand the implications of using a model with a categorical variable in two ways: levels serving as unique predictors versus levels serving as a comparison to a baseline. Construct and interpret linear regression models with interaction terms. Identify categorical variables in a data set and convert them into factor variables, if necessary, using R. So far in each of our analyses, we have only used numeric variables as predictors. We have also only used additive models, meaning the effect any predictor had on the response was not dependent on the other predictors. In this chapter, we will remove both of these restrictions. We will fit models with categorical predictors, and use models that allow predictors to interact. The mathematics of multiple regression will remain largely unchanging, however, we will pay close attention to interpretation, as well as some difference in R usage. 11.1 Dummy Variables For this chapter, we will briefly use the built-in dataset mtcars before returning to our autompg dataset that we created in the last chapter. The mtcars dataset is somewhat smaller, so we’ll quickly take a look at the entire dataset. mtcars ## mpg cyl disp hp drat wt qsec vs am gear carb ## Mazda RX4 21.0 6 160.0 110 3.90 2.620 16.46 0 1 4 4 ## Mazda RX4 Wag 21.0 6 160.0 110 3.90 2.875 17.02 0 1 4 4 ## Datsun 710 22.8 4 108.0 93 3.85 2.320 18.61 1 1 4 1 ## Hornet 4 Drive 21.4 6 258.0 110 3.08 3.215 19.44 1 0 3 1 ## Hornet Sportabout 18.7 8 360.0 175 3.15 3.440 17.02 0 0 3 2 ## Valiant 18.1 6 225.0 105 2.76 3.460 20.22 1 0 3 1 ## Duster 360 14.3 8 360.0 245 3.21 3.570 15.84 0 0 3 4 ## Merc 240D 24.4 4 146.7 62 3.69 3.190 20.00 1 0 4 2 ## Merc 230 22.8 4 140.8 95 3.92 3.150 22.90 1 0 4 2 ## Merc 280 19.2 6 167.6 123 3.92 3.440 18.30 1 0 4 4 ## Merc 280C 17.8 6 167.6 123 3.92 3.440 18.90 1 0 4 4 ## Merc 450SE 16.4 8 275.8 180 3.07 4.070 17.40 0 0 3 3 ## Merc 450SL 17.3 8 275.8 180 3.07 3.730 17.60 0 0 3 3 ## Merc 450SLC 15.2 8 275.8 180 3.07 3.780 18.00 0 0 3 3 ## Cadillac Fleetwood 10.4 8 472.0 205 2.93 5.250 17.98 0 0 3 4 ## Lincoln Continental 10.4 8 460.0 215 3.00 5.424 17.82 0 0 3 4 ## Chrysler Imperial 14.7 8 440.0 230 3.23 5.345 17.42 0 0 3 4 ## Fiat 128 32.4 4 78.7 66 4.08 2.200 19.47 1 1 4 1 ## Honda Civic 30.4 4 75.7 52 4.93 1.615 18.52 1 1 4 2 ## Toyota Corolla 33.9 4 71.1 65 4.22 1.835 19.90 1 1 4 1 ## Toyota Corona 21.5 4 120.1 97 3.70 2.465 20.01 1 0 3 1 ## Dodge Challenger 15.5 8 318.0 150 2.76 3.520 16.87 0 0 3 2 ## AMC Javelin 15.2 8 304.0 150 3.15 3.435 17.30 0 0 3 2 ## Camaro Z28 13.3 8 350.0 245 3.73 3.840 15.41 0 0 3 4 ## Pontiac Firebird 19.2 8 400.0 175 3.08 3.845 17.05 0 0 3 2 ## Fiat X1-9 27.3 4 79.0 66 4.08 1.935 18.90 1 1 4 1 ## Porsche 914-2 26.0 4 120.3 91 4.43 2.140 16.70 0 1 5 2 ## Lotus Europa 30.4 4 95.1 113 3.77 1.513 16.90 1 1 5 2 ## Ford Pantera L 15.8 8 351.0 264 4.22 3.170 14.50 0 1 5 4 ## Ferrari Dino 19.7 6 145.0 175 3.62 2.770 15.50 0 1 5 6 ## Maserati Bora 15.0 8 301.0 335 3.54 3.570 14.60 0 1 5 8 ## Volvo 142E 21.4 4 121.0 109 4.11 2.780 18.60 1 1 4 2 We will be interested in three of the variables: mpg, hp, and am. mpg: fuel efficiency, in miles per gallon. hp: horsepower, in foot-pounds per second. am: transmission. Automatic or manual. As we often do, we will start by plotting the data. We are interested in mpg as the response variable, and hp as a predictor. plot(mpg ~ hp, data = mtcars, cex = 2) Since we are also interested in the transmission type, we could also label the points accordingly. plot(mpg ~ hp, data = mtcars, col = am + 1, pch = am + 1, cex = 2) legend(&quot;topright&quot;, c(&quot;Automatic&quot;, &quot;Manual&quot;), col = c(1, 2), pch = c(1, 2)) We used a common R “trick” when plotting this data. The am variable takes two possible values; 0 for automatic transmission, and 1 for manual transmissions. R can use numbers to represent colors, however the color for 0 is white. So we take the am vector and add 1 to it. Then observations with automatic transmissions are now represented by 1, which is black in R, and manual transmission are represented by 2, which is red in R. (Note, we are only adding 1 inside the call to plot(), we are not actually modifying the values stored in am.) We now fit the SLR model \\[ Y = \\beta_0 + \\beta_1 x_1 + \\epsilon, \\] where \\(Y\\) is mpg and \\(x_1\\) is hp. For notational brevity, we drop the index \\(i\\) for observations. mpg_hp_slr = lm(mpg ~ hp, data = mtcars) We then re-plot the data and add the fitted line to the plot. plot(mpg ~ hp, data = mtcars, col = am + 1, pch = am + 1, cex = 2) abline(mpg_hp_slr, lwd = 3, col = &quot;grey&quot;) legend(&quot;topright&quot;, c(&quot;Automatic&quot;, &quot;Manual&quot;), col = c(1, 2), pch = c(1, 2)) We should notice a pattern here. The red, manual observations largely fall above the line, while the black, automatic observations are mostly below the line. This means our model underestimates the fuel efficiency of manual transmissions, and overestimates the fuel efficiency of automatic transmissions. To correct for this, we will add a predictor to our model, namely, am as \\(x_2\\). Our new model is \\[ Y = \\beta_0 + \\beta_1 x_1 + \\beta_2 x_2 + \\epsilon, \\] where \\(x_1\\) and \\(Y\\) remain the same, but now \\[ x_2 = \\begin{cases} 1 &amp; \\text{manual transmission} \\\\ 0 &amp; \\text{automatic transmission} \\end{cases}. \\] In this case, we call \\(x_2\\) a dummy variable. A dummy variable is somewhat unfortunately named, as it is in no way “dumb.” In fact, it is actually somewhat clever. A dummy variable is a numerical variable that is used in a regression analysis to “code” for a binary categorical variable. Let’s see how this works. First, note that am is already a dummy variable, since it uses the values 0 and 1 to represent automatic and manual transmissions. Often, a variable like am would store the character values auto and man and we would either have to convert these to 0 and 1, or, as we will see later, R will take care of creating dummy variables for us. So, to fit the above model, we do so like any other multiple regression model we have seen before. mpg_hp_add = lm(mpg ~ hp + am, data = mtcars) Briefly checking the output, we see that R has estimated the three \\(\\beta\\) parameters. mpg_hp_add ## ## Call: ## lm(formula = mpg ~ hp + am, data = mtcars) ## ## Coefficients: ## (Intercept) hp am ## 26.58491 -0.05889 5.27709 Since \\(x_2\\) can only take values 0 and 1, we can effectively write two different models, one for manual and one for automatic transmissions. For automatic transmissions, that is \\(x_2 = 0\\), we have, \\[ Y = \\beta_0 + \\beta_1 x_1 + \\epsilon. \\] Then for manual transmissions, that is \\(x_2 = 1\\), we have, \\[ Y = (\\beta_0 + \\beta_2) + \\beta_1 x_1 + \\epsilon. \\] Notice that these models share the same slope, \\(\\beta_1\\), but have different intercepts, differing by \\(\\beta_2\\). So the change in mpg is the same for both models, but on average mpg differs by \\(\\beta_2\\) between the two transmission types. We’ll now calculate the estimated slope and intercept of these two models so that we can add them to a plot. Note that: \\(\\hat{\\beta}_0\\) = coef(mpg_hp_add)[1] = 26.5849137 \\(\\hat{\\beta}_1\\) = coef(mpg_hp_add)[2] = -0.0588878 \\(\\hat{\\beta}_2\\) = coef(mpg_hp_add)[3] = 5.2770853 We can then combine these to calculate the estimated slope and intercepts. int_auto = coef(mpg_hp_add)[1] int_manu = coef(mpg_hp_add)[1] + coef(mpg_hp_add)[3] slope_auto = coef(mpg_hp_add)[2] slope_manu = coef(mpg_hp_add)[2] Re-plotting the data, we use these slopes and intercepts to add the “two” fitted models to the plot. plot(mpg ~ hp, data = mtcars, col = am + 1, pch = am + 1, cex = 2) abline(int_auto, slope_auto, col = 1, lty = 1, lwd = 2) # add line for auto abline(int_manu, slope_manu, col = 2, lty = 2, lwd = 2) # add line for manual legend(&quot;topright&quot;, c(&quot;Automatic&quot;, &quot;Manual&quot;), col = c(1, 2), pch = c(1, 2)) We notice right away that the points are no longer systematically incorrect. The red, manual observations vary about the red line in no particular pattern without underestimating the observations as before. The black, automatic points vary about the black line, also without an obvious pattern. They say a picture is worth a thousand words, but as a statistician, sometimes a picture is worth an entire analysis. The above picture makes it plainly obvious that \\(\\beta_2\\) is significant, but let’s verify mathematically. Essentially we would like to test: \\[ H_0: \\beta_2 = 0 \\quad \\text{vs} \\quad H_1: \\beta_2 \\neq 0. \\] This is nothing new. Again, the math is the same as the multiple regression analyses we have seen before. We could perform either a \\(t\\) or \\(F\\) test here. The only difference is a slight change in interpretation. We could think of this as testing a model with a single line (\\(H_0\\)) against a model that allows two lines (\\(H_1\\)). To obtain the test statistic and p-value for the \\(t\\)-test, we would use summary(mpg_hp_add)$coefficients[&quot;am&quot;,] ## Estimate Std. Error t value Pr(&gt;|t|) ## 5.277085e+00 1.079541e+00 4.888270e+00 3.460318e-05 To do the same for the \\(F\\) test, we would use anova(mpg_hp_slr, mpg_hp_add) ## Analysis of Variance Table ## ## Model 1: mpg ~ hp ## Model 2: mpg ~ hp + am ## Res.Df RSS Df Sum of Sq F Pr(&gt;F) ## 1 30 447.67 ## 2 29 245.44 1 202.24 23.895 3.46e-05 *** ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 Notice that these are indeed testing the same thing, as the p-values are exactly equal. (And the \\(F\\) test statistic is the \\(t\\) test statistic squared.) Recapping some interpretations: \\(\\hat{\\beta}_0 = 26.5849137\\) is the estimated average mpg for a car with an automatic transmission and 0 hp. \\(\\hat{\\beta}_0 + \\hat{\\beta}_2 = 31.8619991\\) is the estimated average mpg for a car with a manual transmission and 0 hp. \\(\\hat{\\beta}_2 = 5.2770853\\) is the estimated difference in average mpg for cars with manual transmissions as compared to those with automatic transmission, for any hp. \\(\\hat{\\beta}_1 = -0.0588878\\) is the estimated change in average mpg for an increase in one hp, for either transmission type. We should take special notice of those last two. In the model, \\[ Y = \\beta_0 + \\beta_1 x_1 + \\beta_2 x_2 + \\epsilon, \\] we see \\(\\beta_1\\) is the average change in \\(Y\\) for an increase in \\(x_1\\), no matter the value of \\(x_2\\). Also, \\(\\beta_2\\) is always the difference in the average of \\(Y\\) for any value of \\(x_1\\). These are two restrictions we won’t always want, so we need a way to specify a more flexible model. Here we restricted ourselves to a single numerical predictor \\(x_1\\) and one dummy variable \\(x_2\\). However, the concept of a dummy variable can be used with larger multiple regression models. We only use a single numerical predictor here for ease of visualization since we can think of the “two lines” interpretation. But in general, we can think of a dummy variable as creating “two models,” one for each category of a binary categorical variable. 11.2 Interactions To remove the “same slope” restriction, we will now discuss interaction. To illustrate this concept, we will return to the autompg dataset we created in the last chapter, with a few more modifications. # read data frame from the web autompg = read.table( &quot;http://archive.ics.uci.edu/ml/machine-learning-databases/auto-mpg/auto-mpg.data&quot;, quote = &quot;\\&quot;&quot;, comment.char = &quot;&quot;, stringsAsFactors = FALSE) # give the dataframe headers colnames(autompg) = c(&quot;mpg&quot;, &quot;cyl&quot;, &quot;disp&quot;, &quot;hp&quot;, &quot;wt&quot;, &quot;acc&quot;, &quot;year&quot;, &quot;origin&quot;, &quot;name&quot;) # remove missing data, which is stored as &quot;?&quot; autompg = subset(autompg, autompg$hp != &quot;?&quot;) # remove the plymouth reliant, as it causes some issues autompg = subset(autompg, autompg$name != &quot;plymouth reliant&quot;) # give the dataset row names, based on the engine, year and name rownames(autompg) = paste(autompg$cyl, &quot;cylinder&quot;, autompg$year, autompg$name) # remove the variable for name autompg = subset(autompg, select = c(&quot;mpg&quot;, &quot;cyl&quot;, &quot;disp&quot;, &quot;hp&quot;, &quot;wt&quot;, &quot;acc&quot;, &quot;year&quot;, &quot;origin&quot;)) # change horsepower from character to numeric autompg$hp = as.numeric(autompg$hp) # create a dummy variable for foreign vs domestic cars. domestic = 1. autompg$domestic = as.numeric(autompg$origin == 1) # remove 3 and 5 cylinder cars (which are very rare.) autompg = autompg[autompg$cyl != 5,] autompg = autompg[autompg$cyl != 3,] # the following line would verify the remaining cylinder possibilities are 4, 6, 8 #unique(autompg$cyl) # change cyl to a factor variable autompg$cyl = as.factor(autompg$cyl) str(autompg) ## &#39;data.frame&#39;: 383 obs. of 9 variables: ## $ mpg : num 18 15 18 16 17 15 14 14 14 15 ... ## $ cyl : Factor w/ 3 levels &quot;4&quot;,&quot;6&quot;,&quot;8&quot;: 3 3 3 3 3 3 3 3 3 3 ... ## $ disp : num 307 350 318 304 302 429 454 440 455 390 ... ## $ hp : num 130 165 150 150 140 198 220 215 225 190 ... ## $ wt : num 3504 3693 3436 3433 3449 ... ## $ acc : num 12 11.5 11 12 10.5 10 9 8.5 10 8.5 ... ## $ year : int 70 70 70 70 70 70 70 70 70 70 ... ## $ origin : int 1 1 1 1 1 1 1 1 1 1 ... ## $ domestic: num 1 1 1 1 1 1 1 1 1 1 ... We’ve removed cars with 3 and 5 cylinders , as well as created a new variable domestic which indicates whether or not a car was built in the United States. Removing the 3 and 5 cylinders is simply for ease of demonstration later in the chapter and would not be done in practice. The new variable domestic takes the value 1 if the car was built in the United States, and 0 otherwise, which we will refer to as “foreign.” (We are arbitrarily using the United States as the reference point here.) We have also made cyl and origin into factor variables, which we will discuss later. We’ll now be concerned with three variables: mpg, disp, and domestic. We will use mpg as the response. We can fit a model, \\[ Y = \\beta_0 + \\beta_1 x_1 + \\beta_2 x_2 + \\epsilon, \\] where \\(Y\\) is mpg, the fuel efficiency in miles per gallon, \\(x_1\\) is disp, the displacement in cubic inches, \\(x_2\\) is domestic as described above, which is a dummy variable. \\[ x_2 = \\begin{cases} 1 &amp; \\text{Domestic} \\\\ 0 &amp; \\text{Foreign} \\end{cases} \\] We will fit this model, extract the slope and intercept for the “two lines,” plot the data and add the lines. mpg_disp_add = lm(mpg ~ disp + domestic, data = autompg) int_for = coef(mpg_disp_add)[1] int_dom = coef(mpg_disp_add)[1] + coef(mpg_disp_add)[3] slope_for = coef(mpg_disp_add)[2] slope_dom = coef(mpg_disp_add)[2] plot(mpg ~ disp, data = autompg, col = domestic + 1, pch = domestic + 1) abline(int_for, slope_for, col = 1, lty = 1, lwd = 2) # add line for foreign cars abline(int_dom, slope_dom, col = 2, lty = 2, lwd = 2) # add line for domestic cars legend(&quot;topright&quot;, c(&quot;Foreign&quot;, &quot;Domestic&quot;), pch = c(1, 2), col = c(1, 2)) This is a model that allows for two parallel lines, meaning the mpg can be different on average between foreign and domestic cars of the same engine displacement, but the change in average mpg for an increase in displacement is the same for both. We can see this model isn’t doing very well here. The red line fits the red points fairly well, but the black line isn’t doing very well for the black points, it should clearly have a more negative slope. Essentially, we would like a model that allows for two different slopes. Consider the following model, \\[ Y = \\beta_0 + \\beta_1 x_1 + \\beta_2 x_2 + \\beta_3 x_1 x_2 + \\epsilon, \\] where \\(x_1\\), \\(x_2\\), and \\(Y\\) are the same as before, but we have added a new interaction term \\(x_1 x_2\\) which multiplies \\(x_1\\) and \\(x_2\\), so we also have an additional \\(\\beta\\) parameter \\(\\beta_3\\). This model essentially creates two slopes and two intercepts, \\(\\beta_2\\) being the difference in intercepts and \\(\\beta_3\\) being the difference in slopes. To see this, we will break down the model into the two “sub-models” for foreign and domestic cars. For foreign cars, that is \\(x_2 = 0\\), we have \\[ Y = \\beta_0 + \\beta_1 x_1 + \\epsilon. \\] For domestic cars, that is \\(x_2 = 1\\), we have \\[ Y = (\\beta_0 + \\beta_2) + (\\beta_1 + \\beta_3) x_1 + \\epsilon. \\] These two models have both different slopes and intercepts. \\(\\beta_0\\) is the average mpg for a foreign car with 0 disp. \\(\\beta_1\\) is the change in average mpg for an increase of one disp, for foreign cars. \\(\\beta_0 + \\beta_2\\) is the average mpg for a domestic car with 0 disp. \\(\\beta_1 + \\beta_3\\) is the change in average mpg for an increase of one disp, for domestic cars. How do we fit this model in R? There are a number of ways. One method would be to simply create a new variable, then fit a model like any other. autompg$x3 = autompg$disp * autompg$domestic # THIS CODE NOT RUN! do_not_do_this = lm(mpg ~ disp + domestic + x3, data = autompg) # THIS CODE NOT RUN! You should only do this as a last resort. We greatly prefer not to have to modify our data simply to fit a model. Instead, we can tell R we would like to use the existing data with an interaction term, which it will create automatically when we use the : operator. mpg_disp_int = lm(mpg ~ disp + domestic + disp:domestic, data = autompg) An alternative method, which will fit the exact same model as above would be to use the * operator. This method automatically creates the interaction term, as well as any “lower order terms,” which in this case are the first order terms for disp and domestic mpg_disp_int2 = lm(mpg ~ disp * domestic, data = autompg) We can quickly verify that these are doing the same thing. coef(mpg_disp_int) ## (Intercept) disp domestic disp:domestic ## 46.0548423 -0.1569239 -12.5754714 0.1025184 coef(mpg_disp_int2) ## (Intercept) disp domestic disp:domestic ## 46.0548423 -0.1569239 -12.5754714 0.1025184 We see that both the variables, and their coefficient estimates are indeed the same for both models. summary(mpg_disp_int) ## ## Call: ## lm(formula = mpg ~ disp + domestic + disp:domestic, data = autompg) ## ## Residuals: ## Min 1Q Median 3Q Max ## -10.8332 -2.8956 -0.8332 2.2828 18.7749 ## ## Coefficients: ## Estimate Std. Error t value Pr(&gt;|t|) ## (Intercept) 46.05484 1.80582 25.504 &lt; 2e-16 *** ## disp -0.15692 0.01668 -9.407 &lt; 2e-16 *** ## domestic -12.57547 1.95644 -6.428 3.90e-10 *** ## disp:domestic 0.10252 0.01692 6.060 3.29e-09 *** ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 ## ## Residual standard error: 4.308 on 379 degrees of freedom ## Multiple R-squared: 0.7011, Adjusted R-squared: 0.6987 ## F-statistic: 296.3 on 3 and 379 DF, p-value: &lt; 2.2e-16 We see that using summary() gives the usual output for a multiple regression model. We pay close attention to the row for disp:domestic which tests, \\[ H_0: \\beta_3 = 0. \\] In this case, testing for \\(\\beta_3 = 0\\) is testing for two lines with parallel slopes versus two lines with possibly different slopes. The disp:domestic line in the summary() output uses a \\(t\\)-test to perform the test. We could also use an ANOVA \\(F\\)-test. The additive model without interaction is our null model, and the interaction model is the alternative. anova(mpg_disp_add, mpg_disp_int) ## Analysis of Variance Table ## ## Model 1: mpg ~ disp + domestic ## Model 2: mpg ~ disp + domestic + disp:domestic ## Res.Df RSS Df Sum of Sq F Pr(&gt;F) ## 1 380 7714.0 ## 2 379 7032.6 1 681.36 36.719 3.294e-09 *** ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 Again we see this test has the same p-value as the \\(t\\)-test. Also the p-value is extremely low, so between the two, we choose the interaction model. int_for = coef(mpg_disp_int)[1] int_dom = coef(mpg_disp_int)[1] + coef(mpg_disp_int)[3] slope_for = coef(mpg_disp_int)[2] slope_dom = coef(mpg_disp_int)[2] + coef(mpg_disp_int)[4] Here we again calculate the slope and intercepts for the two lines for use in plotting. plot(mpg ~ disp, data = autompg, col = domestic + 1, pch = domestic + 1) abline(int_for, slope_for, col = 1, lty = 1, lwd = 2) # line for foreign cars abline(int_dom, slope_dom, col = 2, lty = 2, lwd = 2) # line for domestic cars legend(&quot;topright&quot;, c(&quot;Foreign&quot;, &quot;Domestic&quot;), pch = c(1, 2), col = c(1, 2)) We see that these lines fit the data much better, which matches the result of our tests. So far we have only seen interaction between a categorical variable (domestic) and a numerical variable (disp). While this is easy to visualize, since it allows for different slopes for two lines, it is not the only type of interaction we can use in a model. We can also consider interactions between two numerical variables. Consider the model, \\[ Y = \\beta_0 + \\beta_1 x_1 + \\beta_2 x_2 + \\beta_3 x_1 x_2 + \\epsilon, \\] where \\(Y\\) is mpg, the fuel efficiency in miles per gallon, \\(x_1\\) is disp, the displacement in cubic inches, \\(x_2\\) is hp, the horsepower, in foot-pounds per second. How does mpg change based on disp in this model? We can rearrange some terms to see how. \\[ Y = \\beta_0 + (\\beta_1 + \\beta_3 x_2) x_1 + \\beta_2 x_2 + \\epsilon \\] So, for a one unit increase in \\(x_1\\) (disp), the mean of \\(Y\\) (mpg) increases \\(\\beta_1 + \\beta_3 x_2\\), which is a different value depending on the value of \\(x_2\\) (hp)! Since we’re now working in three dimensions, this model can’t be easily justified via visualizations like the previous example. Instead, we will have to rely on a test. mpg_disp_add_hp = lm(mpg ~ disp + hp, data = autompg) mpg_disp_int_hp = lm(mpg ~ disp * hp, data = autompg) summary(mpg_disp_int_hp) ## ## Call: ## lm(formula = mpg ~ disp * hp, data = autompg) ## ## Residuals: ## Min 1Q Median 3Q Max ## -10.7849 -2.3104 -0.5699 2.1453 17.9211 ## ## Coefficients: ## Estimate Std. Error t value Pr(&gt;|t|) ## (Intercept) 5.241e+01 1.523e+00 34.42 &lt;2e-16 *** ## disp -1.002e-01 6.638e-03 -15.09 &lt;2e-16 *** ## hp -2.198e-01 1.987e-02 -11.06 &lt;2e-16 *** ## disp:hp 5.658e-04 5.165e-05 10.96 &lt;2e-16 *** ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 ## ## Residual standard error: 3.896 on 379 degrees of freedom ## Multiple R-squared: 0.7554, Adjusted R-squared: 0.7535 ## F-statistic: 390.2 on 3 and 379 DF, p-value: &lt; 2.2e-16 Using summary() we focus on the row for disp:hp which tests, \\[ H_0: \\beta_3 = 0. \\] Again, we see a very low p-value so we reject the null (additive model) in favor of the interaction model. Again, there is an equivalent \\(F\\)-test. anova(mpg_disp_add_hp, mpg_disp_int_hp) ## Analysis of Variance Table ## ## Model 1: mpg ~ disp + hp ## Model 2: mpg ~ disp * hp ## Res.Df RSS Df Sum of Sq F Pr(&gt;F) ## 1 380 7576.6 ## 2 379 5754.2 1 1822.3 120.03 &lt; 2.2e-16 *** ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 We can take a closer look at the coefficients of our fitted interaction model. coef(mpg_disp_int_hp) ## (Intercept) disp hp disp:hp ## 52.4081997848 -0.1001737655 -0.2198199720 0.0005658269 \\(\\hat{\\beta}_0 = 52.4081998\\) is the estimated average mpg for a car with 0 disp and 0 hp. \\(\\hat{\\beta}_1 = -0.1001738\\) is the estimated change in average mpg for an increase in 1 disp, for a car with 0 hp. \\(\\hat{\\beta}_2 = -0.21982\\) is the estimated change in average mpg for an increase in 1 hp, for a car with 0 disp. \\(\\hat{\\beta}_3 = 5.658269\\times 10^{-4}\\) is an estimate of the modification to the change in average mpg for an increase in disp, for a car of a certain hp (or vice versa). That last coefficient needs further explanation. Recall the rearrangement we made earlier \\[ Y = \\beta_0 + (\\beta_1 + \\beta_3 x_2) x_1 + \\beta_2 x_2 + \\epsilon. \\] So, our estimate for \\(\\beta_1 + \\beta_3 x_2\\), is \\(\\hat{\\beta}_1 + \\hat{\\beta}_3 x_2\\), which in this case is \\[ -0.1001738 + 5.658269\\times 10^{-4} x_2. \\] This says that, for an increase of one disp we see an estimated change in average mpg of \\(-0.1001738 + 5.658269\\times 10^{-4} x_2\\). So how disp and mpg are related, depends on the hp of the car. So for a car with 50 hp, the estimated change in average mpg for an increase of one disp is \\[ -0.1001738 + 5.658269\\times 10^{-4} \\cdot 50 = -0.0718824 \\] And for a car with 350 hp, the estimated change in average mpg for an increase of one disp is \\[ -0.1001738 + 5.658269\\times 10^{-4} \\cdot 350 = 0.0978657 \\] Notice the sign changed! 11.3 Factor Variables So far in this chapter, we have limited our use of categorical variables to binary categorical variables. Specifically, we have limited ourselves to dummy variables which take a value of 0 or 1 and represent a categorical variable numerically. We will now discuss factor variables, which is a special way that R deals with categorical variables. With factor variables, a human user can simply think about the categories of a variable, and R will take care of the necessary dummy variables without any 0/1 assignment being done by the user. is.factor(autompg$domestic) ## [1] FALSE Earlier when we used the domestic variable, it was not a factor variable. It was simply a numerical variable that only took two possible values, 1 for domestic, and 0 for foreign. Let’s create a new variable origin that stores the same information, but in a different way. autompg$origin[autompg$domestic == 1] = &quot;domestic&quot; autompg$origin[autompg$domestic == 0] = &quot;foreign&quot; head(autompg$origin) ## [1] &quot;domestic&quot; &quot;domestic&quot; &quot;domestic&quot; &quot;domestic&quot; &quot;domestic&quot; &quot;domestic&quot; Now the origin variable stores \"domestic\" for domestic cars and \"foreign\" for foreign cars. is.factor(autompg$origin) ## [1] FALSE However, this is simply a vector of character values. A vector of car models is a character variable in R. A vector of Vehicle Identification Numbers (VINs) is a character variable as well. But those don’t represent a short list of levels that might influence a response variable. We will want to coerce this origin variable to be something more: a factor variable. autompg$origin = as.factor(autompg$origin) Now when we check the structure of the autompg dataset, we see that origin is a factor variable. str(autompg) ## &#39;data.frame&#39;: 383 obs. of 9 variables: ## $ mpg : num 18 15 18 16 17 15 14 14 14 15 ... ## $ cyl : Factor w/ 3 levels &quot;4&quot;,&quot;6&quot;,&quot;8&quot;: 3 3 3 3 3 3 3 3 3 3 ... ## $ disp : num 307 350 318 304 302 429 454 440 455 390 ... ## $ hp : num 130 165 150 150 140 198 220 215 225 190 ... ## $ wt : num 3504 3693 3436 3433 3449 ... ## $ acc : num 12 11.5 11 12 10.5 10 9 8.5 10 8.5 ... ## $ year : int 70 70 70 70 70 70 70 70 70 70 ... ## $ origin : Factor w/ 2 levels &quot;domestic&quot;,&quot;foreign&quot;: 1 1 1 1 1 1 1 1 1 1 ... ## $ domestic: num 1 1 1 1 1 1 1 1 1 1 ... Factor variables have levels which are the possible values (categories) that the variable may take, in this case foreign or domestic. levels(autompg$origin) ## [1] &quot;domestic&quot; &quot;foreign&quot; Recall that previously we have fit the model \\[ Y = \\beta_0 + \\beta_1 x_1 + \\beta_2 x_2 + \\beta_3 x_1 x_2 + \\epsilon, \\] where \\(Y\\) is mpg, the fuel efficiency in miles per gallon, \\(x_1\\) is disp, the displacement in cubic inches, \\(x_2\\) is domestic, a dummy variable where 1 indicates a domestic car. (mod_dummy = lm(mpg ~ disp * domestic, data = autompg)) ## ## Call: ## lm(formula = mpg ~ disp * domestic, data = autompg) ## ## Coefficients: ## (Intercept) disp domestic disp:domestic ## 46.0548 -0.1569 -12.5755 0.1025 So here we see that \\[ \\hat{\\beta}_0 + \\hat{\\beta}_2 = 46.0548423 + -12.5754714 = 33.4793709 \\] is the estimated average mpg for a domestic car with 0 disp. Now let’s try to do the same, but using our new factor variable. (mod_factor = lm(mpg ~ disp * origin, data = autompg)) ## ## Call: ## lm(formula = mpg ~ disp * origin, data = autompg) ## ## Coefficients: ## (Intercept) disp originforeign disp:originforeign ## 33.47937 -0.05441 12.57547 -0.10252 It seems that it doesn’t produce the same results. Right away we notice that the intercept is different, as is the coefficient in front of disp. We also notice that the remaining two coefficients are of the same magnitude as their respective counterparts using the domestic variable, but with a different sign. Why is this happening? It turns out that by using a factor variable, R is automatically creating a dummy variable for us. However, it is not the dummy variable that we had originally used ourselves. R is fitting the model \\[ Y = \\beta_0 + \\beta_1 x_1 + \\beta_2 x_2 + \\beta_3 x_1 x_2 + \\epsilon, \\] where \\(Y\\) is mpg, the fuel efficiency in miles per gallon, \\(x_1\\) is disp, the displacement in cubic inches, \\(x_2\\) is a dummy variable created by R. It uses 1 to represent a foreign car. So now, \\[ \\hat{\\beta}_0 = 33.4793709 \\] is the estimated average mpg for a domestic car with 0 disp, which is indeed the same as before. When R created \\(x_2\\), the dummy variable, it used domestic cars as the reference level, that is the default value of the factor variable. So when the dummy variable is 0, the model represents this reference level, which is domestic. (R makes this choice because domestic comes before foreign alphabetically.) So the two models have different estimated coefficients, but due to the different model representations, they are actually the same model. 11.3.1 Factors with More Than Two Levels Let’s now consider a factor variable with more than two levels. In this dataset, cyl is an example. is.factor(autompg$cyl) ## [1] TRUE levels(autompg$cyl) ## [1] &quot;4&quot; &quot;6&quot; &quot;8&quot; Here the cyl variable has three possible levels: 4, 6, and 8. You may wonder, why not simply use cyl as a numerical variable? You certainly could. However, that would force the difference in average mpg between 4 and 6 cylinders to be the same as the difference in average mpg between 6 and 8 cylinders. That usually makes sense for a continuous variable, but not for a discrete variable with so few possible values. In the case of this variable, there is no such thing as a 7-cylinder engine or a 6.23-cylinder engine in personal vehicles. For these reasons, we will simply consider cyl to be categorical. This is a decision that will commonly need to be made with ordinal variables. Often, with a large number of categories, the decision to treat them as numerical variables is appropriate because, otherwise, a large number of dummy variables are then needed to represent these variables. Let’s define three dummy variables related to the cyl factor variable. \\[ v_1 = \\begin{cases} 1 &amp; \\text{4 cylinder} \\\\ 0 &amp; \\text{not 4 cylinder} \\end{cases} \\] \\[ v_2 = \\begin{cases} 1 &amp; \\text{6 cylinder} \\\\ 0 &amp; \\text{not 6 cylinder} \\end{cases} \\] \\[ v_3 = \\begin{cases} 1 &amp; \\text{8 cylinder} \\\\ 0 &amp; \\text{not 8 cylinder} \\end{cases} \\] Now, let’s fit an additive model in R, using mpg as the response, and disp and cyl as predictors. This should be a model that uses “three regression lines” to model mpg, one for each of the possible cyl levels. They will all have the same slope (since it is an additive model), but each will have its own intercept. (mpg_disp_add_cyl = lm(mpg ~ disp + cyl, data = autompg)) ## ## Call: ## lm(formula = mpg ~ disp + cyl, data = autompg) ## ## Coefficients: ## (Intercept) disp cyl6 cyl8 ## 34.99929 -0.05217 -3.63325 -2.03603 The question is, what is the model that R has fit here? It has chosen to use the model \\[ Y = \\beta_0 + \\beta_1 x + \\beta_2 v_2 + \\beta_3 v_3 + \\epsilon, \\] where \\(Y\\) is mpg, the fuel efficiency in miles per gallon, \\(x\\) is disp, the displacement in cubic inches, \\(v_2\\) and \\(v_3\\) are the dummy variables defined above. Why doesn’t R use \\(v_1\\)? Essentially because it doesn’t need to. To create three lines, it only needs two dummy variables since it is using a reference level, which in this case is a 4 cylinder car. The three “sub models” are then: 4 Cylinder: \\(Y = \\beta_0 + \\beta_1 x + \\epsilon\\) 6 Cylinder: \\(Y = (\\beta_0 + \\beta_2) + \\beta_1 x + \\epsilon\\) 8 Cylinder: \\(Y = (\\beta_0 + \\beta_3) + \\beta_1 x + \\epsilon\\) Notice that they all have the same slope. However, using the two dummy variables, we achieve the three intercepts. \\(\\beta_0\\) is the average mpg for a 4 cylinder car with 0 disp. \\(\\beta_0 + \\beta_2\\) is the average mpg for a 6 cylinder car with 0 disp. \\(\\beta_0 + \\beta_3\\) is the average mpg for a 8 cylinder car with 0 disp. So because 4 cylinder is the reference level, \\(\\beta_0\\) is specific to 4 cylinders, but \\(\\beta_2\\) and \\(\\beta_3\\) are used to represent quantities relative to 4 cylinders. As we have done before, we can extract these intercepts and slopes for the three lines, and plot them accordingly. int_4cyl = coef(mpg_disp_add_cyl)[1] int_6cyl = coef(mpg_disp_add_cyl)[1] + coef(mpg_disp_add_cyl)[3] int_8cyl = coef(mpg_disp_add_cyl)[1] + coef(mpg_disp_add_cyl)[4] slope_all_cyl = coef(mpg_disp_add_cyl)[2] plot_colors = c(&quot;Darkorange&quot;, &quot;Darkgrey&quot;, &quot;Dodgerblue&quot;) plot(mpg ~ disp, data = autompg, col = plot_colors[cyl], pch = as.numeric(cyl)) abline(int_4cyl, slope_all_cyl, col = plot_colors[1], lty = 1, lwd = 2) abline(int_6cyl, slope_all_cyl, col = plot_colors[2], lty = 2, lwd = 2) abline(int_8cyl, slope_all_cyl, col = plot_colors[3], lty = 3, lwd = 2) legend(&quot;topright&quot;, c(&quot;4 Cylinder&quot;, &quot;6 Cylinder&quot;, &quot;8 Cylinder&quot;), col = plot_colors, lty = c(1, 2, 3), pch = c(1, 2, 3)) On this plot, we have 4 Cylinder: orange dots, solid orange line. 6 Cylinder: grey dots, dashed grey line. 8 Cylinder: blue dots, dotted blue line. The odd result here is that we’re estimating that 8 cylinder cars have better fuel efficiency than 6 cylinder cars at any displacement! The dotted blue line is always above the dashed grey line. That doesn’t seem right. Maybe for very large displacement engines that could be true, but that seems wrong for medium to low displacement. To attempt to fix this, we will try using an interaction model, that is, instead of simply three intercepts and one slope, we will allow for three slopes. Again, we’ll let R take the wheel (no pun intended), then figure out what model it has applied. (mpg_disp_int_cyl = lm(mpg ~ disp * cyl, data = autompg)) ## ## Call: ## lm(formula = mpg ~ disp * cyl, data = autompg) ## ## Coefficients: ## (Intercept) disp cyl6 cyl8 disp:cyl6 disp:cyl8 ## 43.59052 -0.13069 -13.20026 -20.85706 0.08299 0.10817 # could also use mpg ~ disp + cyl + disp:cyl R has again chosen to use 4 cylinder cars as the reference level, but this also now has an effect on the interaction terms. R has fit the model. \\[ Y = \\beta_0 + \\beta_1 x + \\beta_2 v_2 + \\beta_3 v_3 + \\gamma_2 x v_2 + \\gamma_3 x v_3 + \\epsilon \\] We’re using \\(\\gamma\\) like a \\(\\beta\\) parameter for simplicity, so that, for example \\(\\beta_2\\) and \\(\\gamma_2\\) are both associated with \\(v_2\\). Now, the three “sub models” are: 4 Cylinder: \\(Y = \\beta_0 + \\beta_1 x + \\epsilon\\). 6 Cylinder: \\(Y = (\\beta_0 + \\beta_2) + (\\beta_1 + \\gamma_2) x + \\epsilon\\). 8 Cylinder: \\(Y = (\\beta_0 + \\beta_3) + (\\beta_1 + \\gamma_3) x + \\epsilon\\). Interpreting some parameters and coefficients then: \\((\\beta_0 + \\beta_2)\\) is the average mpg of a 6 cylinder car with 0 disp \\((\\hat{\\beta}_1 + \\hat{\\gamma}_3) = -0.1306935 + 0.1081714 = -0.0225221\\) is the estimated change in average mpg for an increase of one disp, for an 8 cylinder car. So, as we have seen before \\(\\beta_2\\) and \\(\\beta_3\\) change the intercepts for 6 and 8 cylinder cars relative to the reference level of \\(\\beta_0\\) for 4 cylinder cars. Now, similarly \\(\\gamma_2\\) and \\(\\gamma_3\\) change the slopes for 6 and 8 cylinder cars relative to the reference level of \\(\\beta_1\\) for 4 cylinder cars. Once again, we extract the coefficients and plot the results. int_4cyl = coef(mpg_disp_int_cyl)[1] int_6cyl = coef(mpg_disp_int_cyl)[1] + coef(mpg_disp_int_cyl)[3] int_8cyl = coef(mpg_disp_int_cyl)[1] + coef(mpg_disp_int_cyl)[4] slope_4cyl = coef(mpg_disp_int_cyl)[2] slope_6cyl = coef(mpg_disp_int_cyl)[2] + coef(mpg_disp_int_cyl)[5] slope_8cyl = coef(mpg_disp_int_cyl)[2] + coef(mpg_disp_int_cyl)[6] plot_colors = c(&quot;Darkorange&quot;, &quot;Darkgrey&quot;, &quot;Dodgerblue&quot;) plot(mpg ~ disp, data = autompg, col = plot_colors[cyl], pch = as.numeric(cyl)) abline(int_4cyl, slope_4cyl, col = plot_colors[1], lty = 1, lwd = 2) abline(int_6cyl, slope_6cyl, col = plot_colors[2], lty = 2, lwd = 2) abline(int_8cyl, slope_8cyl, col = plot_colors[3], lty = 3, lwd = 2) legend(&quot;topright&quot;, c(&quot;4 Cylinder&quot;, &quot;6 Cylinder&quot;, &quot;8 Cylinder&quot;), col = plot_colors, lty = c(1, 2, 3), pch = c(1, 2, 3)) This looks much better! We can see that for medium displacement cars, 6 cylinder cars now perform better than 8 cylinder cars, which seems much more reasonable than before. To completely justify the interaction model (i.e., a unique slope for each cyl level) compared to the additive model (single slope), we can perform an \\(F\\)-test. Notice first, that there is no \\(t\\)-test that will be able to do this since the difference between the two models is not a single parameter. We will test, \\[ H_0: \\gamma_2 = \\gamma_3 = 0 \\] which represents the parallel regression lines we saw before, \\[ Y = \\beta_0 + \\beta_1 x + \\beta_2 v_2 + \\beta_3 v_3 + \\epsilon. \\] Again, this is a difference of two parameters, thus no \\(t\\)-test will be useful. anova(mpg_disp_add_cyl, mpg_disp_int_cyl) ## Analysis of Variance Table ## ## Model 1: mpg ~ disp + cyl ## Model 2: mpg ~ disp * cyl ## Res.Df RSS Df Sum of Sq F Pr(&gt;F) ## 1 379 7299.5 ## 2 377 6551.7 2 747.79 21.515 1.419e-09 *** ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 As expected, we see a very low p-value, and thus reject the null. We prefer the interaction model over the additive model. Recapping a bit: Null Model: \\(Y = \\beta_0 + \\beta_1 x + \\beta_2 v_2 + \\beta_3 v_3 + \\epsilon\\) Number of parameters: \\(q = 4\\) Full Model: \\(Y = \\beta_0 + \\beta_1 x + \\beta_2 v_2 + \\beta_3 v_3 + \\gamma_2 x v_2 + \\gamma_3 x v_3 + \\epsilon\\) Number of parameters: \\(p = 6\\) length(coef(mpg_disp_int_cyl)) - length(coef(mpg_disp_add_cyl)) ## [1] 2 We see there is a difference of two parameters, which is also displayed in the resulting ANOVA table from R. Notice that the following two values also appear on the ANOVA table. nrow(autompg) - length(coef(mpg_disp_int_cyl)) ## [1] 377 nrow(autompg) - length(coef(mpg_disp_add_cyl)) ## [1] 379 11.4 Parameterization So far we have been simply letting R decide how to create the dummy variables, and thus R has been deciding the parameterization of the models. To illustrate the ability to use alternative parameterizations, we will recreate the data, but directly creating the dummy variables ourselves. new_param_data = data.frame( y = autompg$mpg, x = autompg$disp, v1 = 1 * as.numeric(autompg$cyl == 4), v2 = 1 * as.numeric(autompg$cyl == 6), v3 = 1 * as.numeric(autompg$cyl == 8)) head(new_param_data, 20) ## y x v1 v2 v3 ## 1 18 307 0 0 1 ## 2 15 350 0 0 1 ## 3 18 318 0 0 1 ## 4 16 304 0 0 1 ## 5 17 302 0 0 1 ## 6 15 429 0 0 1 ## 7 14 454 0 0 1 ## 8 14 440 0 0 1 ## 9 14 455 0 0 1 ## 10 15 390 0 0 1 ## 11 15 383 0 0 1 ## 12 14 340 0 0 1 ## 13 15 400 0 0 1 ## 14 14 455 0 0 1 ## 15 24 113 1 0 0 ## 16 22 198 0 1 0 ## 17 18 199 0 1 0 ## 18 21 200 0 1 0 ## 19 27 97 1 0 0 ## 20 26 97 1 0 0 Now, y is mpg x is disp, the displacement in cubic inches, v1, v2, and v3 are dummy variables as defined above. First let’s try to fit an additive model using x as well as the three dummy variables. lm(y ~ x + v1 + v2 + v3, data = new_param_data) ## ## Call: ## lm(formula = y ~ x + v1 + v2 + v3, data = new_param_data) ## ## Coefficients: ## (Intercept) x v1 v2 v3 ## 32.96326 -0.05217 2.03603 -1.59722 NA What is happening here? Notice that R is essentially ignoring v3, but why? Well, because R uses an intercept, it cannot also use v3. This is because \\[ \\boldsymbol{1} = v_1 + v_2 + v_3 \\] which means that \\(\\boldsymbol{1}\\), \\(v_1\\), \\(v_2\\), and \\(v_3\\) are linearly dependent. This would make the \\(X^\\top X\\) matrix singular, but we need to be able to invert it to solve the normal equations and obtain \\(\\hat{\\beta}.\\) With the intercept, v1, and v2, R can make the necessary “three intercepts.” So, in this case v3 is the reference level. If we remove the intercept, then we can directly obtain all “three intercepts” without a reference level. lm(y ~ 0 + x + v1 + v2 + v3, data = new_param_data) ## ## Call: ## lm(formula = y ~ 0 + x + v1 + v2 + v3, data = new_param_data) ## ## Coefficients: ## x v1 v2 v3 ## -0.05217 34.99929 31.36604 32.96326 Here, we are fitting the model \\[ Y = \\mu_1 v_1 + \\mu_2 v_2 + \\mu_3 v_3 + \\beta x +\\epsilon. \\] Thus we have: 4 Cylinder: \\(Y = \\mu_1 + \\beta x + \\epsilon\\) 6 Cylinder: \\(Y = \\mu_2 + \\beta x + \\epsilon\\) 8 Cylinder: \\(Y = \\mu_3 + \\beta x + \\epsilon\\) We could also do something similar with the interaction model, and give each line an intercept and slope, without the need for a reference level. lm(y ~ 0 + v1 + v2 + v3 + x:v1 + x:v2 + x:v3, data = new_param_data) ## ## Call: ## lm(formula = y ~ 0 + v1 + v2 + v3 + x:v1 + x:v2 + x:v3, data = new_param_data) ## ## Coefficients: ## v1 v2 v3 v1:x v2:x v3:x ## 43.59052 30.39026 22.73346 -0.13069 -0.04770 -0.02252 \\[ Y = \\mu_1 v_1 + \\mu_2 v_2 + \\mu_3 v_3 + \\beta_1 x v_1 + \\beta_2 x v_2 + \\beta_3 x v_3 +\\epsilon \\] 4 Cylinder: \\(Y = \\mu_1 + \\beta_1 x + \\epsilon\\) 6 Cylinder: \\(Y = \\mu_2 + \\beta_2 x + \\epsilon\\) 8 Cylinder: \\(Y = \\mu_3 + \\beta_3 x + \\epsilon\\) Using the original data, we have (at least) three equivalent ways to specify the interaction model with R. lm(mpg ~ disp * cyl, data = autompg) ## ## Call: ## lm(formula = mpg ~ disp * cyl, data = autompg) ## ## Coefficients: ## (Intercept) disp cyl6 cyl8 disp:cyl6 disp:cyl8 ## 43.59052 -0.13069 -13.20026 -20.85706 0.08299 0.10817 lm(mpg ~ 0 + cyl + disp : cyl, data = autompg) ## ## Call: ## lm(formula = mpg ~ 0 + cyl + disp:cyl, data = autompg) ## ## Coefficients: ## cyl4 cyl6 cyl8 cyl4:disp cyl6:disp cyl8:disp ## 43.59052 30.39026 22.73346 -0.13069 -0.04770 -0.02252 lm(mpg ~ 0 + disp + cyl + disp : cyl, data = autompg) ## ## Call: ## lm(formula = mpg ~ 0 + disp + cyl + disp:cyl, data = autompg) ## ## Coefficients: ## disp cyl4 cyl6 cyl8 disp:cyl6 disp:cyl8 ## -0.13069 43.59052 30.39026 22.73346 0.08299 0.10817 They all fit the same model, importantly each using six parameters, but the coefficients mean slightly different things in each. However, once they are interpreted as slopes and intercepts for the “three lines” they will have the same result. Use ?all.equal to learn about the all.equal() function, and think about how the following code verifies that the residuals of the two models are the same. all.equal(fitted(lm(mpg ~ disp * cyl, data = autompg)), fitted(lm(mpg ~ 0 + cyl + disp : cyl, data = autompg))) ## [1] TRUE 11.5 Building Larger Models Now that we have seen how to incorporate categorical predictors as well as interaction terms, we can start to build much larger, much more flexible models which can potentially fit data better. Let’s define a “big” model, \\[ Y = \\beta_0 + \\beta_1 x_1 + \\beta_2 x_2 + \\beta_3 x_3 + \\beta_4 x_1 x_2 + \\beta_5 x_1 x_3 + \\beta_6 x_2 x_3 + \\beta_7 x_1 x_2 x_3 + \\epsilon. \\] Here, \\(Y\\) is mpg. \\(x_1\\) is disp. \\(x_2\\) is hp. \\(x_3\\) is domestic, which is a dummy variable we defined, where 1 is a domestic vehicle. First thing to note here, we have included a new term \\(x_1 x_2 x_3\\) which is a three-way interaction. Interaction terms can be larger and larger, up to the number of predictors in the model. Since we are using the three-way interaction term, we also use all possible two-way interactions, as well as each of the first order (main effect) terms. This is the concept of a hierarchy. Any time a “higher-order” term is in a model, the related “lower-order” terms should also be included. Mathematically their inclusion or exclusion is sometimes irrelevant, but from an interpretation standpoint, it is best to follow the hierarchy rules. Let’s do some rearrangement to obtain a “coefficient” in front of \\(x_1\\). \\[ Y = \\beta_0 + \\beta_2 x_2 + \\beta_3 x_3 + \\beta_6 x_2 x_3 + (\\beta_1 + \\beta_4 x_2 + \\beta_5 x_3 + \\beta_7 x_2 x_3)x_1 + \\epsilon. \\] Specifically, the “coefficient” in front of \\(x_1\\) is \\[ (\\beta_1 + \\beta_4 x_2 + \\beta_5 x_3 + \\beta_7 x_2 x_3). \\] Let’s discuss this “coefficient” to help us understand the idea of the flexibility of a model. Recall that, \\(\\beta_1\\) is the coefficient for a first order term, \\(\\beta_4\\) and \\(\\beta_5\\) are coefficients for two-way interactions, \\(\\beta_7\\) is the coefficient for the three-way interaction. If the two and three way interactions were not in the model, the whole “coefficient” would simply be \\[ \\beta_1. \\] Thus, no matter the values of \\(x_2\\) and \\(x_3\\), \\(\\beta_1\\) would determine the relationship between \\(x_1\\) (disp) and \\(Y\\) (mpg). With the addition of the two-way interactions, now the “coefficient” would be \\[ (\\beta_1 + \\beta_4 x_2 + \\beta_5 x_3). \\] Now, changing \\(x_1\\) (disp) has a different effect on \\(Y\\) (mpg), depending on the values of \\(x_2\\) and \\(x_3\\). Lastly, adding the three-way interaction gives the whole “coefficient” \\[ (\\beta_1 + \\beta_4 x_2 + \\beta_5 x_3 + \\beta_7 x_2 x_3) \\] which is even more flexible. Now changing \\(x_1\\) (disp) has a different effect on \\(Y\\) (mpg), depending on the values of \\(x_2\\) and \\(x_3\\), but in a more flexible way which we can see with some more rearrangement. Now the “coefficient” in front of \\(x_3\\) in this “coefficient” is dependent on \\(x_2\\). \\[ (\\beta_1 + \\beta_4 x_2 + (\\beta_5 + \\beta_7 x_2) x_3) \\] It is so flexible, it is becoming hard to interpret! Let’s fit this three-way interaction model in R. big_model = lm(mpg ~ disp * hp * domestic, data = autompg) summary(big_model) ## ## Call: ## lm(formula = mpg ~ disp * hp * domestic, data = autompg) ## ## Residuals: ## Min 1Q Median 3Q Max ## -11.9410 -2.2147 -0.4008 1.9430 18.4094 ## ## Coefficients: ## Estimate Std. Error t value Pr(&gt;|t|) ## (Intercept) 6.065e+01 6.600e+00 9.189 &lt; 2e-16 *** ## disp -1.416e-01 6.344e-02 -2.232 0.0262 * ## hp -3.545e-01 8.123e-02 -4.364 1.65e-05 *** ## domestic -1.257e+01 7.064e+00 -1.780 0.0759 . ## disp:hp 1.369e-03 6.727e-04 2.035 0.0426 * ## disp:domestic 4.933e-02 6.400e-02 0.771 0.4414 ## hp:domestic 1.852e-01 8.709e-02 2.126 0.0342 * ## disp:hp:domestic -9.163e-04 6.768e-04 -1.354 0.1766 ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 ## ## Residual standard error: 3.88 on 375 degrees of freedom ## Multiple R-squared: 0.76, Adjusted R-squared: 0.7556 ## F-statistic: 169.7 on 7 and 375 DF, p-value: &lt; 2.2e-16 Do we actually need this large of a model? Let’s first test for the necessity of the three-way interaction term. That is, \\[ H_0: \\beta_7 = 0. \\] So, Full Model: \\(Y = \\beta_0 + \\beta_1 x_1 + \\beta_2 x_2 + \\beta_3 x_3 + \\beta_4 x_1 x_2 + \\beta_5 x_1 x_3 + \\beta_6 x_2 x_3 + \\beta_7 x_1 x_2 x_3 + \\epsilon\\) Null Model: \\(Y = \\beta_0 + \\beta_1 x_1 + \\beta_2 x_2 + \\beta_3 x_3 + \\beta_4 x_1 x_2 + \\beta_5 x_1 x_3 + \\beta_6 x_2 x_3 + \\epsilon\\) We fit the null model in R as two_way_int_mod, then use anova() to perform an \\(F\\)-test as usual. two_way_int_mod = lm(mpg ~ disp * hp + disp * domestic + hp * domestic, data = autompg) #two_way_int_mod = lm(mpg ~ (disp + hp + domestic) ^ 2, data = autompg) anova(two_way_int_mod, big_model) ## Analysis of Variance Table ## ## Model 1: mpg ~ disp * hp + disp * domestic + hp * domestic ## Model 2: mpg ~ disp * hp * domestic ## Res.Df RSS Df Sum of Sq F Pr(&gt;F) ## 1 376 5673.2 ## 2 375 5645.6 1 27.599 1.8332 0.1766 We see the p-value is somewhat large, so we would fail to reject. We prefer the smaller, less flexible, null model, without the three-way interaction. A quick note here: the full model does still “fit better.” Notice that it has a smaller RMSE than the null model, which means the full model makes smaller (squared) errors on average. mean(resid(big_model) ^ 2) ## [1] 14.74053 mean(resid(two_way_int_mod) ^ 2) ## [1] 14.81259 However, it is not much smaller. We could even say that the difference is insignificant. This is an idea we will return to later in greater detail. Now that we have chosen the model without the three-way interaction, can we go further? Do we need the two-way interactions? Let’s test \\[ H_0: \\beta_4 = \\beta_5 = \\beta_6 = 0. \\] Remember we already chose \\(\\beta_7 = 0\\), so, Full Model: \\(Y = \\beta_0 + \\beta_1 x_1 + \\beta_2 x_2 + \\beta_3 x_3 + \\beta_4 x_1 x_2 + \\beta_5 x_1 x_3 + \\beta_6 x_2 x_3 + \\epsilon\\) Null Model: \\(Y = \\beta_0 + \\beta_1 x_1 + \\beta_2 x_2 + \\beta_3 x_3 + \\epsilon\\) We fit the null model in R as additive_mod, then use anova() to perform an \\(F\\)-test as usual. additive_mod = lm(mpg ~ disp + hp + domestic, data = autompg) anova(additive_mod, two_way_int_mod) ## Analysis of Variance Table ## ## Model 1: mpg ~ disp + hp + domestic ## Model 2: mpg ~ disp * hp + disp * domestic + hp * domestic ## Res.Df RSS Df Sum of Sq F Pr(&gt;F) ## 1 379 7369.7 ## 2 376 5673.2 3 1696.5 37.478 &lt; 2.2e-16 *** ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 Here the p-value is small, so we reject the null, and we prefer the full (alternative) model. Of the models we have considered, our final preference is for \\[ Y = \\beta_0 + \\beta_1 x_1 + \\beta_2 x_2 + \\beta_3 x_3 + \\beta_4 x_1 x_2 + \\beta_5 x_1 x_3 + \\beta_6 x_2 x_3 + \\epsilon. \\] 11.6 R Markdown The R Markdown file for this chapter can be found here: cat-int.Rmd The file was created using R version 4.3.2. "],["analysis-of-variance.html", "Chapter 12 Analysis of Variance 12.1 Experiments 12.2 Two-Sample t-Test 12.3 One-Way ANOVA 12.4 Post Hoc Testing 12.5 Two-Way ANOVA 12.6 R Markdown", " Chapter 12 Analysis of Variance Chapter Status: This chapter should be considered optional for a first reading of this text. Its inclusion is mostly for the benefit of some courses that use the text. Additionally, this chapter is currently somewhat underdeveloped compared to the rest of the text. If you are interested in contributing, you can find several lines marked “TODO” in the source. Pull requests encouraged! .table { width: 70%; margin-left:10%; margin-right:10%; } “To find out what happens when you change something, it is necessary to change it.” — Box, Hunter, and Hunter, Statistics for Experimenters (1978) Thus far, we have built models for numeric responses, when the predictors are all numeric. We’ll take a minor detour to go back and consider models which only have categorical predictors. A categorical predictor is a variable which takes only a finite number of values, which are not ordered. For example a variable which takes possible values red, blue, green is categorical. In the context of using a categorical variable as a predictor, it would place observations into different groups (categories). We’ve also mostly been dealing with observational data. The methods in this section are most useful in experimental settings, but still work with observational data. (However, for determining causation, we require experiments.) 12.1 Experiments The biggest difference between an observational study and an experiment is how the predictor data is obtained. Is the experimenter in control? In an observational study, both response and predictor data are obtained via observation. In an experiment, the predictor data are values determined by the experimenter. The experiment is run and the response is observed. In an experiment, the predictors, which are controlled by the experimenter, are called factors. The possible values of these factors are called levels. Subjects are randomly assigned to a level of each of the factors. The design of experiments could be a course by itself. The Wikipedia article on design of experiments gives a good overview. Originally, most of the methodology was developed for agricultural applications by R. A. Fisher, but is still in use today, now in a wide variety of application areas. Notably, these methods have seen a resurgence as a part of “A/B Testing.” 12.2 Two-Sample t-Test The simplest example of an experimental design is the setup for a two-sample \\(t\\)-test. There is a single factor variable with two levels which split the subjects into two groups. Often, one level is considered the control, while the other is the treatment. The subjects are randomly assigned to one of the two groups. After being assigned to a group, each subject has some quantity measured, which is the response variable. Mathematically, we consider the model \\[ y_{ij} \\sim N(\\mu_i, \\sigma^2) \\] where \\(i = 1, 2\\) for the two groups and \\(j = 1, 2, \\ldots n_i\\). Here \\(n_i\\) is the number of subjects in group \\(i\\). So \\(y_{13}\\) would be the measurement for the third member of the first group. So measurements of subjects in group \\(1\\) follow a normal distribution with mean \\(\\mu_1\\). \\[ y_{1j} \\sim N(\\mu_1, \\sigma^2) \\] Then measurements of subjects in group \\(2\\) follow a normal distribution with mean \\(\\mu_2\\). \\[ y_{2j} \\sim N(\\mu_2, \\sigma^2) \\] This model makes a number of assumptions. Specifically, The observations follow a normal distribution. The mean of each group is different. Equal variance for each group. Independence. Which is believable if groups were randomly assigned. Later, we will investigate the normal and equal variance assumptions. For now, we will continue to assume they are reasonable. The natural question to ask: Is there a difference between the two groups? The specific question we’ll answer: Are the means of the two groups different? Mathematically, that is \\[ H_0: \\mu_1 = \\mu_2 \\quad \\text{vs} \\quad H_1: \\mu_1 \\neq \\mu_2 \\] For the stated model and assuming the null hypothesis is true, the \\(t\\) test statistic would follow a \\(t\\) distribution with degrees of freedom \\(n_1 + n_2 - 2\\). As an example, suppose we are interested in the effect of melatonin on sleep duration. A researcher obtains a random sample of 20 adult males. Of these subjects, 10 are randomly chosen for the control group, which will receive a placebo. The remaining 10 will be given 5mg of melatonin before bed. The sleep duration in hours of each subject is then measured. The researcher chooses a significance level of \\(\\alpha = 0.10\\). Was sleep duration affected by the melatonin? melatonin ## sleep group ## 1 8.145150 control ## 2 7.522362 treatment ## 3 6.935754 control ## 4 8.959435 treatment ## 5 6.985122 control ## 6 8.072651 treatment ## 7 8.313826 control ## 8 8.086409 treatment ## 9 8.922108 control ## 10 8.124743 treatment ## 11 8.065844 control ## 12 10.943974 treatment ## 13 4.833367 control ## 14 7.865453 treatment ## 15 6.340014 control ## 16 8.963140 treatment ## 17 6.158896 control ## 18 5.012253 treatment ## 19 3.571440 control ## 20 9.784136 treatment Here, we would like to test, \\[ H_0: \\mu_C = \\mu_T \\quad \\text{vs} \\quad H_1: \\mu_C \\neq \\mu_T \\] To do so in R, we use the t.test() function, with the var.equal argument set to TRUE. t.test(sleep ~ group, data = melatonin, var.equal = TRUE) ## ## Two Sample t-test ## ## data: sleep by group ## t = -2.0854, df = 18, p-value = 0.05154 ## alternative hypothesis: true difference in means between group control and group treatment is not equal to 0 ## 95 percent confidence interval: ## -3.02378261 0.01117547 ## sample estimates: ## mean in group control mean in group treatment ## 6.827152 8.333456 At a significance level of \\(\\alpha = 0.10\\), we reject the null hypothesis. It seems that the melatonin had a statistically significant effect. Be aware that statistical significance is not always the same as scientific or practical significance. To determine practical significance, we need to investigate the effect size in the context of the situation. Here the effect size is the difference of the sample means. t.test(sleep ~ group, data = melatonin, var.equal = TRUE)$estimate ## mean in group control mean in group treatment ## 6.827152 8.333456 Here we see that the subjects in the melatonin group sleep an average of about 1.5 hours longer than the control group. An hour and a half of sleep is certainly important! With a big enough sample size, we could make an effect size of say, four minutes statistically significant. Is it worth taking a pill every night to get an extra four minutes of sleep? (Probably not.) boxplot(sleep ~ group, data = melatonin, col = 5:6) 12.3 One-Way ANOVA What if there are more than two groups? Consider the model \\[ y_{ij} = \\mu + \\alpha_i + e_{ij} \\] where \\[ \\sum_{i=1}^{g} \\alpha_i = 0 \\] and \\[ e_{ij} \\sim N(0,\\sigma^{2}). \\] Here, \\(i = 1, 2, \\ldots g\\) where \\(g\\) is the number of groups. \\(j = 1, 2, \\ldots n_i\\) where \\(n_i\\) is the number of observations in group \\(i\\). Then the total sample size is \\[ N = \\sum_{i = 1}^{g} n_i \\] Observations from group \\(i\\) follow a normal distribution \\[ y_{ij} \\sim N(\\mu_i,\\sigma^{2}) \\] where the mean of each group is given by \\[ \\mu_i = \\mu + \\alpha_i. \\] Here \\(\\alpha_i\\) measures the effect of group \\(i\\). It is the difference between the overall mean and the mean of group \\(i\\). Essentially, the assumptions here are the same as the two-sample case, however now, we simply have more groups. Much like the two-sample case, we would again like to test if the means of the groups are equal. \\[ H_0: \\mu_1 = \\mu_2 = \\ldots \\mu_g \\quad \\text{vs} \\quad H_1: \\text{ Not all } \\mu_i \\text{ are equal.} \\] Notice that the alternative simply indicates that some of the means are not equal, not specifically which are not equal. More on that later. Alternatively, we could write \\[ H_0: \\alpha_1 = \\alpha_2 = \\ldots = \\alpha_g = 0 \\quad \\text{vs} \\quad H_1: \\text{ Not all } \\alpha_i \\text{ are } 0. \\] This test is called Analysis of Variance (ANOVA). ANOVA compares the variation due to specific sources (between groups) with the variation among individuals who should be similar (within groups). In particular, ANOVA tests whether several populations have the same mean by comparing how far apart the sample means are with how much variation there is within the samples. We use variability of means to test for equality of means, thus the use of variance in the name for a test about means. We’ll leave out most of the details about how the estimation is done, but we’ll see later, that it is done via least squares. We’ll use R to obtain these estimates, but they are actually rather simple. We only need to think about the sample means of the groups. \\(\\bar{y}_i\\) is the sample mean of group \\(i\\). \\(\\bar{y}\\) is the overall sample mean. \\(s_{i}^{2}\\) is the sample variance of group \\(i\\). We’ll then decompose the variance, as we’ve seen before in regression. The total variation measures how much the observations vary about the overall sample mean, ignoring the groups. \\[ SST = \\sum_{i = 1}^{g} \\sum_{j = 1}^{n_i} (y_{ij} - \\bar{y})^2 \\] The variation between groups looks at how far the individual sample means are from the overall sample mean. \\[ SSB = \\sum_{i = 1}^{g} \\sum_{j = 1}^{n_i} (\\bar{y}_i - \\bar{y})^2 = \\sum_{i = 1}^{g} n_i (\\bar{y}_i - \\bar{y})^2 \\] Lastly, the within group variation measures how far observations are from the sample mean of its group. \\[ SSW = \\sum_{i = 1}^{g} \\sum_{j = 1}^{n_i} (y_{ij} - \\bar{y}_i)^2 = \\sum_{i = 1}^{g} (n_i - 1) s_{i}^{2} \\] This could also be thought of as the error sum of squares, where \\(y_{ij}\\) is an observation and \\(\\bar{y}_i\\) is its fitted (predicted) value from the model. To develop the test statistic for ANOVA, we place this information into an ANVOA table. Source Sum of Squares Degrees of Freedom Mean Square \\(F\\) Between SSB \\(g - 1\\) SSB / DFB MSB / MSW Within SSW \\(N - g\\) SSW / DFW Total SST \\(N - 1\\) We reject the null (equal means) when the \\(F\\) statistic is large. This occurs when the variation between groups is large compared to the variation within groups. Under the null hypothesis, the distribution of the test statistic is \\(F\\) with degrees of freedom \\(g - 1\\) and \\(N - g\\). Let’s see what this looks like in a few situations. In each of the following examples, we’ll consider sampling 20 observations (\\(n_i = 20\\)) from three populations (groups; \\(g = 3\\)). First, consider \\(\\mu_A = -5, \\mu_B = 0, \\mu_C = 5\\) with \\(\\sigma = 1\\). The left panel shows the three normal distributions we are sampling from. The ticks along the \\(x\\)-axis show the randomly sampled observations. The right panel re-displays only the sampled values in a boxplot. Note that the mid-line of the boxes is usually the sample median. These boxplots have been modified to use the sample mean. Here the sample means vary a lot around the overall sample mean, which is the solid grey line on the right panel. Within the groups there is variability, but it is still obvious that the sample means are very different. As a result, we obtain a large test statistic, thus small p-value. \\(F = 374.4469511\\) \\(\\text{p-value} = 1.6349862\\times 10^{-33}\\) Now consider \\(\\mu_A = 0, \\mu_B = 0, \\mu_C = 0\\) with \\(\\sigma = 1\\). That is, equal means for the groups. Here the sample means vary only a tiny bit around the overall sample mean. Within the groups there is variability, this time much larger than the variability of the sample means. As a result, we obtain a small test statistic, thus large p-value. \\(F = 2.667892\\) \\(\\text{p-value} = 0.0780579\\) The next two examples show different means, with different levels of noise. Notice how these affect the test statistic and p-value. \\(\\mu_A = -1, \\mu_B = 0, \\mu_C = 1, \\sigma = 1\\) \\(F = 16.4879492\\) \\(\\text{p-value} = 2.2378806\\times 10^{-6}\\) Above, there isn’t obvious separation between the groups like the first example, but it is still obvious the means are different. Below, there is more noise. Visually it is somewhat hard to tell, but the test still suggests a difference of means. (At an \\(\\alpha\\) of 0.05.) \\(\\mu_A = -1, \\mu_B = 0, \\mu_C = 1, \\sigma = 2\\) \\(F = 4.6256472\\) \\(\\text{p-value} = 0.0137529\\) Let’s consider an example with real data. We’ll use the coagulation dataset from the faraway package. Here four different diets (A, B, C, D) were administered to a random sample of 24 animals. The subjects were randomly assigned to one of the four diets. For each, their blood coagulation time was measured in seconds. Here we would like to test \\[ H_0: \\mu_A = \\mu_B = \\mu_C = \\mu_D \\] where, for example, \\(\\mu_A\\) is the mean blood coagulation time for an animal that ate diet A. library(faraway) names(coagulation) ## [1] &quot;coag&quot; &quot;diet&quot; plot(coag ~ diet, data = coagulation, col = 2:5) We first load the data and create the relevant boxplot. The plot alone suggests a difference of means. The aov() function is used to obtain the relevant sums of squares. Using the summary() function on the output from aov() creates the desired ANOVA table (without the unneeded row for total). coag_aov = aov(coag ~ diet, data = coagulation) coag_aov ## Call: ## aov(formula = coag ~ diet, data = coagulation) ## ## Terms: ## diet Residuals ## Sum of Squares 228 112 ## Deg. of Freedom 3 20 ## ## Residual standard error: 2.366432 ## Estimated effects may be unbalanced summary(coag_aov) ## Df Sum Sq Mean Sq F value Pr(&gt;F) ## diet 3 228 76.0 13.57 4.66e-05 *** ## Residuals 20 112 5.6 ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 Were we to run this experiment, we would have pre-specified a significance level. However, notice that the p-value of this test is incredibly low, so using any reasonable significance level we would reject the null hypothesis. Thus we believe the diets had an effect on blood coagulation time. diets = data.frame(diet = unique(coagulation$diet)) data.frame(diets, coag = predict(coag_aov, diets)) ## diet coag ## 1 A 61 ## 2 B 66 ## 3 C 68 ## 4 D 61 Here, we’ve created a dataframe with a row for each diet. By predicting on this dataframe, we obtain the sample means of each diet (group). 12.3.1 Factor Variables When performing ANOVA in R, be sure the grouping variable is a factor variable. If it is not, your result might not be ANOVA, but instead a linear regression with the predictor variable considered numeric. set.seed(42) response = rnorm(15) group = c(rep(1, 5), rep(2, 5), rep(3, 5)) bad = data.frame(response, group) summary(aov(response ~ group, data = bad)) # wrong DF! ## Df Sum Sq Mean Sq F value Pr(&gt;F) ## group 1 0.017 0.0173 0.015 0.903 ## Residuals 13 14.698 1.1306 good = data.frame(response, group = as.factor(group)) summary(aov(response ~ group, data = good)) ## Df Sum Sq Mean Sq F value Pr(&gt;F) ## group 2 0.232 0.1158 0.096 0.909 ## Residuals 12 14.484 1.2070 is.factor(bad$group) # 1, 2, and 3 are numbers. ## [1] FALSE is.factor(good$group) # 1, 2, and 3 are labels. ## [1] TRUE 12.3.2 Some Simulation Here we verify the distribution of the test statistic under the null hypothesis. We simulate from a null model (equal variance) to obtain an empirical distribution of the \\(F\\) statistic. We add the curve for the expected distribution. library(broom) sim_anova = function(n = 10, mu_a = 0, mu_b = 0, mu_c = 0, mu_d = 0, sigma = 1, stat = TRUE) { # create data from one-way ANOVA model with four groups of equal size # response simulated from normal with group mean, shared variance # group variable indicates group A, B, C or D sim_data = data.frame( response = c(rnorm(n = n, mean = mu_a, sd = sigma), rnorm(n = n, mean = mu_b, sd = sigma), rnorm(n = n, mean = mu_c, sd = sigma), rnorm(n = n, mean = mu_d, sd = sigma)), group = c(rep(&quot;A&quot;, times = n), rep(&quot;B&quot;, times = n), rep(&quot;C&quot;, times = n), rep(&quot;D&quot;, times = n)) ) # obtain F-statistic and p-value for testing difference of means # use lm instead of aov for better result formatting with glance aov_results = lm(response ~ group, data = sim_data) f_stat = glance(aov_results)$statistic p_val = glance(aov_results)$p.value # return f_stat if stat = TRUE, otherwise, p-value ifelse(stat, f_stat, p_val) } f_stats = replicate(n = 5000, sim_anova(stat = TRUE)) hist(f_stats, breaks = 100, prob = TRUE, border = &quot;dodgerblue&quot;, main = &quot;Empirical Distribution of F&quot;) curve(df(x, df1 = 4 - 1, df2 = 40 - 4), col = &quot;darkorange&quot;, add = TRUE, lwd = 2) 12.3.3 Power Now that we’re performing experiments, getting more data means finding more test subjects, running more lab tests, etc. In other words, it will cost more time and money. We’d like to design our experiment so that we have a good chance of detecting an interesting effect size, without spending too much money. There’s no point in running an experiment if there’s only a very low chance that it has a significant result that you care about. (Remember, not all statistically significant results have practical value.) We’d like the ANOVA test to have high power for an alternative hypothesis with a minimum desired effect size. \\[ \\text{Power } = P(\\text{Reject } H_0 \\mid H_0 \\text{ False}) \\] That is, for a true difference of means that we deem interesting, we want the test to reject with high probability. A number of things can affect the power of a test: Effect size. It is easier to detect larger effects. Noise level \\(\\sigma\\). The less noise, the easier it is to detect signal (effect). We don’t have much ability to control this, except maybe to measure more accurately. Significance level \\(\\alpha\\). Lower significance level makes rejecting more difficult. (But also allows for fewer false positives.) Sample size. Large samples mean easier to detect effects. Balanced design. An equal number of observations per group leads to higher power. The following simulations look at the effect of significance level, effect size, and noise level on the power of an ANOVA \\(F\\)-test. Homework will look into sample size and balance. p_vals = replicate(n = 1000, sim_anova(mu_a = -1, mu_b = 0, mu_c = 0, mu_d = 1, sigma = 1.5, stat = FALSE)) mean(p_vals &lt; 0.05) ## [1] 0.661 mean(p_vals &lt; 0.01) ## [1] 0.384 p_vals = replicate(n = 1000, sim_anova(mu_a = -1, mu_b = 0, mu_c = 0, mu_d = 1, sigma = 2.0, stat = FALSE)) mean(p_vals &lt; 0.05) ## [1] 0.425 mean(p_vals &lt; 0.01) ## [1] 0.19 p_vals = replicate(n = 1000, sim_anova(mu_a = -2, mu_b = 0, mu_c = 0, mu_d = 2, sigma = 2.0, stat = FALSE)) mean(p_vals &lt; 0.05) ## [1] 0.956 mean(p_vals &lt; 0.01) ## [1] 0.837 12.4 Post Hoc Testing Suppose we reject the null hypothesis from the ANOVA test for equal means. That tells us that the means are different. But which means? All of them? Some of them? The obvious strategy is to test all possible comparisons of two means. We can do this easily in R. with(coagulation, pairwise.t.test(coag, diet, p.adj = &quot;none&quot;)) ## ## Pairwise comparisons using t tests with pooled SD ## ## data: coag and diet ## ## A B C ## B 0.00380 - - ## C 0.00018 0.15878 - ## D 1.00000 0.00086 2.3e-05 ## ## P value adjustment method: none # pairwise.t.test(coagulation$coag, coagulation$diet, p.adj = &quot;none&quot;) Notice the pairwise.t.test() function does not have a data argument. To avoid using attach() or the $ operator, we introduce the with() function. The commented line would perform the same operation. Also note that we are using the argument p.adj = \"none\". What is this? An adjustment (in this case not an adjustment) to the p-value of each test. Why would we need to do this? The adjustment is an attempt to correct for the multiple testing problem. (See also: Relevant XKCD. ) Imagine that you knew ahead of time that you were going to perform 100 \\(t\\)-tests. Suppose you wish to do this with a false positive rate of \\(\\alpha = 0.05\\). If we use this significance level for each test, for 100 tests, we then expect 5 false positives. That means, with 100 tests, we’re almost guaranteed to have at least one error. What we’d really like is for the family-wise error rate (FWER) to be 0.05. If we consider the 100 tests to be a single “experiment,” the FWER is the rate of one or more false positives for the full experiment (100 tests). Consider it an error rate for an entire procedure, instead of a single test. With this in mind, one of the simplest adjustments we can make, is to increase the p-values for each test, depending on the number of tests. In particular the Bonferroni correction simply multiplies by the number of tests. \\[ \\text{p-value-bonf} = \\min(1, n_{tests} \\cdot \\text{p-value}) \\] with(coagulation, pairwise.t.test(coag, diet, p.adj = &quot;bonferroni&quot;)) ## ## Pairwise comparisons using t tests with pooled SD ## ## data: coag and diet ## ## A B C ## B 0.02282 - - ## C 0.00108 0.95266 - ## D 1.00000 0.00518 0.00014 ## ## P value adjustment method: bonferroni We see that these p-values are much higher than the unadjusted p-values, thus, we are less likely to reject each test. As a result, the FWER is 0.05, instead of an error rate of 0.05 for each test. We can simulate the 100 test scenarios to illustrate this point. get_p_val = function() { # create data for two groups, equal mean y = rnorm(20, mean = 0, sd = 1) g = c(rep(&quot;A&quot;, 10), rep(&quot;B&quot;, 10)) # p-value of t-test when null is true glance(t.test(y ~ g, var.equal = TRUE))$p.value } set.seed(1337) # FWER with 100 tests # desired rate = 0.05 # no adjustment mean(replicate(1000, any(replicate(100, get_p_val()) &lt; 0.05))) ## [1] 0.994 # FWER with 100 tests # desired rate = 0.05 # bonferroni adjustment mean(replicate(1000, any(p.adjust(replicate(100, get_p_val()), &quot;bonferroni&quot;) &lt; 0.05))) ## [1] 0.058 For the specific case of testing all two-way mean differences after an ANOVA test, there are a number of potential methods for making an adjustment of this type. The pros and cons of the potential methods are beyond the scope of this course. We choose a method for its ease of use, and to a lesser extent, its developer. Tukey’s Honest Significance difference can be applied directly to an object which was created using aov(). It will adjust the p-values of the pairwise comparisons of the means to control the FWER, in this case, for 0.05. Notice it also gives confidence intervals for the difference of the means. TukeyHSD(coag_aov, conf.level = 0.95) ## Tukey multiple comparisons of means ## 95% family-wise confidence level ## ## Fit: aov(formula = coag ~ diet, data = coagulation) ## ## $diet ## diff lwr upr p adj ## B-A 5 0.7245544 9.275446 0.0183283 ## C-A 7 2.7245544 11.275446 0.0009577 ## D-A 0 -4.0560438 4.056044 1.0000000 ## C-B 2 -1.8240748 5.824075 0.4766005 ## D-B -5 -8.5770944 -1.422906 0.0044114 ## D-C -7 -10.5770944 -3.422906 0.0001268 Based on these results, we see no difference between A and D as well as B and C. All other pairwise comparisons are significant. If you return to the original boxplot, these results should not be surprising. Also, nicely, we can easily produce a plot of these confidence intervals. plot(TukeyHSD(coag_aov, conf.level = 0.95)) The creator of this method, John Tukey, is an important figure in the history of data science. He essentially predicted the rise of data science over 50 years ago. For some retrospective thoughts on those 50 years, see this paper from David Donoho. 12.5 Two-Way ANOVA What if there is more than one factor variable? Why do we need to limit ourselves to experiments with only one factor? We don’t! Consider the model \\[ y_{ijk} = \\mu + \\alpha_i + \\beta_j + (\\alpha \\beta)_{ij} + \\epsilon_{ijk} \\] where \\(\\epsilon_{ijk}\\) are \\(N(0, \\sigma^2)\\) random variables. We add constraints \\[ \\sum_{i=1}^{I} \\alpha_i = 0 \\quad \\quad \\sum_{j=1}^{J} \\beta_j = 0. \\] and \\[ (\\alpha \\beta)_{1j} + (\\alpha \\beta)_{2j} + (\\alpha \\beta)_{3j} = 0 \\\\ (\\alpha \\beta)_{i1} + (\\alpha \\beta)_{i2} + (\\alpha \\beta)_{i3} + (\\alpha \\beta)_{i4} = 0 \\] for any \\(i\\) or \\(j\\). Here, \\(i = 1, 2, \\ldots I\\) where \\(I\\) is the number of levels of factor \\(A\\). \\(j = 1, 2, \\ldots J\\) where \\(J\\) is the number of levels of factor \\(B\\). \\(k = 1, 2, \\ldots K\\) where \\(K\\) is the number of replicates per group. Here, we can think of a group as a combination of a level from each of the factors. So for example, one group will receive level \\(2\\) of factor \\(A\\) and level \\(3\\) of factor \\(B\\). The number of replicates is the number of subjects in each group. Here \\(y_{135}\\) would be the measurement for the fifth member (replicate) of the group for level \\(1\\) of factor \\(A\\) and level \\(3\\) of factor \\(B\\). We call this setup an \\(I \\times J\\) factorial design with \\(K\\) replicates. (Our current notation only allows for equal replicates in each group. It isn’t difficult to allow for different replicates for different groups, but we’ll proceed using equal replicates per group, which if possible, is desirable.) \\(\\alpha_i\\) measures the effect of level \\(i\\) of factor \\(A\\). We call these the main effects of factor \\(A\\). \\(\\beta_j\\) measures the effect of level \\(j\\) of factor \\(B\\). We call these the main effects of factor \\(B\\). \\((\\alpha \\beta)_{ij}\\) is a single parameter. We use \\(\\alpha \\beta\\) to note that this parameter measures the interaction between the two main effects. Under this setup, there are a number of models that we can compare. Consider a \\(2 \\times 2\\) factorial design. The following tables show the means for each of the possible groups under each model. Interaction Model: \\(y_{ijk} = \\mu + \\alpha_i + \\beta_j + (\\alpha \\beta)_{ij} + \\epsilon_{ijk}\\) Factor B, Level 1 Factor B, Level 2 Factor A, Level 1 \\(\\mu + \\alpha_1 + \\beta_1 + (\\alpha\\beta)_{11}\\) \\(\\mu + \\alpha_1 + \\beta_2 + (\\alpha\\beta)_{12}\\) Factor A, Level 2 \\(\\mu + \\alpha_2 + \\beta_1 + (\\alpha\\beta)_{21}\\) \\(\\mu + \\alpha_2 + \\beta_2 + (\\alpha\\beta)_{22}\\) Additive Model: \\(y_{ijk} = \\mu + \\alpha_i + \\beta_j + \\epsilon_{ijk}\\) Factor B, Level 1 Factor B, Level 2 Factor A, Level 1 \\(\\mu + \\alpha_1 + \\beta_1\\) \\(\\mu + \\alpha_1 + \\beta_2\\) Factor A, Level 2 \\(\\mu + \\alpha_2 + \\beta_1\\) \\(\\mu + \\alpha_2 + \\beta_2\\) Factor B Only Model (One-Way): \\(y_{ijk} = \\mu + \\beta_j + \\epsilon_{ijk}\\) Factor B, Level 1 Factor B, Level 2 Factor A, Level 1 \\(\\mu + \\beta_1\\) \\(\\mu + \\beta_2\\) Factor A, Level 2 \\(\\mu + \\beta_1\\) \\(\\mu + \\beta_2\\) Factor A Only Model (One-Way): \\(y_{ijk} = \\mu + \\alpha_i + \\epsilon_{ijk}\\) Factor B, Level 1 Factor B, Level 2 Factor A, Level 1 \\(\\mu + \\alpha_1\\) \\(\\mu + \\alpha_1\\) Factor A, Level 2 \\(\\mu + \\alpha_2\\) \\(\\mu + \\alpha_2\\) Null Model: \\(y_{ijk} = \\mu + \\epsilon_{ijk}\\) Factor B, Level 1 Factor B, Level 2 Factor A, Level 1 \\(\\mu\\) \\(\\mu\\) Factor A, Level 2 \\(\\mu\\) \\(\\mu\\) The question then, is which of these models should we use if we have two factors? The most important question to consider is whether or not we should model the interaction. Is the effect of Factor A the same for all levels of Factor B? In the additive model, yes. In the interaction model, no. Both models would use a different mean for each group, but in a very specific way in both cases. Let’s discuss these comparisons by looking at some examples. We’ll first look at the rats data from the faraway package. There are two factors here: poison and treat. We use the levels() function to extract the levels of a factor variable. levels(rats$poison) ## [1] &quot;I&quot; &quot;II&quot; &quot;III&quot; levels(rats$treat) ## [1] &quot;A&quot; &quot;B&quot; &quot;C&quot; &quot;D&quot; Here, 48 rats were randomly assigned both one of three poisons and one of four possible treatments. The experimenters then measured their survival time in tens of hours. A total of 12 groups, each with 4 replicates. Before running any tests, we should first look at the data. We will create interaction plots, which will help us visualize the effect of one factor, as we move through the levels of another factor. par(mfrow = c(1, 2)) with(rats, interaction.plot(poison, treat, time, lwd = 2, col = 1:4)) with(rats, interaction.plot(treat, poison, time, lwd = 2, col = 1:3)) If there is not interaction, thus an additive model, we would expect to see parallel lines. That would mean, when we change the level of one factor, there can be an effect on the response. However, the difference between the levels of the other factor should still be the same. The obvious indication of interaction would be lines that cross while heading in different directions. Here we don’t see that, but the lines aren’t strictly parallel, and there is some overlap on the right panel. However, is this interaction effect significant? Let’s fit each of the possible models, then investigate their estimates for each of the group means. rats_int = aov(time ~ poison * treat, data = rats) # interaction model rats_add = aov(time ~ poison + treat, data = rats) # additive model rats_pois = aov(time ~ poison , data = rats) # single factor model rats_treat = aov(time ~ treat, data = rats) # single factor model rats_null = aov(time ~ 1, data = rats) # null model To get the estimates, we’ll create a table which we will predict on. rats_table = expand.grid(poison = unique(rats$poison), treat = unique(rats$treat)) rats_table ## poison treat ## 1 I A ## 2 II A ## 3 III A ## 4 I B ## 5 II B ## 6 III B ## 7 I C ## 8 II C ## 9 III C ## 10 I D ## 11 II D ## 12 III D matrix(paste0(rats_table$poison, &quot;-&quot;, rats_table$treat) , 4, 3, byrow = TRUE) ## [,1] [,2] [,3] ## [1,] &quot;I-A&quot; &quot;II-A&quot; &quot;III-A&quot; ## [2,] &quot;I-B&quot; &quot;II-B&quot; &quot;III-B&quot; ## [3,] &quot;I-C&quot; &quot;II-C&quot; &quot;III-C&quot; ## [4,] &quot;I-D&quot; &quot;II-D&quot; &quot;III-D&quot; Since we’ll be repeating ourselves a number of times, we write a function to perform the prediction. Some housekeeping is done to keep the estimates in order, and provide row and column names. Above, we’ve shown where each of the estimates will be placed in the resulting matrix. get_est_means = function(model, table) { mat = matrix(predict(model, table), nrow = 4, ncol = 3, byrow = TRUE) colnames(mat) = c(&quot;I&quot;, &quot;II&quot;, &quot;III&quot;) rownames(mat) = c(&quot;A&quot;, &quot;B&quot;, &quot;C&quot;, &quot;D&quot;) mat } First, we obtain the estimates from the interaction model. Note that each cell has a different value. knitr::kable(get_est_means(model = rats_int, table = rats_table)) I II III A 0.4125 0.3200 0.210 B 0.8800 0.8150 0.335 C 0.5675 0.3750 0.235 D 0.6100 0.6675 0.325 Next, we obtain the estimates from the additive model. Again, each cell has a different value. We also see that these estimates are somewhat close to those from the interaction model. knitr::kable(get_est_means(model = rats_add, table = rats_table)) I II III A 0.4522917 0.3791667 0.1110417 B 0.8147917 0.7416667 0.4735417 C 0.5306250 0.4575000 0.1893750 D 0.6722917 0.5991667 0.3310417 To understand the difference, let’s consider the effect of the treatments. additive_means = get_est_means(model = rats_add, table = rats_table) additive_means[&quot;A&quot;,] - additive_means[&quot;B&quot;,] ## I II III ## -0.3625 -0.3625 -0.3625 interaction_means = get_est_means(model = rats_int, table = rats_table) interaction_means[&quot;A&quot;,] - interaction_means[&quot;B&quot;,] ## I II III ## -0.4675 -0.4950 -0.1250 This is the key difference between the interaction and additive models. The difference between the effect of treatments A and B is the same for each poison in the additive model. They are different in the interaction model. The remaining three models are much simpler, having either only row or only column effects. Or no effects in the case of the null model. knitr::kable(get_est_means(model = rats_pois, table = rats_table)) I II III A 0.6175 0.544375 0.27625 B 0.6175 0.544375 0.27625 C 0.6175 0.544375 0.27625 D 0.6175 0.544375 0.27625 knitr::kable(get_est_means(model = rats_treat, table = rats_table)) I II III A 0.3141667 0.3141667 0.3141667 B 0.6766667 0.6766667 0.6766667 C 0.3925000 0.3925000 0.3925000 D 0.5341667 0.5341667 0.5341667 knitr::kable(get_est_means(model = rats_null, table = rats_table)) I II III A 0.479375 0.479375 0.479375 B 0.479375 0.479375 0.479375 C 0.479375 0.479375 0.479375 D 0.479375 0.479375 0.479375 To perform the needed tests, we will need to create another ANOVA table. (We’ll skip the details of the sums of squares calculations and simply let R take care of them.) Source Sum of Squares Degrees of Freedom Mean Square \\(F\\) Factor A SSA \\(I -1\\) SSA / DFA MSA / MSE Factor B SSB \\(J -1\\) SSB / DFB MSB / MSE AB Interaction SSAB \\((I -1)(J -1)\\) SSAB / DFAB MSAB / MSE Error SSE \\(IJ(K - 1)\\) SSE / DFE Total SST \\(IJK - 1\\) The row for AB Interaction tests: \\[ H_0: \\text{ All }(\\alpha\\beta)_{ij} = 0. \\quad \\text{vs} \\quad H_1: \\text{ Not all } (\\alpha\\beta)_{ij} \\text{ are } 0. \\] Null Model: \\(y_{ijk} = \\mu + \\alpha_i + \\beta_j + \\epsilon_{ijk}.\\) (Additive Model.) Alternative Model: \\(y_{ijk} = \\mu + \\alpha_i + \\beta_j + (\\alpha \\beta)_{ij} + \\epsilon_{ijk}.\\) (Interaction Model.) We reject the null when the \\(F\\) statistic is large. Under the null hypothesis, the distribution of the test statistic is \\(F\\) with degrees of freedom \\((I -1)(J -1)\\) and \\(IJ(K - 1)\\). The row for Factor B tests: \\[ H_0: \\text{ All }\\beta_{j} = 0. \\quad \\text{vs} \\quad H_1: \\text{ Not all } \\beta_{j} \\text{ are } 0. \\] Null Model: \\(y_{ijk} = \\mu + \\alpha_i + \\epsilon_{ijk}.\\) (Only Factor A Model.) Alternative Model: \\(y_{ijk} = \\mu + \\alpha_i + \\beta_j + \\epsilon_{ijk}.\\) (Additive Model.) We reject the null when the \\(F\\) statistic is large. Under the null hypothesis, the distribution of the test statistic is \\(F\\) with degrees of freedom \\(J - 1\\) and \\(IJ(K - 1)\\). The row for Factor A tests: \\[ H_0: \\text{ All }\\alpha_{i} = 0. \\quad \\text{vs} \\quad H_1: \\text{ Not all } \\alpha_{i} \\text{ are } 0. \\] Null Model: \\(y_{ijk} = \\mu + \\beta_j + \\epsilon_{ijk}.\\) (Only Factor B Model.) Alternative Model: \\(y_{ijk} = \\mu + \\alpha_i + \\beta_j + \\epsilon_{ijk}.\\) (Additive Model.) We reject the null when the \\(F\\) statistic is large. Under the null hypothesis, the distribution of the test statistic is \\(F\\) with degrees of freedom \\(I - 1\\) and \\(IJ(K - 1)\\). These tests should be performed according to the model hierarchy. First consider the test of interaction. If it is significant, we select the interaction model and perform no further testing. If interaction is not significant, we then consider the necessity of the individual factors of the additive model. Model Hierarchy summary(aov(time ~ poison * treat, data = rats)) ## Df Sum Sq Mean Sq F value Pr(&gt;F) ## poison 2 1.0330 0.5165 23.222 3.33e-07 *** ## treat 3 0.9212 0.3071 13.806 3.78e-06 *** ## poison:treat 6 0.2501 0.0417 1.874 0.112 ## Residuals 36 0.8007 0.0222 ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 Using a significance level of \\(\\alpha = 0.05\\), we see that the interaction is not significant. Within the additive model, both factors are significant, so we select the additive model. Within the additive model, we could do further testing about the main effects. TukeyHSD(aov(time ~ poison + treat, data = rats)) ## Tukey multiple comparisons of means ## 95% family-wise confidence level ## ## Fit: aov(formula = time ~ poison + treat, data = rats) ## ## $poison ## diff lwr upr p adj ## II-I -0.073125 -0.2089936 0.0627436 0.3989657 ## III-I -0.341250 -0.4771186 -0.2053814 0.0000008 ## III-II -0.268125 -0.4039936 -0.1322564 0.0000606 ## ## $treat ## diff lwr upr p adj ## B-A 0.36250000 0.18976135 0.53523865 0.0000083 ## C-A 0.07833333 -0.09440532 0.25107198 0.6221729 ## D-A 0.22000000 0.04726135 0.39273865 0.0076661 ## C-B -0.28416667 -0.45690532 -0.11142802 0.0004090 ## D-B -0.14250000 -0.31523865 0.03023865 0.1380432 ## D-C 0.14166667 -0.03107198 0.31440532 0.1416151 For an example with interaction, we investigate the warpbreaks dataset, a default dataset in R. par(mfrow = c(1, 2)) with(warpbreaks, interaction.plot(wool, tension, breaks, lwd = 2, col = 2:4)) with(warpbreaks, interaction.plot(tension, wool, breaks, lwd = 2, col = 2:3)) Either plot makes it rather clear that the wool and tensions factors interact. summary(aov(breaks ~ wool * tension, data = warpbreaks)) ## Df Sum Sq Mean Sq F value Pr(&gt;F) ## wool 1 451 450.7 3.765 0.058213 . ## tension 2 2034 1017.1 8.498 0.000693 *** ## wool:tension 2 1003 501.4 4.189 0.021044 * ## Residuals 48 5745 119.7 ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 Using an \\(\\alpha\\) of \\(0.05\\), the ANOVA test finds that the interaction is significant, so we use the interaction model here. 12.6 R Markdown The R Markdown file for this chapter can be found here: anova.Rmd The file was created using R version 4.3.2. "],["model-diagnostics.html", "Chapter 13 Model Diagnostics 13.1 Model Assumptions 13.2 Checking Assumptions 13.3 Unusual Observations 13.4 Data Analysis Examples 13.5 R Markdown", " Chapter 13 Model Diagnostics “Your assumptions are your windows on the world. Scrub them off every once in a while, or the light won’t come in.” — Isaac Asimov After reading this chapter you will be able to: Understand the assumptions of a regression model. Assess regression model assumptions using visualizations and tests. Understand leverage, outliers, and influential points. Be able to identify unusual observations in regression models. 13.1 Model Assumptions Recall the multiple linear regression model that we have defined. \\[ Y_i = \\beta_0 + \\beta_1 x_{i1} + \\beta_2 x_{i2} + \\cdots + \\beta_{p-1} x_{i(p-1)} + \\epsilon_i, \\qquad i = 1, 2, \\ldots, n. \\] Using matrix notation, this model can be written much more succinctly as \\[ Y = X \\beta + \\epsilon. \\] Given data, we found the estimates for the \\(\\beta\\) parameters using \\[ \\hat{\\beta} = \\left( X^\\top X \\right)^{-1}X^\\top y. \\] We then noted that these estimates had mean \\[ \\text{E}[\\hat{\\beta}] = \\beta, \\] and variance \\[ \\text{Var}[\\hat{\\beta}] = \\sigma^2 \\left( X^\\top X \\right)^{-1}. \\] In particular, an individual parameter, say \\(\\hat{\\beta}_j\\) had a normal distribution \\[ \\hat{\\beta}_j \\sim N\\left(\\beta_j, \\sigma^2 C_{jj} \\right) \\] where \\(C\\) was the matrix defined as \\[ C = \\left(X^\\top X\\right)^{-1}. \\] We then used this fact to define \\[ \\frac{\\hat{\\beta}_j - \\beta_j}{s_e \\sqrt{C_{jj}}} \\sim t_{n-p}, \\] which we used to perform hypothesis testing. So far we have looked at various metrics such as RMSE, RSE, and \\(R^2\\) to determine how well our model fits our data. Each of these in some way considers the expression \\[ \\sum_{i = 1}^n (y_i - \\hat{y}_i)^2. \\] So, essentially each of these looks at how close the data points are to the model. However is that all we care about? It could be that the errors are made in a systematic way, which means that our model is misspecified. We may need additional interaction terms, or polynomial terms which we will see later. It is also possible that at a particular set of predictor values, the errors are very small, but at a different set of predictor values, the errors are large. Perhaps most of the errors are very small, but some are very large. This would suggest that the errors do not follow a normal distribution. Are these issues that we care about? If all we would like to do is predict, possibly not, since we would only care about the size of our errors. However, if we would like to perform inference, for example to determine if a particular predictor is important, we care a great deal. All of the distributional results, such as a \\(t\\)-test for a single predictor, are derived under the assumptions of our model. Technically, the assumptions of the model are encoded directly in a model statement such as, \\[ Y_i = \\beta_0 + \\beta_1 x_{i1} + \\beta_2 x_{i2} + \\cdots + \\beta_{p-1} x_{i(p-1)} + \\epsilon_i \\] where \\(\\epsilon_i \\sim N(0, \\sigma^2).\\) Often, the assumptions of linear regression, are stated as, Linearity: the response can be written as a linear combination of the predictors. (With noise about this true linear relationship.) Independence: the errors are independent. Normality: the distribution of the errors should follow a normal distribution. Equal Variance: the error variance is the same at any set of predictor values. The linearity assumption is encoded as \\[ \\beta_0 + \\beta_1 x_{i1} + \\beta_2 x_{i2} + \\cdots + \\beta_{p-1} x_{i(p-1)}, \\] while the remaining three, are all encoded in \\[ \\epsilon_i \\sim N(0, \\sigma^2), \\] since the \\(\\epsilon_i\\) are \\(iid\\) normal random variables with constant variance. If these assumptions are met, great! We can perform inference, and it is valid. If these assumptions are not met, we can still “perform” a \\(t\\)-test using R, but the results are not valid. The distributions of the parameter estimates will not be what we expect. Hypothesis tests will then accept or reject incorrectly. Essentially, garbage in, garbage out. 13.2 Checking Assumptions We’ll now look at a number of tools for checking the assumptions of a linear model. To test these tools, we’ll use data simulated from three models: \\[ \\text{Model 1:} \\quad Y = 3 + 5x + \\epsilon, \\quad \\epsilon \\sim N(0, 1) \\] \\[ \\text{Model 2:} \\quad Y = 3 + 5x + \\epsilon, \\quad \\epsilon \\sim N(0, x^2) \\] \\[ \\text{Model 3:} \\quad Y = 3 + 5x^2 + \\epsilon, \\quad \\epsilon \\sim N(0, 25) \\] sim_1 = function(sample_size = 500) { x = runif(n = sample_size) * 5 y = 3 + 5 * x + rnorm(n = sample_size, mean = 0, sd = 1) data.frame(x, y) } sim_2 = function(sample_size = 500) { x = runif(n = sample_size) * 5 y = 3 + 5 * x + rnorm(n = sample_size, mean = 0, sd = x) data.frame(x, y) } sim_3 = function(sample_size = 500) { x = runif(n = sample_size) * 5 y = 3 + 5 * x ^ 2 + rnorm(n = sample_size, mean = 0, sd = 5) data.frame(x, y) } 13.2.1 Fitted versus Residuals Plot Probably our most useful tool will be a Fitted versus Residuals Plot. It will be useful for checking both the linearity and constant variance assumptions. Data generated from Model 1 above should not show any signs of violating assumptions, so we’ll use this to see what a good fitted versus residuals plot should look like. First, we’ll simulate observations from this model. set.seed(42) sim_data_1 = sim_1() head(sim_data_1) ## x y ## 1 4.574030 24.773995 ## 2 4.685377 26.475936 ## 3 1.430698 8.954993 ## 4 4.152238 23.951210 ## 5 3.208728 20.341344 ## 6 2.595480 14.943525 We then fit the model and add the fitted line to a scatterplot. plot(y ~ x, data = sim_data_1, col = &quot;grey&quot;, pch = 20, main = &quot;Data from Model 1&quot;) fit_1 = lm(y ~ x, data = sim_data_1) abline(fit_1, col = &quot;darkorange&quot;, lwd = 3) We now plot a fitted versus residuals plot. Note, this is residuals on the \\(y\\)-axis despite the ordering in the name. Sometimes you will see this called a residuals versus fitted, or residuals versus predicted plot. plot(fitted(fit_1), resid(fit_1), col = &quot;grey&quot;, pch = 20, xlab = &quot;Fitted&quot;, ylab = &quot;Residuals&quot;, main = &quot;Data from Model 1&quot;) abline(h = 0, col = &quot;darkorange&quot;, lwd = 2) We should look for two things in this plot. At any fitted value, the mean of the residuals should be roughly 0. If this is the case, the linearity assumption is valid. For this reason, we generally add a horizontal line at \\(y = 0\\) to emphasize this point. At every fitted value, the spread of the residuals should be roughly the same. If this is the case, the constant variance assumption is valid. Here we see this is the case for both. To get a better idea of how a fitted versus residuals plot can be useful, we will simulate from models with violated assumptions. Model 2 is an example of non-constant variance. In this case, the variance is larger for larger values of the predictor variable \\(x\\). set.seed(42) sim_data_2 = sim_2() fit_2 = lm(y ~ x, data = sim_data_2) plot(y ~ x, data = sim_data_2, col = &quot;grey&quot;, pch = 20, main = &quot;Data from Model 2&quot;) abline(fit_2, col = &quot;darkorange&quot;, lwd = 3) This actually is rather easy to see here by adding the fitted line to a scatterplot. This is because we are only performing simple linear regression. With multiple regression, a fitted versus residuals plot is a necessity, since adding a fitted regression to a scatterplot isn’t exactly possible. plot(fitted(fit_2), resid(fit_2), col = &quot;grey&quot;, pch = 20, xlab = &quot;Fitted&quot;, ylab = &quot;Residuals&quot;, main = &quot;Data from Model 2&quot;) abline(h = 0, col = &quot;darkorange&quot;, lwd = 2) On the fitted versus residuals plot, we see two things very clearly. For any fitted value, the residuals seem roughly centered at 0. This is good! The linearity assumption is not violated. However, we also see very clearly, that for larger fitted values, the spread of the residuals is larger. This is bad! The constant variance assumption is violated here. Now we will demonstrate a model which does not meet the linearity assumption. Model 3 is an example of a model where \\(Y\\) is not a linear combination of the predictors. In this case the predictor is \\(x\\), but the model uses \\(x^2\\). (We’ll see later that this is something that a “linear” model can deal with. The fix is simple, just make \\(x^2\\) a predictor!) set.seed(42) sim_data_3 = sim_3() fit_3 = lm(y ~ x, data = sim_data_3) plot(y ~ x, data = sim_data_3, col = &quot;grey&quot;, pch = 20, main = &quot;Data from Model 3&quot;) abline(fit_3, col = &quot;darkorange&quot;, lwd = 3) Again, this is rather clear on the scatterplot, but again, we wouldn’t be able to check this plot for multiple regression. plot(fitted(fit_3), resid(fit_3), col = &quot;grey&quot;, pch = 20, xlab = &quot;Fitted&quot;, ylab = &quot;Residuals&quot;, main = &quot;Data from Model 3&quot;) abline(h = 0, col = &quot;darkorange&quot;, lwd = 2) This time on the fitted versus residuals plot, for any fitted value, the spread of the residuals is about the same. However, they are not even close to centered at zero! At small and large fitted values the model is underestimating, while at medium fitted values, the model is overestimating. These are systematic errors, not random noise. So the constant variance assumption is met, but the linearity assumption is violated. The form of our model is simply wrong. We’re trying to fit a line to a curve! 13.2.2 Breusch-Pagan Test Constant variance is often called homoscedasticity. Conversely, non-constant variance is called heteroscedasticity. We’ve seen how we can use a fitted versus residuals plot to look for these attributes. While a fitted versus residuals plot can give us an idea about homoscedasticity, sometimes we would prefer a more formal test. There are many tests for constant variance, but here we will present one, the Breusch-Pagan Test. The exact details of the test will be omitted here, but importantly the null and alternative can be considered to be, \\(H_0\\): Homoscedasticity. The errors have constant variance about the true model. \\(H_1\\): Heteroscedasticity. The errors have non-constant variance about the true model. Isn’t that convenient? A test that will specifically test the constant variance assumption. The Breusch-Pagan Test can not be performed by default in R, however the function bptest in the lmtest package implements the test. #install.packages(&quot;lmtest&quot;) library(lmtest) Let’s try it on the three models we fit above. Recall, fit_1 had no violation of assumptions, fit_2 violated the constant variance assumption, but not linearity, fit_3 violated linearity, but not constant variance. bptest(fit_1) ## ## studentized Breusch-Pagan test ## ## data: fit_1 ## BP = 1.0234, df = 1, p-value = 0.3117 For fit_1 we see a large p-value, so we do not reject the null of homoscedasticity, which is what we would expect. bptest(fit_2) ## ## studentized Breusch-Pagan test ## ## data: fit_2 ## BP = 76.693, df = 1, p-value &lt; 2.2e-16 For fit_2 we see a small p-value, so we reject the null of homoscedasticity. The constant variance assumption is violated. This matches our findings with a fitted versus residuals plot. bptest(fit_3) ## ## studentized Breusch-Pagan test ## ## data: fit_3 ## BP = 0.33466, df = 1, p-value = 0.5629 Lastly, for fit_3 we again see a large p-value, so we do not reject the null of homoscedasticity, which matches our findings with a fitted versus residuals plot. 13.2.3 Histograms We have a number of tools for assessing the normality assumption. The most obvious would be to make a histogram of the residuals. If it appears roughly normal, then we’ll believe the errors could truly be normal. par(mfrow = c(1, 3)) hist(resid(fit_1), xlab = &quot;Residuals&quot;, main = &quot;Histogram of Residuals, fit_1&quot;, col = &quot;darkorange&quot;, border = &quot;dodgerblue&quot;, breaks = 20) hist(resid(fit_2), xlab = &quot;Residuals&quot;, main = &quot;Histogram of Residuals, fit_2&quot;, col = &quot;darkorange&quot;, border = &quot;dodgerblue&quot;, breaks = 20) hist(resid(fit_3), xlab = &quot;Residuals&quot;, main = &quot;Histogram of Residuals, fit_3&quot;, col = &quot;darkorange&quot;, border = &quot;dodgerblue&quot;, breaks = 20) Above are histograms for each of the three regression models we have been considering. Notice that the first, for fit_1 appears very normal. The third, for fit_3, appears to be very non-normal. However fit_2 is not as clear. It does have a rough bell shape, however, it also has a very sharp peak. For this reason we will usually use more powerful tools such as Q-Q plots and the Shapiro-Wilk test for assessing the normality of errors. 13.2.4 Q-Q Plots Another visual method for assessing the normality of errors, which is more powerful than a histogram, is a normal quantile-quantile plot, or Q-Q plot for short. In R these are very easy to make. The qqnorm() function plots the points, and the qqline() function adds the necessary line. We create a Q-Q plot for the residuals of fit_1 to check if the errors could truly be normally distributed. qqnorm(resid(fit_1), main = &quot;Normal Q-Q Plot, fit_1&quot;, col = &quot;darkgrey&quot;) qqline(resid(fit_1), col = &quot;dodgerblue&quot;, lwd = 2) In short, if the points of the plot do not closely follow a straight line, this would suggest that the data do not come from a normal distribution. The calculations required to create the plot vary depending on the implementation, but essentially the \\(y\\)-axis is the sorted data (observed, or sample quantiles), and the \\(x\\)-axis is the values we would expect if the data did come from a normal distribution (theoretical quantiles). The Wikipedia page for Normal probability plots gives details on how this is implemented in R if you are interested. Also, to get a better idea of how Q-Q plots work, here is a quick function which creates a Q-Q plot: qq_plot = function(e) { n = length(e) normal_quantiles = qnorm(((1:n - 0.5) / n)) # normal_quantiles = qnorm(((1:n) / (n + 1))) # plot theoretical verus observed quantiles plot(normal_quantiles, sort(e), xlab = c(&quot;Theoretical Quantiles&quot;), ylab = c(&quot;Sample Quantiles&quot;), col = &quot;darkgrey&quot;) title(&quot;Normal Q-Q Plot&quot;) # calculate line through the first and third quartiles slope = (quantile(e, 0.75) - quantile(e, 0.25)) / (qnorm(0.75) - qnorm(0.25)) intercept = quantile(e, 0.25) - slope * qnorm(0.25) # add to existing plot abline(intercept, slope, lty = 2, lwd = 2, col = &quot;dodgerblue&quot;) } We can then verify that it is essentially equivalent to using qqnorm() and qqline() in R. set.seed(420) x = rnorm(100, mean = 0 , sd = 1) par(mfrow = c(1, 2)) qqnorm(x, col = &quot;darkgrey&quot;) qqline(x, lty = 2, lwd = 2, col = &quot;dodgerblue&quot;) qq_plot(x) To get a better idea of what “close to the line” means, we perform a number of simulations, and create Q-Q plots. First we simulate data from a normal distribution with different sample sizes, and each time create a Q-Q plot. par(mfrow = c(1, 3)) set.seed(420) qq_plot(rnorm(10)) qq_plot(rnorm(25)) qq_plot(rnorm(100)) Since this data is sampled from a normal distribution, these are all, by definition, good Q-Q plots. The points are “close to the line” and we would conclude that this data could have been sampled from a normal distribution. Notice in the first plot, one point is somewhat far from the line, but just one point, in combination with the small sample size, is not enough to make us worried. We see with the large sample size, all of the points are rather close to the line. Next, we simulate data from a \\(t\\) distribution with a small degrees of freedom, for different sample sizes. par(mfrow = c(1, 3)) set.seed(420) qq_plot(rt(10, df = 4)) qq_plot(rt(25, df = 4)) qq_plot(rt(100, df = 4)) Recall that as the degrees of freedom for a \\(t\\) distribution become larger, the distribution becomes more and more similar to a normal. Here, using 4 degrees of freedom, we have a distribution that is somewhat normal, it is symmetrical and roughly bell-shaped, however it has “fat tails.” This presents itself clearly in the third panel. While many of the points are close to the line, at the edges, there are large discrepancies. This indicates that the values are too small (negative) or too large (positive) compared to what we would expect for a normal distribution. So for the sample size of 100, we would conclude that the normality assumption is violated. (If these were residuals of a model.) For sample sizes of 10 and 25 we may be suspicious, but not entirely confident. Reading Q-Q plots, is a bit of an art, not completely a science. Next, we simulate data from an exponential distribution. par(mfrow = c(1, 3)) set.seed(420) qq_plot(rexp(10)) qq_plot(rexp(25)) qq_plot(rexp(100)) This is a distribution that is not very similar to a normal, so in all three cases, we see points that are far from the lines, so we would think that the normality assumption is violated. For a better understanding of which Q-Q plots are “good,” repeat the simulations above a number of times (without setting the seed) and pay attention to the differences between those that are simulated from normal, and those that are not. Also consider different sample sizes and distribution parameters. Returning to our three regressions, recall, fit_1 had no violation of assumptions, fit_2 violated the constant variance assumption, but not linearity, fit_3 violated linearity, but not constant variance. We’ll now create a Q-Q plot for each to assess normality of errors. qqnorm(resid(fit_1), main = &quot;Normal Q-Q Plot, fit_1&quot;, col = &quot;darkgrey&quot;) qqline(resid(fit_1), col = &quot;dodgerblue&quot;, lwd = 2) For fit_1, we have a near perfect Q-Q plot. We would believe the errors follow a normal distribution. qqnorm(resid(fit_2), main = &quot;Normal Q-Q Plot, fit_2&quot;, col = &quot;darkgrey&quot;) qqline(resid(fit_2), col = &quot;dodgerblue&quot;, lwd = 2) For fit_2, we have a suspect Q-Q plot. We would probably not believe the errors follow a normal distribution. qqnorm(resid(fit_3), main = &quot;Normal Q-Q Plot, fit_3&quot;, col = &quot;darkgrey&quot;) qqline(resid(fit_3), col = &quot;dodgerblue&quot;, lwd = 2) Lastly, for fit_3, we again have a suspect Q-Q plot. We would probably not believe the errors follow a normal distribution. 13.2.5 Shapiro-Wilk Test Histograms and Q-Q Plots give a nice visual representation of the residuals distribution, however if we are interested in formal testing, there are a number of options available. A commonly used test is the Shapiro–Wilk test, which is implemented in R. set.seed(42) shapiro.test(rnorm(25)) ## ## Shapiro-Wilk normality test ## ## data: rnorm(25) ## W = 0.9499, p-value = 0.2495 shapiro.test(rexp(25)) ## ## Shapiro-Wilk normality test ## ## data: rexp(25) ## W = 0.71164, p-value = 1.05e-05 This gives us the value of the test statistic and its p-value. The null hypothesis assumes the data were sampled from a normal distribution, thus a small p-value indicates we believe there is only a small probability the data could have been sampled from a normal distribution. For details, see: Wikipedia: Shapiro–Wilk test. In the above examples, we see we fail to reject for the data sampled from normal, and reject on the non-normal data, for any reasonable \\(\\alpha\\). Returning again to fit_1, fit_2, and fit_3, we see the result of running shapiro.test() on the residuals of each, returns a result for each that matches decisions based on the Q-Q plots. shapiro.test(resid(fit_1)) ## ## Shapiro-Wilk normality test ## ## data: resid(fit_1) ## W = 0.99858, p-value = 0.9622 shapiro.test(resid(fit_2)) ## ## Shapiro-Wilk normality test ## ## data: resid(fit_2) ## W = 0.93697, p-value = 1.056e-13 shapiro.test(resid(fit_3)) ## ## Shapiro-Wilk normality test ## ## data: resid(fit_3) ## W = 0.97643, p-value = 3.231e-07 13.3 Unusual Observations In addition to checking the assumptions of regression, we also look for any “unusual observations” in the data. Often a small number of data points can have an extremely large influence on a regression, sometimes so much so that the regression assumptions are violated as a result of these points. The following three plots are inspired by an example from Linear Models with R. par(mfrow = c(1, 3)) set.seed(42) ex_data = data.frame(x = 1:10, y = 10:1 + rnorm(n = 10)) ex_model = lm(y ~ x, data = ex_data) # low leverage, large residual, small influence point_1 = c(5.4, 11) ex_data_1 = rbind(ex_data, point_1) model_1 = lm(y ~ x, data = ex_data_1) plot(y ~ x, data = ex_data_1, cex = 2, pch = 20, col = &quot;grey&quot;, main = &quot;Low Leverage, Large Residual, Small Influence&quot;) points(x = point_1[1], y = point_1[2], pch = 1, cex = 4, col = &quot;black&quot;, lwd = 2) abline(ex_model, col = &quot;dodgerblue&quot;, lwd = 2) abline(model_1, lty = 2, col = &quot;darkorange&quot;, lwd = 2) legend(&quot;bottomleft&quot;, c(&quot;Original Data&quot;, &quot;Added Point&quot;), lty = c(1, 2), col = c(&quot;dodgerblue&quot;, &quot;darkorange&quot;)) # high leverage, small residual, small influence point_2 = c(18, -5.7) ex_data_2 = rbind(ex_data, point_2) model_2 = lm(y ~ x, data = ex_data_2) plot(y ~ x, data = ex_data_2, cex = 2, pch = 20, col = &quot;grey&quot;, main = &quot;High Leverage, Small Residual, Small Influence&quot;) points(x = point_2[1], y = point_2[2], pch = 1, cex = 4, col = &quot;black&quot;, lwd = 2) abline(ex_model, col = &quot;dodgerblue&quot;, lwd = 2) abline(model_2, lty = 2, col = &quot;darkorange&quot;, lwd = 2) legend(&quot;bottomleft&quot;, c(&quot;Original Data&quot;, &quot;Added Point&quot;), lty = c(1, 2), col = c(&quot;dodgerblue&quot;, &quot;darkorange&quot;)) # high leverage, large residual, large influence point_3 = c(14, 5.1) ex_data_3 = rbind(ex_data, point_3) model_3 = lm(y ~ x, data = ex_data_3) plot(y ~ x, data = ex_data_3, cex = 2, pch = 20, col = &quot;grey&quot;, ylim = c(-3, 12), main = &quot;High Leverage, Large Residual, Large Influence&quot;) points(x = point_3[1], y = point_3[2], pch = 1, cex = 4, col = &quot;black&quot;, lwd = 2) abline(ex_model, col = &quot;dodgerblue&quot;, lwd = 2) abline(model_3, lty = 2, col = &quot;darkorange&quot;, lwd = 2) legend(&quot;bottomleft&quot;, c(&quot;Original Data&quot;, &quot;Added Point&quot;), lty = c(1, 2), col = c(&quot;dodgerblue&quot;, &quot;darkorange&quot;)) The blue solid line in each plot is a regression fit to the 10 original data points stored in ex_data. The dashed orange line in each plot is the result of adding a single point to the original data in ex_data. This additional point is indicated by the circled point. The slope of the regression for the original ten points, the solid blue line, is given by: coef(ex_model)[2] ## x ## -0.9696033 The added point in the first plot has a small effect on the slope, which becomes: coef(model_1)[2] ## x ## -0.9749534 We will say that this point has low leverage, is an outlier due to its large residual, but has small influence. The added point in the second plot also has a small effect on the slope, which is: coef(model_2)[2] ## x ## -0.9507397 We will say that this point has high leverage, is not an outlier due to its small residual, and has a very small influence. Lastly, the added point in the third plot has a large effect on the slope, which is now: coef(model_3)[2] ## x ## -0.5892241 This added point is influential. It both has high leverage, and is an outlier due to its large residual. We’ve now mentioned three new concepts: leverage, outliers, and influential points, each of which we will discuss in detail. 13.3.1 Leverage A data point with high leverage, is a data point that could have a large influence when fitting the model. Recall that, \\[ \\hat{\\beta} = \\left(X^\\top X \\right)^{-1} X^\\top y. \\] Thus, \\[ \\hat{y} = X \\hat{\\beta} = X \\left(X^\\top X \\right)^{-1} X^\\top y \\] Now we define, \\[ H = X \\left(X^\\top X\\right)^{-1} X^\\top \\] which we will refer to as the hat matrix. The hat matrix is used to project onto the subspace spanned by the columns of \\(X\\). It is also simply known as a projection matrix. The hat matrix is a matrix that takes the original \\(y\\) values, and adds a hat! \\[ \\hat{y} = H y \\] The diagonal elements of this matrix are called the leverages \\[ H_{ii} = h_i, \\] where \\(h_i\\) is the leverage for the \\(i\\)th observation. Large values of \\(h_i\\) indicate extreme values in \\(X\\), which may influence regression. Note that leverages only depend on \\(X\\). Here, \\(p\\), the number of \\(\\beta\\)s, is also the trace (and rank) of the hat matrix. \\[ \\sum_{i = 1}^n h_i = p \\] What is a value of \\(h_i\\) that would be considered large? There is no exact answer to this question. A common heuristic would be to compare each leverage to two times the average leverage. A leverage larger than this is considered an observation to be aware of. That is, if \\[ h_i &gt; 2 \\bar{h} \\] we say that observation \\(i\\) has large leverage. Here, \\[ \\bar{h} = \\frac{\\sum_{i = 1}^n h_i}{n} = \\frac{p}{n}. \\] For simple linear regression, the leverage for each point is given by \\[ h_i = \\frac{1}{n} + \\frac{(x_i - \\bar{x})^2}{S_{xx}}. \\] This expression should be familiar. (Think back to inference for SLR.) It suggests that the large leverages occur when \\(x\\) values are far from their mean. Recall that the regression goes through the point \\((\\bar{x},\\bar{y})\\). There are multiple ways to find leverages in R. lev_ex = data.frame( x1 = c(0, 11, 11, 7, 4, 10, 5, 8), x2 = c(1, 5, 4, 3, 1, 4, 4, 2), y = c(11, 15, 13, 14, 0, 19, 16, 8)) plot(x2 ~ x1, data = lev_ex, cex = 2) points(7, 3, pch = 20, col = &quot;red&quot;, cex = 2) Here we’ve created some multivariate data. Notice that we have plotted the \\(x\\) values, not the \\(y\\) values. The red point is \\((7, 3)\\) which is the mean of x1 and the mean of x2 respectively. We could calculate the leverages using the expressions defined above. We first create the \\(X\\) matrix, then calculate \\(H\\) as defined, and extract the diagonal elements. X = cbind(rep(1, 8), lev_ex$x1, lev_ex$x2) H = X %*% solve(t(X) %*% X) %*% t(X) diag(H) ## [1] 0.6000 0.3750 0.2875 0.1250 0.4000 0.2125 0.5875 0.4125 Notice here, we have two predictors, so the regression would have 3 \\(\\beta\\) parameters, so the sum of the diagonal elements is 3. sum(diag(H)) ## [1] 3 Alternatively, the method we will use more often, is to simply fit a regression, then use the hatvalues() function, which returns the leverages. lev_fit = lm(y ~ ., data = lev_ex) hatvalues(lev_fit) ## 1 2 3 4 5 6 7 8 ## 0.6000 0.3750 0.2875 0.1250 0.4000 0.2125 0.5875 0.4125 Again, note that here we have “used” the \\(y\\) values to fit the regression, but R still ignores them when calculating the leverages, as leverages only depend on the \\(x\\) values. coef(lev_fit) ## (Intercept) x1 x2 ## 3.7 -0.7 4.4 Let’s see what happens to these coefficients when we modify the y value of the point with the highest leverage. which.max(hatvalues(lev_fit)) ## 1 ## 1 lev_ex[which.max(hatvalues(lev_fit)),] ## x1 x2 y ## 1 0 1 11 We see that the original y value is 11. We’ll create a copy of the data, and modify this point to have a y value of 20. lev_ex_1 = lev_ex lev_ex_1$y[1] = 20 lm(y ~ ., data = lev_ex_1) ## ## Call: ## lm(formula = y ~ ., data = lev_ex_1) ## ## Coefficients: ## (Intercept) x1 x2 ## 8.875 -1.375 4.625 Notice the large changes in the coefficients. Also notice that each of the coefficients has changed in some way. Note that the leverages of the points would not have changed, as we have not modified any of the \\(x\\) values. Now let’s see what happens to these coefficients when we modify the y value of the point with the lowest leverage. which.min(hatvalues(lev_fit)) ## 4 ## 4 lev_ex[which.min(hatvalues(lev_fit)),] ## x1 x2 y ## 4 7 3 14 We see that the original y value is 14. We’ll again create a copy of the data, and modify this point to have a y value of 30. lev_ex_2 = lev_ex lev_ex_2$y[4] = 30 lm(y ~ ., data = lev_ex_2) ## ## Call: ## lm(formula = y ~ ., data = lev_ex_2) ## ## Coefficients: ## (Intercept) x1 x2 ## 5.7 -0.7 4.4 This time despite a large change in the y value, there is only small change in the coefficients. Also, only the intercept has changed! mean(lev_ex$x1) ## [1] 7 mean(lev_ex$x2) ## [1] 3 lev_ex[4,] ## x1 x2 y ## 4 7 3 14 Notice that this point was the mean of both of the predictors. Returning to our three plots, each with an added point, we can calculate the leverages for each. Note that the 11th data point each time is the added data point. hatvalues(model_1) ## 1 2 3 4 5 6 7 ## 0.33534597 0.23860732 0.16610842 0.11784927 0.09382988 0.09405024 0.11851036 ## 8 9 10 11 ## 0.16721022 0.24014985 0.33732922 0.09100926 hatvalues(model_2) ## 1 2 3 4 5 6 7 ## 0.23238866 0.18663968 0.14979757 0.12186235 0.10283401 0.09271255 0.09149798 ## 8 9 10 11 ## 0.09919028 0.11578947 0.14129555 0.66599190 hatvalues(model_3) ## 1 2 3 4 5 6 7 ## 0.27852761 0.21411043 0.16319018 0.12576687 0.10184049 0.09141104 0.09447853 ## 8 9 10 11 ## 0.11104294 0.14110429 0.18466258 0.49386503 Are any of these large? hatvalues(model_1) &gt; 2 * mean(hatvalues(model_1)) ## 1 2 3 4 5 6 7 8 9 10 11 ## FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE hatvalues(model_2) &gt; 2 * mean(hatvalues(model_2)) ## 1 2 3 4 5 6 7 8 9 10 11 ## FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE hatvalues(model_3) &gt; 2 * mean(hatvalues(model_3)) ## 1 2 3 4 5 6 7 8 9 10 11 ## FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE We see that in the second and third plots, the added point is a point of high leverage. Recall that only in the third plot did that have an influence on the regression. To understand why, we’ll need to discuss outliers. 13.3.2 Outliers Outliers are points which do not fit the model well. They may or may not have a large effect on the model. To identify outliers, we will look for observations with large residuals. Note, \\[ e = y - \\hat{y} = Iy - Hy = (I - H) y \\] Then, under the assumptions of linear regression, \\[ \\text{Var}(e_i) = (1 - h_i) \\sigma^2 \\] and thus estimating \\(\\sigma^2\\) with \\(s_e^2\\) gives \\[ \\text{SE}[e_i] = s_e \\sqrt{(1 - h_i)}. \\] We can then look at the standardized residual for each observation, \\(i = 1, 2, \\ldots n\\), \\[ r_i = \\frac{e_i}{s_e\\sqrt{1 - h_i}} \\overset{approx}{\\sim} N(\\mu = 0, \\sigma^ 2 = 1) \\] when \\(n\\) is large. We can use this fact to identify “large” residuals. For example, standardized residuals greater than 2 in magnitude should only happen approximately 5 percent of the time. Returning again to our three plots, each with an added point, we can calculate the residuals and standardized residuals for each. Standardized residuals can be obtained in R by using rstandard() where we would normally use resid(). resid(model_1) ## 1 2 3 4 5 6 7 ## 0.4949887 -1.4657145 -0.5629345 -0.3182468 -0.5718877 -1.1073271 0.4852728 ## 8 9 10 11 ## -1.1459548 0.9420814 -1.1641029 4.4138254 rstandard(model_1) ## 1 2 3 4 5 6 7 ## 0.3464701 -0.9585470 -0.3517802 -0.1933575 -0.3428264 -0.6638841 0.2949482 ## 8 9 10 11 ## -0.7165857 0.6167268 -0.8160389 2.6418234 rstandard(model_1)[abs(rstandard(model_1)) &gt; 2] ## 11 ## 2.641823 In the first plot, we see that the 11th point, the added point, is a large standardized residual. resid(model_2) ## 1 2 3 4 5 6 ## 1.03288292 -0.95203397 -0.07346766 0.14700626 -0.13084829 -0.69050140 ## 7 8 9 10 11 ## 0.87788484 -0.77755647 1.28626601 -0.84413207 0.12449986 rstandard(model_2) ## 1 2 3 4 5 6 ## 1.41447023 -1.26655590 -0.09559792 0.18822094 -0.16574677 -0.86977220 ## 7 8 9 10 11 ## 1.10506546 -0.98294409 1.64121833 -1.09295417 0.25846620 rstandard(model_2)[abs(rstandard(model_2)) &gt; 2] ## named numeric(0) In the second plot, we see that there are no points with large standardized residuals. resid(model_3) ## 1 2 3 4 5 6 ## 2.30296166 -0.04347087 0.47357980 0.33253808 -0.30683212 -1.22800087 ## 7 8 9 10 11 ## -0.02113027 -2.03808722 -0.33578039 -2.82769411 3.69191633 rstandard(model_3) ## 1 2 3 4 5 6 ## 1.41302755 -0.02555591 0.26980722 0.18535382 -0.16873216 -0.67141143 ## 7 8 9 10 11 ## -0.01157256 -1.12656475 -0.18882474 -1.63206526 2.70453408 rstandard(model_3)[abs(rstandard(model_3)) &gt; 2] ## 11 ## 2.704534 In the last plot, we see that the 11th point, the added point, is a large standardized residual. Recall that the added point in plots two and three were both high leverage, but now only the point in plot three has a large residual. We will now combine this information and discuss influence. 13.3.3 Influence As we have now seen in the three plots, some outliers only change the regression a small amount (plot one) and some outliers have a large effect on the regression (plot three). Observations that fall into the latter category, points with (some combination of) high leverage and large residual, we will call influential. A common measure of influence is Cook’s Distance, which is defined as \\[ D_i = \\frac{1}{p}r_i^2\\frac{h_i}{1-{h_i}}. \\] Notice that this is a function of both leverage and standardized residuals. A Cook’s Distance is often considered large if \\[ D_i &gt; \\frac{4}{n} \\] and an observation with a large Cook’s Distance is called influential. This is again simply a heuristic, and not an exact rule. The Cook’s distance for each point of a regression can be calculated using cooks.distance() which is a default function in R. Let’s look for influential points in the three plots we had been considering. Recall that the circled points in each plot have different characteristics: Plot One: low leverage, large residual. Plot Two: high leverage, small residual. Plot Three: high leverage, large residual. We’ll now directly check if each of these is influential. cooks.distance(model_1)[11] &gt; 4 / length(cooks.distance(model_1)) ## 11 ## FALSE cooks.distance(model_2)[11] &gt; 4 / length(cooks.distance(model_2)) ## 11 ## FALSE cooks.distance(model_3)[11] &gt; 4 / length(cooks.distance(model_3)) ## 11 ## TRUE And, as expected, the added point in the third plot, with high leverage and a large residual, is considered influential! 13.4 Data Analysis Examples 13.4.1 Good Diagnostics Last chapter we fit an additive regression to the mtcars data with mpg as the response and hp and am as predictors. Let’s perform some diagnostics on this model. First, fit the model as we did last chapter. mpg_hp_add = lm(mpg ~ hp + am, data = mtcars) plot(fitted(mpg_hp_add), resid(mpg_hp_add), col = &quot;grey&quot;, pch = 20, xlab = &quot;Fitted&quot;, ylab = &quot;Residual&quot;, main = &quot;mtcars: Fitted versus Residuals&quot;) abline(h = 0, col = &quot;darkorange&quot;, lwd = 2) The fitted versus residuals plot looks good. We don’t see any obvious pattern, and the variance looks roughly constant. (Maybe a little larger for large fitted values, but not enough to worry about.) bptest(mpg_hp_add) ## ## studentized Breusch-Pagan test ## ## data: mpg_hp_add ## BP = 7.5858, df = 2, p-value = 0.02253 The Breusch-Pagan test verifies this, at least for a small \\(\\alpha\\) value. qqnorm(resid(mpg_hp_add), col = &quot;darkgrey&quot;) qqline(resid(mpg_hp_add), col = &quot;dodgerblue&quot;, lwd = 2) The Q-Q plot looks extremely good and the Shapiro-Wilk test agrees. shapiro.test(resid(mpg_hp_add)) ## ## Shapiro-Wilk normality test ## ## data: resid(mpg_hp_add) ## W = 0.96485, p-value = 0.3706 sum(hatvalues(mpg_hp_add) &gt; 2 * mean(hatvalues(mpg_hp_add))) ## [1] 2 We see that there are two points of large leverage. sum(abs(rstandard(mpg_hp_add)) &gt; 2) ## [1] 1 There is also one point with a large residual. Do these result in any points that are considered influential? cd_mpg_hp_add = cooks.distance(mpg_hp_add) sum(cd_mpg_hp_add &gt; 4 / length(cd_mpg_hp_add)) ## [1] 2 large_cd_mpg = cd_mpg_hp_add &gt; 4 / length(cd_mpg_hp_add) cd_mpg_hp_add[large_cd_mpg] ## Toyota Corolla Maserati Bora ## 0.1772555 0.3447994 We find two influential points. Interestingly, they are very different cars. coef(mpg_hp_add) ## (Intercept) hp am ## 26.5849137 -0.0588878 5.2770853 Since the diagnostics looked good, there isn’t much need to worry about these two points, but let’s see how much the coefficients change if we remove them. mpg_hp_add_fix = lm(mpg ~ hp + am, data = mtcars, subset = cd_mpg_hp_add &lt;= 4 / length(cd_mpg_hp_add)) coef(mpg_hp_add_fix) ## (Intercept) hp am ## 27.22190933 -0.06286249 4.29765867 It seems there isn’t much of a change in the coefficients as a result of removing the supposed influential points. Notice we did not create a new dataset to accomplish this. We instead used the subset argument to lm(). Think about what the code cd_mpg_hp_add &lt;= 4 / length(cd_mpg_hp_add) does here. par(mfrow = c(2, 2)) plot(mpg_hp_add) Notice that calling plot() on a variable which stores an object created by lm() outputs four diagnostic plots by default. Use ?plot.lm to learn more. The first two should already be familiar. 13.4.2 Suspect Diagnostics Let’s consider the model big_model from last chapter which was fit to the autompg dataset. It used mpg as the response, and considered many interaction terms between the predictors disp, hp, and domestic. str(autompg) ## &#39;data.frame&#39;: 383 obs. of 9 variables: ## $ mpg : num 18 15 18 16 17 15 14 14 14 15 ... ## $ cyl : Factor w/ 3 levels &quot;4&quot;,&quot;6&quot;,&quot;8&quot;: 3 3 3 3 3 3 3 3 3 3 ... ## $ disp : num 307 350 318 304 302 429 454 440 455 390 ... ## $ hp : num 130 165 150 150 140 198 220 215 225 190 ... ## $ wt : num 3504 3693 3436 3433 3449 ... ## $ acc : num 12 11.5 11 12 10.5 10 9 8.5 10 8.5 ... ## $ year : int 70 70 70 70 70 70 70 70 70 70 ... ## $ origin : int 1 1 1 1 1 1 1 1 1 1 ... ## $ domestic: num 1 1 1 1 1 1 1 1 1 1 ... big_model = lm(mpg ~ disp * hp * domestic, data = autompg) qqnorm(resid(big_model), col = &quot;darkgrey&quot;) qqline(resid(big_model), col = &quot;dodgerblue&quot;, lwd = 2) shapiro.test(resid(big_model)) ## ## Shapiro-Wilk normality test ## ## data: resid(big_model) ## W = 0.96161, p-value = 1.824e-08 Here both the Q-Q plot, and the Shapiro-Wilk test suggest that the normality assumption is violated. big_mod_cd = cooks.distance(big_model) sum(big_mod_cd &gt; 4 / length(big_mod_cd)) ## [1] 31 Here, we find 31, so perhaps removing them will help! big_model_fix = lm(mpg ~ disp * hp * domestic, data = autompg, subset = big_mod_cd &lt; 4 / length(big_mod_cd)) qqnorm(resid(big_model_fix), col = &quot;grey&quot;) qqline(resid(big_model_fix), col = &quot;dodgerblue&quot;, lwd = 2) shapiro.test(resid(big_model_fix)) ## ## Shapiro-Wilk normality test ## ## data: resid(big_model_fix) ## W = 0.99035, p-value = 0.02068 Removing these points results in a much better Q-Q plot, and now Shapiro-Wilk fails to reject for a low \\(\\alpha\\). We’ve now seen that sometimes modifying the data can fix issues with regression. However, next chapter, instead of modifying the data, we will modify the model via transformations. 13.5 R Markdown The R Markdown file for this chapter can be found here: diagnostics.Rmd The file was created using R version 4.3.2. "],["transformations.html", "Chapter 14 Transformations 14.1 Response Transformation 14.2 Predictor Transformation Response Transformations Predictor Transformations 14.3 R Markdown", " Chapter 14 Transformations “Give me a lever long enough and a fulcrum on which to place it, and I shall move the world.” — Archimedes Please note: some data currently used in this chapter was used, changed, and passed around over the years in STAT 420 at UIUC. Its original sources, if they exist, are at this time unknown to the author. As a result, they should only be considered for use with STAT 420. Going forward they will likely be replaced with alternative sourceable data that illustrates the same concepts. At the end of this chapter you can find code seen in videos for Week 8 for STAT 420 in the MCS-DS program. It is currently in the process of being merged into the narrative of this chapter. After reading this chapter you will be able to: Understand the concept of a variance stabilizing transformation. Use transformations of the response to improve regression models. Use polynomial terms as predictors to fit more flexible regression models. Last chapter we checked the assumptions of regression models and looked at ways to diagnose possible issues. This chapter we will use transformations of both response and predictor variables in order to correct issues with model diagnostics, and to also potentially simply make a model fit data better. 14.1 Response Transformation Let’s look at some (fictional) salary data from the (fictional) company Initech. We will try to model salary as a function of years of experience. The data can be found in initech.csv. initech = read.csv(&quot;data/initech.csv&quot;) plot(salary ~ years, data = initech, col = &quot;grey&quot;, pch = 20, cex = 1.5, main = &quot;Salaries at Initech, By Seniority&quot;) We first fit a simple linear model. initech_fit = lm(salary ~ years, data = initech) summary(initech_fit) ## ## Call: ## lm(formula = salary ~ years, data = initech) ## ## Residuals: ## Min 1Q Median 3Q Max ## -57225 -18104 241 15589 91332 ## ## Coefficients: ## Estimate Std. Error t value Pr(&gt;|t|) ## (Intercept) 5302 5750 0.922 0.359 ## years 8637 389 22.200 &lt;2e-16 *** ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 ## ## Residual standard error: 27360 on 98 degrees of freedom ## Multiple R-squared: 0.8341, Adjusted R-squared: 0.8324 ## F-statistic: 492.8 on 1 and 98 DF, p-value: &lt; 2.2e-16 This model appears significant, but does it meet the model assumptions? plot(salary ~ years, data = initech, col = &quot;grey&quot;, pch = 20, cex = 1.5, main = &quot;Salaries at Initech, By Seniority&quot;) abline(initech_fit, col = &quot;darkorange&quot;, lwd = 2) Adding the fitted line to the plot, we see that the linear relationship appears correct. par(mfrow = c(1, 2)) plot(fitted(initech_fit), resid(initech_fit), col = &quot;grey&quot;, pch = 20, xlab = &quot;Fitted&quot;, ylab = &quot;Residuals&quot;, main = &quot;Fitted versus Residuals&quot;) abline(h = 0, col = &quot;darkorange&quot;, lwd = 2) qqnorm(resid(initech_fit), main = &quot;Normal Q-Q Plot&quot;, col = &quot;darkgrey&quot;) qqline(resid(initech_fit), col = &quot;dodgerblue&quot;, lwd = 2) However, from the fitted versus residuals plot it appears there is non-constant variance. Specifically, the variance increases as the fitted value increases. 14.1.1 Variance Stabilizing Transformations Recall the fitted value is our estimate of the mean at a particular value of \\(x\\). Under our usual assumptions, \\[ \\epsilon \\sim N(0,\\sigma^2) \\] and thus \\[ \\text{Var}[Y | X = x] = \\sigma^2 \\] which is a constant value for any value of \\(x\\). However, here we see that the variance is a function of the mean, \\[ \\text{Var}[Y \\mid X = x] = h(\\text{E}[Y \\mid X = x]). \\] In this case, \\(h\\) is some increasing function. In order to correct for this, we would like to find some function of \\(Y\\), \\(g(Y)\\) such that, \\[ \\text{Var}[g(Y) \\mid X = x] = c \\] where \\(c\\) is a constant that does not depend on the mean, \\(\\text{E}[Y \\mid X = x]\\). A transformation that accomplishes this is called a variance stabilizing transformation. A common variance stabilizing transformation (VST) when we see increasing variance in a fitted versus residuals plot is \\(\\log(Y)\\). Also, if the values of a variable range over more than one order of magnitude and the variable is strictly positive, then replacing the variable by its logarithm is likely to be helpful. A reminder, that for our purposes, \\(\\log\\) and \\(\\ln\\) are both the natural log. R uses log to mean the natural log, unless a different base is specified. We will now use a model with a log transformed response for the Initech data, \\[ \\log(Y_i) = \\beta_0 + \\beta_1 x_i + \\epsilon_i. \\] Note, if we re-scale the model from a log scale back to the original scale of the data, we now have \\[ Y_i = \\exp(\\beta_0 + \\beta_1 x_i) \\cdot \\exp(\\epsilon_i) \\] which has the errors entering the model in a multiplicative fashion. Fitting this model in R requires only a minor modification to our formula specification. initech_fit_log = lm(log(salary) ~ years, data = initech) Note that while log(y) is considered the new response variable, we do not actually create a new variable in R, but simply transform the variable inside the model formula. plot(log(salary) ~ years, data = initech, col = &quot;grey&quot;, pch = 20, cex = 1.5, main = &quot;Salaries at Initech, By Seniority&quot;) abline(initech_fit_log, col = &quot;darkorange&quot;, lwd = 2) Plotting the data on the transformed log scale and adding the fitted line, the relationship again appears linear, and we can already see that the variation about the fitted line looks constant. plot(salary ~ years, data = initech, col = &quot;grey&quot;, pch = 20, cex = 1.5, main = &quot;Salaries at Initech, By Seniority&quot;) curve(exp(initech_fit_log$coef[1] + initech_fit_log$coef[2] * x), from = 0, to = 30, add = TRUE, col = &quot;darkorange&quot;, lwd = 2) By plotting the data on the original scale, and adding the fitted regression, we see an exponential relationship. However, this is still a linear model, since the new transformed response, \\(\\log(y)\\), is still a linear combination of the predictors. par(mfrow = c(1, 2)) plot(fitted(initech_fit_log), resid(initech_fit_log), col = &quot;grey&quot;, pch = 20, xlab = &quot;Fitted&quot;, ylab = &quot;Residuals&quot;, main = &quot;Fitted versus Residuals&quot;) abline(h = 0, col = &quot;darkorange&quot;, lwd = 2) qqnorm(resid(initech_fit_log), main = &quot;Normal Q-Q Plot&quot;, col = &quot;darkgrey&quot;) qqline(resid(initech_fit_log), col = &quot;dodgerblue&quot;, lwd = 2) The fitted versus residuals plot looks much better. It appears the constant variance assumption is no longer violated. Comparing the RMSE using the original and transformed response, we also see that the log transformed model simply fits better, with a smaller average squared error. sqrt(mean(resid(initech_fit) ^ 2)) ## [1] 27080.16 sqrt(mean(resid(initech_fit_log) ^ 2)) ## [1] 0.1934907 But wait, that isn’t fair, this difference is simply due to the different scales being used. sqrt(mean((initech$salary - fitted(initech_fit)) ^ 2)) ## [1] 27080.16 sqrt(mean((initech$salary - exp(fitted(initech_fit_log))) ^ 2)) ## [1] 24280.36 Transforming the fitted values of the log model back to the data scale, we do indeed see that it fits better! summary(initech_fit_log) ## ## Call: ## lm(formula = log(salary) ~ years, data = initech) ## ## Residuals: ## Min 1Q Median 3Q Max ## -0.57022 -0.13560 0.03048 0.14157 0.41366 ## ## Coefficients: ## Estimate Std. Error t value Pr(&gt;|t|) ## (Intercept) 10.48381 0.04108 255.18 &lt;2e-16 *** ## years 0.07888 0.00278 28.38 &lt;2e-16 *** ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 ## ## Residual standard error: 0.1955 on 98 degrees of freedom ## Multiple R-squared: 0.8915, Adjusted R-squared: 0.8904 ## F-statistic: 805.2 on 1 and 98 DF, p-value: &lt; 2.2e-16 Again, the transformed response is a linear combination of the predictors, \\[ \\log(\\hat{y}(x)) = \\hat{\\beta}_0 + \\hat{\\beta}_1 x = 10.484 + 0.079x. \\] But now, if we re-scale the data from a log scale back to the original scale of the data, we now have \\[ \\hat{y}(x) = \\exp(\\hat{\\beta}_0) \\exp(\\hat{\\beta}_1 x) = \\exp(10.484)\\exp(0.079x). \\] We see that for every one additional year of experience, average salary increases \\(\\exp(0.079) = 1.0822\\) times. We are now multiplying, not adding. While using a \\(\\log\\) transform is possibly the most common response variable transformation, many others exist. We will now consider a family of transformations and choose the best from among them, which includes the \\(\\log\\) transform. 14.1.2 Box-Cox Transformations The Box-Cox method considers a family of transformations on strictly positive response variables, \\[ g_\\lambda(y) = \\left\\{ \\begin{array}{lr}\\displaystyle\\frac{y^\\lambda - 1}{\\lambda} &amp; \\lambda \\neq 0\\\\ &amp; \\\\ \\log(y) &amp; \\lambda = 0 \\end{array} \\right. \\] The \\(\\lambda\\) parameter is chosen by numerically maximizing the log-likelihood, \\[ L(\\lambda) = -\\frac{n}{2}\\log(RSS_\\lambda / n) + (\\lambda -1)\\sum \\log(y_i). \\] A \\(100(1 - \\alpha)\\%\\) confidence interval for \\(\\lambda\\) is, \\[ \\left\\{ \\lambda : L(\\lambda) &gt; L(\\hat{\\lambda}) - \\frac{1}{2}\\chi_{1,\\alpha}^2 \\right\\} \\] which R will plot for us to help quickly select an appropriate \\(\\lambda\\) value. We often choose a “nice” value from within the confidence interval, instead of the value of \\(\\lambda\\) that truly maximizes the likelihood. library(MASS) library(faraway) Here we need the MASS package for the boxcox() function, and we will consider a couple of datasets from the faraway package. First we will use the savings dataset as an example of using the Box-Cox method to justify the use of no transformation. We fit an additive multiple regression model with sr as the response and each of the other variables as predictors. savings_model = lm(sr ~ ., data = savings) We then use the boxcox() function to find the best transformation of the form considered by the Box-Cox method. boxcox(savings_model, plotit = TRUE) R automatically plots the log-Likelihood as a function of possible \\(\\lambda\\) values. It indicates both the value that maximizes the log-likelihood, as well as a confidence interval for the \\(\\lambda\\) value that maximizes the log-likelihood. boxcox(savings_model, plotit = TRUE, lambda = seq(0.5, 1.5, by = 0.1)) Note that we can specify a range of \\(\\lambda\\) values to consider and thus be plotted. We often specify a range that is more visually interesting. Here we see that \\(\\lambda = 1\\) is both in the confidence interval, and is extremely close to the maximum. This suggests a transformation of the form \\[ \\frac{y^\\lambda - 1}{\\lambda} = \\frac{y^1 - 1}{1} = y - 1. \\] This is essentially not a transformation. It would not change the variance or make the model fit better. By subtracting 1 from every value, we would only change the intercept of the model, and the resulting errors would be the same. plot(fitted(savings_model), resid(savings_model), col = &quot;dodgerblue&quot;, pch = 20, cex = 1.5, xlab = &quot;Fitted&quot;, ylab = &quot;Residuals&quot;) abline(h = 0, lty = 2, col = &quot;darkorange&quot;, lwd = 2) Looking at a fitted versus residuals plot verifies that there likely are not any issues with the assumptions of this model, which Breusch-Pagan and Shapiro-Wilk tests verify. library(lmtest) bptest(savings_model) ## ## studentized Breusch-Pagan test ## ## data: savings_model ## BP = 4.9852, df = 4, p-value = 0.2888 shapiro.test(resid(savings_model)) ## ## Shapiro-Wilk normality test ## ## data: resid(savings_model) ## W = 0.98698, p-value = 0.8524 Now we will use the gala dataset as an example of using the Box-Cox method to justify a transformation other than \\(\\log\\). We fit an additive multiple regression model with Species as the response and most of the other variables as predictors. gala_model = lm(Species ~ Area + Elevation + Nearest + Scruz + Adjacent, data = gala) plot(fitted(gala_model), resid(gala_model), col = &quot;dodgerblue&quot;, pch = 20, cex = 1.5, xlab = &quot;Fitted&quot;, ylab = &quot;Residuals&quot;) abline(h = 0, lty = 2, col = &quot;darkorange&quot;, lwd = 2) Even though there is not a lot of data for large fitted values, it still seems very clear that the constant variance assumption is violated. boxcox(gala_model, lambda = seq(-0.25, 0.75, by = 0.05), plotit = TRUE) Using the Box-Cox method, we see that \\(\\lambda = 0.3\\) is both in the confidence interval, and is extremely close to the maximum, which suggests a transformation of the form \\[ \\frac{y^\\lambda - 1}{\\lambda} = \\frac{y^{0.3} - 1}{0.3}. \\] We then fit a model with this transformation applied to the response. gala_model_cox = lm((((Species ^ 0.3) - 1) / 0.3) ~ Area + Elevation + Nearest + Scruz + Adjacent, data = gala) plot(fitted(gala_model_cox), resid(gala_model_cox), col = &quot;dodgerblue&quot;, pch = 20, cex = 1.5, xlab = &quot;Fitted&quot;, ylab = &quot;Residuals&quot;) abline(h = 0, lty = 2, col = &quot;darkorange&quot;, lwd = 2) The resulting fitted versus residuals plot looks much better! Lastly, we return to the initech data, and the initech_fit model we had used earlier. Recall that this was the untransformed model, that we used a \\(\\log\\) transform to fix. boxcox(initech_fit) Using the Box-Cox method, we see that \\(\\lambda = 0\\) is both in the interval, and extremely close to the maximum, which suggests a transformation of the form \\[ \\log(y). \\] So the Box-Cox method justifies our previous choice of a \\(\\log\\) transform! 14.2 Predictor Transformation In addition to transformation of the response variable, we can also consider transformations of predictor variables. Sometimes these transformations can help with violation of model assumptions, and other times they can be used to simply fit a more flexible model. str(autompg) ## &#39;data.frame&#39;: 383 obs. of 9 variables: ## $ mpg : num 18 15 18 16 17 15 14 14 14 15 ... ## $ cyl : Factor w/ 3 levels &quot;4&quot;,&quot;6&quot;,&quot;8&quot;: 3 3 3 3 3 3 3 3 3 3 ... ## $ disp : num 307 350 318 304 302 429 454 440 455 390 ... ## $ hp : num 130 165 150 150 140 198 220 215 225 190 ... ## $ wt : num 3504 3693 3436 3433 3449 ... ## $ acc : num 12 11.5 11 12 10.5 10 9 8.5 10 8.5 ... ## $ year : int 70 70 70 70 70 70 70 70 70 70 ... ## $ origin : int 1 1 1 1 1 1 1 1 1 1 ... ## $ domestic: num 1 1 1 1 1 1 1 1 1 1 ... Recall the autompg dataset from the previous chapter. Here we will attempt to model mpg as a function of hp. par(mfrow = c(1, 2)) plot(mpg ~ hp, data = autompg, col = &quot;dodgerblue&quot;, pch = 20, cex = 1.5) mpg_hp = lm(mpg ~ hp, data = autompg) abline(mpg_hp, col = &quot;darkorange&quot;, lwd = 2) plot(fitted(mpg_hp), resid(mpg_hp), col = &quot;dodgerblue&quot;, pch = 20, cex = 1.5, xlab = &quot;Fitted&quot;, ylab = &quot;Residuals&quot;) abline(h = 0, lty = 2, col = &quot;darkorange&quot;, lwd = 2) We first attempt SLR, but we see a rather obvious pattern in the fitted versus residuals plot, which includes increasing variance, so we attempt a \\(\\log\\) transform of the response. par(mfrow = c(1, 2)) plot(log(mpg) ~ hp, data = autompg, col = &quot;dodgerblue&quot;, pch = 20, cex = 1.5) mpg_hp_log = lm(log(mpg) ~ hp, data = autompg) abline(mpg_hp_log, col = &quot;darkorange&quot;, lwd = 2) plot(fitted(mpg_hp_log), resid(mpg_hp_log), col = &quot;dodgerblue&quot;, pch = 20, cex = 1.5, xlab = &quot;Fitted&quot;, ylab = &quot;Residuals&quot;) abline(h = 0, lty = 2, col = &quot;darkorange&quot;, lwd = 2) After performing the \\(\\log\\) transform of the response, we still have some of the same issues with the fitted versus response. Now, we will try also \\(\\log\\) transforming the predictor. par(mfrow = c(1, 2)) plot(log(mpg) ~ log(hp), data = autompg, col = &quot;dodgerblue&quot;, pch = 20, cex = 1.5) mpg_hp_loglog = lm(log(mpg) ~ log(hp), data = autompg) abline(mpg_hp_loglog, col = &quot;darkorange&quot;, lwd = 2) plot(fitted(mpg_hp_loglog), resid(mpg_hp_loglog), col = &quot;dodgerblue&quot;, pch = 20, cex = 1.5, xlab = &quot;Fitted&quot;, ylab = &quot;Residuals&quot;) abline(h = 0, lty = 2, col = &quot;darkorange&quot;, lwd = 2) Here, our fitted versus residuals plot looks good. 14.2.1 Polynomials Another very common “transformation” of a predictor variable is the use of polynomial transformations. They are extremely useful as they allow for more flexible models, but do not change the units of the variables. It should come as no surprise that sales of a product are related to the advertising budget for the product, but there are diminishing returns. A company cannot always expect linear returns based on an increased advertising budget. Consider monthly data for the sales of Initech widgets, \\(y\\), as a function of Initech’s advertising expenditure for said widget, \\(x\\), both in ten thousand dollars. The data can be found in marketing.csv. marketing = read.csv(&quot;data/marketing.csv&quot;) plot(sales ~ advert, data = marketing, xlab = &quot;Advert Spending (in $10,000)&quot;, ylab = &quot;Sales (in $10,000)&quot;, pch = 20, cex = 2) We would like to fit the model, \\[ Y_i = \\beta_0 + \\beta_1 x_i + \\beta_2 x_i^2 + \\epsilon_i \\] where \\(\\epsilon_i \\sim N(0,\\sigma^2)\\) for \\(i = 1, 2, \\cdots 21.\\) The response \\(y\\) is now a linear function of “two” variables which now allows \\(y\\) to be a non-linear function of the original single predictor \\(x\\). We consider this a transformation, although we have actually in some sense added another predictor. Thus, our \\(X\\) matrix is, \\[ \\begin{bmatrix} 1 &amp; x_1 &amp; x_1^2 \\\\[3pt] 1 &amp; x_2 &amp; x_2^2 \\\\[3pt] 1 &amp; x_3 &amp; x_3^2 \\\\[3pt] \\vdots &amp; \\vdots &amp; \\vdots \\\\[3pt] 1 &amp; x_{n} &amp; x_{n}^2 \\\\ \\end{bmatrix} \\] We can then proceed to fit the model as we have in the past for multiple linear regression. \\[ \\hat{\\beta} = \\left( X^\\top X \\right)^{-1}X^\\top y. \\] Our estimates will have the usual properties. The mean is still \\[ E[\\hat{\\beta}] = \\beta, \\] and variance \\[ \\text{Var}[\\hat{\\beta}] = \\sigma^2 \\left( X^\\top X \\right)^{-1}. \\] We also maintain the same distributional results \\[ \\hat{\\beta}_j \\sim N\\left(\\beta_j, \\sigma^2 C_{jj} \\right). \\] mark_mod = lm(sales ~ advert, data = marketing) summary(mark_mod) ## ## Call: ## lm(formula = sales ~ advert, data = marketing) ## ## Residuals: ## Min 1Q Median 3Q Max ## -2.7845 -1.4762 -0.5103 1.2361 3.1869 ## ## Coefficients: ## Estimate Std. Error t value Pr(&gt;|t|) ## (Intercept) 9.4502 0.6806 13.88 2.13e-11 *** ## advert 1.1918 0.0937 12.72 9.65e-11 *** ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 ## ## Residual standard error: 1.907 on 19 degrees of freedom ## Multiple R-squared: 0.8949, Adjusted R-squared: 0.8894 ## F-statistic: 161.8 on 1 and 19 DF, p-value: 9.646e-11 While the SLR model is significant, the fitted versus residuals plot would have a very clear pattern. mark_mod_poly2 = lm(sales ~ advert + I(advert ^ 2), data = marketing) summary(mark_mod_poly2) ## ## Call: ## lm(formula = sales ~ advert + I(advert^2), data = marketing) ## ## Residuals: ## Min 1Q Median 3Q Max ## -1.9175 -0.8333 -0.1948 0.9292 2.1385 ## ## Coefficients: ## Estimate Std. Error t value Pr(&gt;|t|) ## (Intercept) 6.76161 0.67219 10.059 8.16e-09 *** ## advert 2.46231 0.24830 9.917 1.02e-08 *** ## I(advert^2) -0.08745 0.01658 -5.275 5.14e-05 *** ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 ## ## Residual standard error: 1.228 on 18 degrees of freedom ## Multiple R-squared: 0.9587, Adjusted R-squared: 0.9541 ## F-statistic: 209 on 2 and 18 DF, p-value: 3.486e-13 To add the second order term we need to use the I() function in the model specification around our newly created predictor. We see that with the first order term in the model, the quadratic term is also significant. n = length(marketing$advert) X = cbind(rep(1, n), marketing$advert, marketing$advert ^ 2) t(X) %*% X ## [,1] [,2] [,3] ## [1,] 21.00 120.70 1107.95 ## [2,] 120.70 1107.95 12385.86 ## [3,] 1107.95 12385.86 151369.12 solve(t(X) %*% X) %*% t(X) %*% marketing$sales ## [,1] ## [1,] 6.76161045 ## [2,] 2.46230964 ## [3,] -0.08745394 Here we verify the parameter estimates were found as we would expect. We could also add higher order terms, such as a third degree predictor. This is easy to do. Our \\(X\\) matrix simply becomes larger again. \\[ Y_i = \\beta_0 + \\beta_1 x_i + \\beta_2 x_i^2 + \\beta_3 x_i^3 + \\epsilon_i \\] \\[ \\begin{bmatrix} 1 &amp; x_1 &amp; x_1^2 &amp; x_1^3 \\\\[3pt] 1 &amp; x_2 &amp; x_2^2 &amp; x_2^3 \\\\[3pt] 1 &amp; x_3 &amp; x_3^2 &amp; x_3^3 \\\\[3pt] \\vdots &amp; \\vdots &amp; \\vdots &amp; \\vdots \\\\[3pt] 1 &amp; x_{n} &amp; x_{n}^2 &amp; x_{n}^3 \\\\ \\end{bmatrix} \\] mark_mod_poly3 = lm(sales ~ advert + I(advert ^ 2) + I(advert ^ 3), data = marketing) summary(mark_mod_poly3) ## ## Call: ## lm(formula = sales ~ advert + I(advert^2) + I(advert^3), data = marketing) ## ## Residuals: ## Min 1Q Median 3Q Max ## -1.44322 -0.61310 -0.01527 0.68131 1.22517 ## ## Coefficients: ## Estimate Std. Error t value Pr(&gt;|t|) ## (Intercept) 3.890070 0.761956 5.105 8.79e-05 *** ## advert 4.681864 0.501032 9.344 4.14e-08 *** ## I(advert^2) -0.455152 0.078977 -5.763 2.30e-05 *** ## I(advert^3) 0.016131 0.003429 4.704 0.000205 *** ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 ## ## Residual standard error: 0.8329 on 17 degrees of freedom ## Multiple R-squared: 0.9821, Adjusted R-squared: 0.9789 ## F-statistic: 310.2 on 3 and 17 DF, p-value: 4.892e-15 Now we see that with the first and second order terms in the model, the third order term is also significant. But does this make sense practically? The following plot should gives hints as to why it doesn’t. (The model with the third order term doesn’t have diminishing returns!) plot(sales ~ advert, data = marketing, xlab = &quot;Advert Spending (in $10,000)&quot;, ylab = &quot;Sales (in $10,000)&quot;, pch = 20, cex = 2) abline(mark_mod, lty = 2, col = &quot;green&quot;, lwd = 2) xplot = seq(0, 16, by = 0.01) lines(xplot, predict(mark_mod_poly2, newdata = data.frame(advert = xplot)), col = &quot;blue&quot;, lwd = 2) lines(xplot, predict(mark_mod_poly3, newdata = data.frame(advert = xplot)), col = &quot;red&quot;, lty = 3, lwd = 3) The previous plot was made using base graphics in R. The next plot was made using the package ggplot2, an increasingly popular plotting method in R. library(ggplot2) ggplot(data = marketing, aes(x = advert, y = sales)) + stat_smooth(method = &quot;lm&quot;, se = FALSE, color = &quot;green&quot;, formula = y ~ x) + stat_smooth(method = &quot;lm&quot;, se = FALSE, color = &quot;blue&quot;, formula = y ~ x + I(x ^ 2)) + stat_smooth(method = &quot;lm&quot;, se = FALSE, color = &quot;red&quot;, formula = y ~ x + I(x ^ 2)+ I(x ^ 3)) + geom_point(colour = &quot;black&quot;, size = 3) Note we could fit a polynomial of an arbitrary order, \\[ Y_i = \\beta_0 + \\beta_1 x_i + \\beta_2 x_i^2 + \\cdots + \\beta_{p-1}x_i^{p-1} + \\epsilon_i \\] However, we should be careful about over-fitting, since with a polynomial of degree one less than the number of observations, it is sometimes possible to fit a model perfectly. set.seed(1234) x = seq(0, 10) y = 3 + x + 4 * x ^ 2 + rnorm(11, 0, 20) plot(x, y, ylim = c(-300, 400), cex = 2, pch = 20) fit = lm(y ~ x + I(x ^ 2)) #summary(fit) fit_perf = lm(y ~ x + I(x ^ 2) + I(x ^ 3) + I(x ^ 4) + I(x ^ 5) + I(x ^ 6) + I(x ^ 7) + I(x ^ 8) + I(x ^ 9) + I(x ^ 10)) summary(fit_perf) ## ## Call: ## lm(formula = y ~ x + I(x^2) + I(x^3) + I(x^4) + I(x^5) + I(x^6) + ## I(x^7) + I(x^8) + I(x^9) + I(x^10)) ## ## Residuals: ## ALL 11 residuals are 0: no residual degrees of freedom! ## ## Coefficients: ## Estimate Std. Error t value Pr(&gt;|t|) ## (Intercept) -2.114e+01 NaN NaN NaN ## x -1.918e+03 NaN NaN NaN ## I(x^2) 4.969e+03 NaN NaN NaN ## I(x^3) -4.932e+03 NaN NaN NaN ## I(x^4) 2.581e+03 NaN NaN NaN ## I(x^5) -8.035e+02 NaN NaN NaN ## I(x^6) 1.570e+02 NaN NaN NaN ## I(x^7) -1.947e+01 NaN NaN NaN ## I(x^8) 1.490e+00 NaN NaN NaN ## I(x^9) -6.424e-02 NaN NaN NaN ## I(x^10) 1.195e-03 NaN NaN NaN ## ## Residual standard error: NaN on 0 degrees of freedom ## Multiple R-squared: 1, Adjusted R-squared: NaN ## F-statistic: NaN on 10 and 0 DF, p-value: NA xplot = seq(0, 10, by = 0.1) lines(xplot, predict(fit, newdata = data.frame(x = xplot)), col = &quot;dodgerblue&quot;, lwd = 2, lty = 1) lines(xplot, predict(fit_perf, newdata = data.frame(x = xplot)), col = &quot;darkorange&quot;, lwd = 2, lty = 2) Notice in the summary, R could not calculate standard errors. This is a result of being “out” of degrees of freedom. With 11 \\(\\beta\\) parameters and 11 data points, we use up all the degrees of freedom before we can estimate \\(\\sigma\\). In this example, the true relationship is quadratic, but the order 10 polynomial’s fit is “perfect”. Next chapter we will focus on the trade-off between goodness of fit (minimizing errors) and complexity of model. Suppose you work for an automobile manufacturer which makes a large luxury sedan. You would like to know how the car performs from a fuel efficiency standpoint when it is driven at various speeds. Instead of testing the car at every conceivable speed (which would be impossible) you create an experiment where the car is driven at speeds of interest in increments of 5 miles per hour. Our goal then, is to fit a model to this data in order to be able to predict fuel efficiency when driving at certain speeds. The data from this example can be found in fuel_econ.csv. econ = read.csv(&quot;data/fuel_econ.csv&quot;) In this example, we will be frequently looking at the fitted versus residuals plot, so we should write a function to make our life easier, but this is left as an exercise for homework. We will also be adding fitted curves to scatterplots repeatedly, so smartly we will write a function to do so. plot_econ_curve = function(model) { plot(mpg ~ mph, data = econ, xlab = &quot;Speed (Miles per Hour)&quot;, ylab = &quot;Fuel Efficiency (Miles per Gallon)&quot;, col = &quot;dodgerblue&quot;, pch = 20, cex = 2) xplot = seq(10, 75, by = 0.1) lines(xplot, predict(model, newdata = data.frame(mph = xplot)), col = &quot;darkorange&quot;, lwd = 2, lty = 1) } So now we first fit a simple linear regression to this data. fit1 = lm(mpg ~ mph, data = econ) par(mfrow = c(1, 2)) plot_econ_curve(fit1) plot(fitted(fit1), resid(fit1), xlab = &quot;Fitted&quot;, ylab = &quot;Residuals&quot;, col = &quot;dodgerblue&quot;, pch = 20, cex = 2) abline(h = 0, col = &quot;darkorange&quot;, lwd = 2) Pretty clearly we can do better. Yes fuel efficiency does increase as speed increases, but only up to a certain point. We will now add polynomial terms until we fit a suitable fit. fit2 = lm(mpg ~ mph + I(mph ^ 2), data = econ) summary(fit2) ## ## Call: ## lm(formula = mpg ~ mph + I(mph^2), data = econ) ## ## Residuals: ## Min 1Q Median 3Q Max ## -2.8411 -0.9694 0.0017 1.0181 3.3900 ## ## Coefficients: ## Estimate Std. Error t value Pr(&gt;|t|) ## (Intercept) 2.4444505 1.4241091 1.716 0.0984 . ## mph 1.2716937 0.0757321 16.792 3.99e-15 *** ## I(mph^2) -0.0145014 0.0008719 -16.633 4.97e-15 *** ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 ## ## Residual standard error: 1.663 on 25 degrees of freedom ## Multiple R-squared: 0.9188, Adjusted R-squared: 0.9123 ## F-statistic: 141.5 on 2 and 25 DF, p-value: 2.338e-14 par(mfrow = c(1, 2)) plot_econ_curve(fit2) plot(fitted(fit2), resid(fit2), xlab = &quot;Fitted&quot;, ylab = &quot;Residuals&quot;, col = &quot;dodgerblue&quot;, pch = 20, cex = 2) abline(h = 0, col = &quot;darkorange&quot;, lwd = 2) While this model clearly fits much better, and the second order term is significant, we still see a pattern in the fitted versus residuals plot which suggests higher order terms will help. Also, we would expect the curve to flatten as speed increases or decreases, not go sharply downward as we see here. fit3 = lm(mpg ~ mph + I(mph ^ 2) + I(mph ^ 3), data = econ) summary(fit3) ## ## Call: ## lm(formula = mpg ~ mph + I(mph^2) + I(mph^3), data = econ) ## ## Residuals: ## Min 1Q Median 3Q Max ## -2.8112 -0.9677 0.0264 1.0345 3.3827 ## ## Coefficients: ## Estimate Std. Error t value Pr(&gt;|t|) ## (Intercept) 2.258e+00 2.768e+00 0.816 0.4227 ## mph 1.291e+00 2.529e-01 5.103 3.2e-05 *** ## I(mph^2) -1.502e-02 6.604e-03 -2.274 0.0322 * ## I(mph^3) 4.066e-06 5.132e-05 0.079 0.9375 ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 ## ## Residual standard error: 1.697 on 24 degrees of freedom ## Multiple R-squared: 0.9188, Adjusted R-squared: 0.9087 ## F-statistic: 90.56 on 3 and 24 DF, p-value: 3.17e-13 par(mfrow = c(1, 2)) plot_econ_curve(fit3) plot(fitted(fit3), resid(fit3), xlab = &quot;Fitted&quot;, ylab = &quot;Residuals&quot;, col = &quot;dodgerblue&quot;, pch = 20, cex = 2) abline(h = 0, col = &quot;darkorange&quot;, lwd = 2) Adding the third order term doesn’t seem to help at all. The fitted curve hardly changes. This makes sense, since what we would like is for the curve to flatten at the extremes. For this we will need an even degree polynomial term. fit4 = lm(mpg ~ mph + I(mph ^ 2) + I(mph ^ 3) + I(mph ^ 4), data = econ) summary(fit4) ## ## Call: ## lm(formula = mpg ~ mph + I(mph^2) + I(mph^3) + I(mph^4), data = econ) ## ## Residuals: ## Min 1Q Median 3Q Max ## -1.57410 -0.60308 0.04236 0.74481 1.93038 ## ## Coefficients: ## Estimate Std. Error t value Pr(&gt;|t|) ## (Intercept) 2.146e+01 2.965e+00 7.238 2.28e-07 *** ## mph -1.468e+00 3.913e-01 -3.751 0.00104 ** ## I(mph^2) 1.081e-01 1.673e-02 6.463 1.35e-06 *** ## I(mph^3) -2.130e-03 2.844e-04 -7.488 1.31e-07 *** ## I(mph^4) 1.255e-05 1.665e-06 7.539 1.17e-07 *** ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 ## ## Residual standard error: 0.9307 on 23 degrees of freedom ## Multiple R-squared: 0.9766, Adjusted R-squared: 0.9726 ## F-statistic: 240.2 on 4 and 23 DF, p-value: &lt; 2.2e-16 par(mfrow = c(1, 2)) plot_econ_curve(fit4) plot(fitted(fit4), resid(fit4), xlab = &quot;Fitted&quot;, ylab = &quot;Residuals&quot;, col = &quot;dodgerblue&quot;, pch = 20, cex = 2) abline(h = 0, col = &quot;darkorange&quot;, lwd = 2) Now we are making progress. The fourth order term is significant with the other terms in the model. Also we are starting to see what we expected for low and high speed. However, there still seems to be a bit of a pattern in the residuals, so we will again try more higher order terms. We will add the fifth and sixth together, since adding the fifth will be similar to adding the third. fit6 = lm(mpg ~ mph + I(mph ^ 2) + I(mph ^ 3) + I(mph ^ 4) + I(mph ^ 5) + I(mph^6), data = econ) summary(fit6) ## ## Call: ## lm(formula = mpg ~ mph + I(mph^2) + I(mph^3) + I(mph^4) + I(mph^5) + ## I(mph^6), data = econ) ## ## Residuals: ## Min 1Q Median 3Q Max ## -1.1129 -0.5717 -0.1707 0.5026 1.5288 ## ## Coefficients: ## Estimate Std. Error t value Pr(&gt;|t|) ## (Intercept) -4.206e+00 1.204e+01 -0.349 0.7304 ## mph 4.203e+00 2.553e+00 1.646 0.1146 ## I(mph^2) -3.521e-01 2.012e-01 -1.750 0.0947 . ## I(mph^3) 1.579e-02 7.691e-03 2.053 0.0527 . ## I(mph^4) -3.473e-04 1.529e-04 -2.271 0.0338 * ## I(mph^5) 3.585e-06 1.518e-06 2.362 0.0279 * ## I(mph^6) -1.402e-08 5.941e-09 -2.360 0.0280 * ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 ## ## Residual standard error: 0.8657 on 21 degrees of freedom ## Multiple R-squared: 0.9815, Adjusted R-squared: 0.9762 ## F-statistic: 186 on 6 and 21 DF, p-value: &lt; 2.2e-16 par(mfrow = c(1, 2)) plot_econ_curve(fit6) plot(fitted(fit6), resid(fit6), xlab = &quot;Fitted&quot;, ylab = &quot;Residuals&quot;, col = &quot;dodgerblue&quot;, pch = 20, cex = 2) abline(h = 0, col = &quot;darkorange&quot;, lwd = 2) Again the sixth order term is significant with the other terms in the model and here we see less pattern in the residuals plot. Let’s now test which of the previous two models we prefer. We will test \\[ H_0: \\beta_5 = \\beta_6 = 0. \\] anova(fit4, fit6) ## Analysis of Variance Table ## ## Model 1: mpg ~ mph + I(mph^2) + I(mph^3) + I(mph^4) ## Model 2: mpg ~ mph + I(mph^2) + I(mph^3) + I(mph^4) + I(mph^5) + I(mph^6) ## Res.Df RSS Df Sum of Sq F Pr(&gt;F) ## 1 23 19.922 ## 2 21 15.739 2 4.1828 2.7905 0.0842 . ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 So, this test does not reject the null hypothesis at a level of significance of \\(\\alpha = 0.05\\), however the p-value is still rather small, and the fitted versus residuals plot is much better for the model with the sixth order term. This makes the sixth order model a good choice. We could repeat this process one more time. fit8 = lm(mpg ~ mph + I(mph ^ 2) + I(mph ^ 3) + I(mph ^ 4) + I(mph ^ 5) + I(mph ^ 6) + I(mph ^ 7) + I(mph ^ 8), data = econ) summary(fit8) ## ## Call: ## lm(formula = mpg ~ mph + I(mph^2) + I(mph^3) + I(mph^4) + I(mph^5) + ## I(mph^6) + I(mph^7) + I(mph^8), data = econ) ## ## Residuals: ## Min 1Q Median 3Q Max ## -1.21938 -0.50464 -0.09105 0.49029 1.45440 ## ## Coefficients: ## Estimate Std. Error t value Pr(&gt;|t|) ## (Intercept) -1.202e+01 7.045e+01 -0.171 0.866 ## mph 6.021e+00 2.014e+01 0.299 0.768 ## I(mph^2) -5.037e-01 2.313e+00 -0.218 0.830 ## I(mph^3) 2.121e-02 1.408e-01 0.151 0.882 ## I(mph^4) -4.008e-04 5.017e-03 -0.080 0.937 ## I(mph^5) 1.789e-06 1.080e-04 0.017 0.987 ## I(mph^6) 4.486e-08 1.381e-06 0.032 0.974 ## I(mph^7) -6.456e-10 9.649e-09 -0.067 0.947 ## I(mph^8) 2.530e-12 2.835e-11 0.089 0.930 ## ## Residual standard error: 0.9034 on 19 degrees of freedom ## Multiple R-squared: 0.9818, Adjusted R-squared: 0.9741 ## F-statistic: 128.1 on 8 and 19 DF, p-value: 7.074e-15 par(mfrow = c(1, 2)) plot_econ_curve(fit8) plot(fitted(fit8), resid(fit8), xlab = &quot;Fitted&quot;, ylab = &quot;Residuals&quot;, col = &quot;dodgerblue&quot;, pch = 20, cex = 2) abline(h = 0, col = &quot;darkorange&quot;, lwd = 2) summary(fit8) ## ## Call: ## lm(formula = mpg ~ mph + I(mph^2) + I(mph^3) + I(mph^4) + I(mph^5) + ## I(mph^6) + I(mph^7) + I(mph^8), data = econ) ## ## Residuals: ## Min 1Q Median 3Q Max ## -1.21938 -0.50464 -0.09105 0.49029 1.45440 ## ## Coefficients: ## Estimate Std. Error t value Pr(&gt;|t|) ## (Intercept) -1.202e+01 7.045e+01 -0.171 0.866 ## mph 6.021e+00 2.014e+01 0.299 0.768 ## I(mph^2) -5.037e-01 2.313e+00 -0.218 0.830 ## I(mph^3) 2.121e-02 1.408e-01 0.151 0.882 ## I(mph^4) -4.008e-04 5.017e-03 -0.080 0.937 ## I(mph^5) 1.789e-06 1.080e-04 0.017 0.987 ## I(mph^6) 4.486e-08 1.381e-06 0.032 0.974 ## I(mph^7) -6.456e-10 9.649e-09 -0.067 0.947 ## I(mph^8) 2.530e-12 2.835e-11 0.089 0.930 ## ## Residual standard error: 0.9034 on 19 degrees of freedom ## Multiple R-squared: 0.9818, Adjusted R-squared: 0.9741 ## F-statistic: 128.1 on 8 and 19 DF, p-value: 7.074e-15 anova(fit6, fit8) ## Analysis of Variance Table ## ## Model 1: mpg ~ mph + I(mph^2) + I(mph^3) + I(mph^4) + I(mph^5) + I(mph^6) ## Model 2: mpg ~ mph + I(mph^2) + I(mph^3) + I(mph^4) + I(mph^5) + I(mph^6) + ## I(mph^7) + I(mph^8) ## Res.Df RSS Df Sum of Sq F Pr(&gt;F) ## 1 21 15.739 ## 2 19 15.506 2 0.2324 0.1424 0.8682 Here we would clearly stick with fit6. The eighth order term is not significant with the other terms in the model and the F-test does not reject. As an aside, be aware that there is a quicker way to specify a model with many higher order terms. fit6_alt = lm(mpg ~ poly(mph, 6), data = econ) all.equal(fitted(fit6), fitted(fit6_alt)) ## [1] TRUE We first verify that this method produces the same fitted values. However, the estimated coefficients are different. coef(fit6) ## (Intercept) mph I(mph^2) I(mph^3) I(mph^4) ## -4.206224e+00 4.203382e+00 -3.521452e-01 1.579340e-02 -3.472665e-04 ## I(mph^5) I(mph^6) ## 3.585201e-06 -1.401995e-08 coef(fit6_alt) ## (Intercept) poly(mph, 6)1 poly(mph, 6)2 poly(mph, 6)3 poly(mph, 6)4 ## 24.40714286 4.16769628 -27.66685755 0.13446747 7.01671480 ## poly(mph, 6)5 poly(mph, 6)6 ## 0.09288754 -2.04307796 This is because poly() uses orthogonal polynomials, which solves an issue we will discuss in the next chapter. summary(fit6) ## ## Call: ## lm(formula = mpg ~ mph + I(mph^2) + I(mph^3) + I(mph^4) + I(mph^5) + ## I(mph^6), data = econ) ## ## Residuals: ## Min 1Q Median 3Q Max ## -1.1129 -0.5717 -0.1707 0.5026 1.5288 ## ## Coefficients: ## Estimate Std. Error t value Pr(&gt;|t|) ## (Intercept) -4.206e+00 1.204e+01 -0.349 0.7304 ## mph 4.203e+00 2.553e+00 1.646 0.1146 ## I(mph^2) -3.521e-01 2.012e-01 -1.750 0.0947 . ## I(mph^3) 1.579e-02 7.691e-03 2.053 0.0527 . ## I(mph^4) -3.473e-04 1.529e-04 -2.271 0.0338 * ## I(mph^5) 3.585e-06 1.518e-06 2.362 0.0279 * ## I(mph^6) -1.402e-08 5.941e-09 -2.360 0.0280 * ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 ## ## Residual standard error: 0.8657 on 21 degrees of freedom ## Multiple R-squared: 0.9815, Adjusted R-squared: 0.9762 ## F-statistic: 186 on 6 and 21 DF, p-value: &lt; 2.2e-16 summary(fit6_alt) ## ## Call: ## lm(formula = mpg ~ poly(mph, 6), data = econ) ## ## Residuals: ## Min 1Q Median 3Q Max ## -1.1129 -0.5717 -0.1707 0.5026 1.5288 ## ## Coefficients: ## Estimate Std. Error t value Pr(&gt;|t|) ## (Intercept) 24.40714 0.16360 149.184 &lt; 2e-16 *** ## poly(mph, 6)1 4.16770 0.86571 4.814 9.31e-05 *** ## poly(mph, 6)2 -27.66686 0.86571 -31.958 &lt; 2e-16 *** ## poly(mph, 6)3 0.13447 0.86571 0.155 0.878 ## poly(mph, 6)4 7.01671 0.86571 8.105 6.68e-08 *** ## poly(mph, 6)5 0.09289 0.86571 0.107 0.916 ## poly(mph, 6)6 -2.04308 0.86571 -2.360 0.028 * ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 ## ## Residual standard error: 0.8657 on 21 degrees of freedom ## Multiple R-squared: 0.9815, Adjusted R-squared: 0.9762 ## F-statistic: 186 on 6 and 21 DF, p-value: &lt; 2.2e-16 Notice though that the p-value for testing the degree 6 term is the same. Because of this, for the most part we can use these interchangeably. To use poly() to obtain the same results as using I() repeatedly, we would need to set raw = TRUE. fit6_alt2 = lm(mpg ~ poly(mph, 6, raw = TRUE), data = econ) coef(fit6_alt2) ## (Intercept) poly(mph, 6, raw = TRUE)1 poly(mph, 6, raw = TRUE)2 ## -4.206224e+00 4.203382e+00 -3.521452e-01 ## poly(mph, 6, raw = TRUE)3 poly(mph, 6, raw = TRUE)4 poly(mph, 6, raw = TRUE)5 ## 1.579340e-02 -3.472665e-04 3.585201e-06 ## poly(mph, 6, raw = TRUE)6 ## -1.401995e-08 We’ve now seen how to transform predictor and response variables. In this chapter we have mostly focused on using this in the context of fixing SLR models. However, these concepts can easily be used together with categorical variables and interactions to build larger, more flexible models. In the next chapter, we will discuss how to choose a good model from a collection of possible models. Material below here is currently being merged into the content above. Response Transformations initech = read.csv(&quot;data/initech.csv&quot;) plot(salary ~ years, data = initech, col = &quot;grey&quot;, pch = 20, cex = 1.5, main = &quot;Salaries at Initech, By Seniority&quot;) initech_fit = lm(salary ~ years, data = initech) summary(initech_fit) ## ## Call: ## lm(formula = salary ~ years, data = initech) ## ## Residuals: ## Min 1Q Median 3Q Max ## -57225 -18104 241 15589 91332 ## ## Coefficients: ## Estimate Std. Error t value Pr(&gt;|t|) ## (Intercept) 5302 5750 0.922 0.359 ## years 8637 389 22.200 &lt;2e-16 *** ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 ## ## Residual standard error: 27360 on 98 degrees of freedom ## Multiple R-squared: 0.8341, Adjusted R-squared: 0.8324 ## F-statistic: 492.8 on 1 and 98 DF, p-value: &lt; 2.2e-16 plot(salary ~ years, data = initech, col = &quot;grey&quot;, pch = 20, cex = 1.5, main = &quot;Salaries at Initech, By Seniority&quot;) abline(initech_fit, col = &quot;darkorange&quot;, lwd = 2) par(mfrow = c(1, 2)) plot(fitted(initech_fit), resid(initech_fit), col = &quot;grey&quot;, pch = 20, xlab = &quot;Fitted&quot;, ylab = &quot;Residuals&quot;, main = &quot;Fitted versus Residuals&quot;) abline(h = 0, col = &quot;darkorange&quot;, lwd = 2) qqnorm(resid(initech_fit), main = &quot;Normal Q-Q Plot&quot;, col = &quot;darkgrey&quot;) qqline(resid(initech_fit), col = &quot;dodgerblue&quot;, lwd = 2) initech_fit_log = lm(log(salary) ~ years, data = initech) \\[ \\log(Y_i) = \\beta_0 + \\beta_1 x_i + \\epsilon_i \\] plot(log(salary) ~ years, data = initech, col = &quot;grey&quot;, pch = 20, cex = 1.5, main = &quot;Salaries at Initech, By Seniority&quot;) abline(initech_fit_log, col = &quot;darkorange&quot;, lwd = 2) \\[ Y_i = \\exp(\\beta_0 + \\beta_1 x_i) \\cdot \\exp(\\epsilon_i) \\] plot(salary ~ years, data = initech, col = &quot;grey&quot;, pch = 20, cex = 1.5, main = &quot;Salaries at Initech, By Seniority&quot;) curve(exp(initech_fit_log$coef[1] + initech_fit_log$coef[2] * x), from = 0, to = 30, add = TRUE, col = &quot;darkorange&quot;, lwd = 2) par(mfrow = c(1, 2)) plot(fitted(initech_fit_log), resid(initech_fit_log), col = &quot;grey&quot;, pch = 20, xlab = &quot;Fitted&quot;, ylab = &quot;Residuals&quot;, main = &quot;Fitted versus Residuals&quot;) abline(h = 0, col = &quot;darkorange&quot;, lwd = 2) qqnorm(resid(initech_fit_log), main = &quot;Normal Q-Q Plot&quot;, col = &quot;darkgrey&quot;) qqline(resid(initech_fit_log), col = &quot;dodgerblue&quot;, lwd = 2) sqrt(mean(resid(initech_fit) ^ 2)) ## [1] 27080.16 sqrt(mean(resid(initech_fit_log) ^ 2)) ## [1] 0.1934907 sqrt(mean((initech$salary - fitted(initech_fit)) ^ 2)) ## [1] 27080.16 sqrt(mean((initech$salary - exp(fitted(initech_fit_log))) ^ 2)) ## [1] 24280.36 Predictor Transformations 14.2.2 A Quadratic Model sim_quad = function(sample_size = 500) { x = runif(n = sample_size) * 5 y = 3 + 5 * x ^ 2 + rnorm(n = sample_size, mean = 0, sd = 5) data.frame(x, y) } set.seed(314) quad_data = sim_quad(sample_size = 200) lin_fit = lm(y ~ x, data = quad_data) summary(lin_fit) ## ## Call: ## lm(formula = y ~ x, data = quad_data) ## ## Residuals: ## Min 1Q Median 3Q Max ## -20.363 -7.550 -3.416 8.472 26.181 ## ## Coefficients: ## Estimate Std. Error t value Pr(&gt;|t|) ## (Intercept) -18.3271 1.5494 -11.83 &lt;2e-16 *** ## x 24.8716 0.5343 46.55 &lt;2e-16 *** ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 ## ## Residual standard error: 10.79 on 198 degrees of freedom ## Multiple R-squared: 0.9163, Adjusted R-squared: 0.9158 ## F-statistic: 2167 on 1 and 198 DF, p-value: &lt; 2.2e-16 plot(y ~ x, data = quad_data, col = &quot;grey&quot;, pch = 20, cex = 1.5, main = &quot;Simulated Quadratic Data&quot;) abline(lin_fit, col = &quot;darkorange&quot;, lwd = 2) par(mfrow = c(1, 2)) plot(fitted(lin_fit), resid(lin_fit), col = &quot;grey&quot;, pch = 20, xlab = &quot;Fitted&quot;, ylab = &quot;Residuals&quot;, main = &quot;Fitted versus Residuals&quot;) abline(h = 0, col = &quot;darkorange&quot;, lwd = 2) qqnorm(resid(lin_fit), main = &quot;Normal Q-Q Plot&quot;, col = &quot;darkgrey&quot;) qqline(resid(lin_fit), col = &quot;dodgerblue&quot;, lwd = 2) \\[ Y_i = \\beta_0 + \\beta_1 x_i + \\beta_2 x_i^2 + \\epsilon_i \\] quad_fit = lm(y ~ x + I(x^2), data = quad_data) summary(quad_fit) ## ## Call: ## lm(formula = y ~ x + I(x^2), data = quad_data) ## ## Residuals: ## Min 1Q Median 3Q Max ## -11.4167 -3.0581 0.2297 3.1024 12.1256 ## ## Coefficients: ## Estimate Std. Error t value Pr(&gt;|t|) ## (Intercept) 3.0649 0.9577 3.200 0.0016 ** ## x -0.5108 0.8637 -0.591 0.5549 ## I(x^2) 5.0740 0.1667 30.433 &lt;2e-16 *** ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 ## ## Residual standard error: 4.531 on 197 degrees of freedom ## Multiple R-squared: 0.9853, Adjusted R-squared: 0.9852 ## F-statistic: 6608 on 2 and 197 DF, p-value: &lt; 2.2e-16 plot(y ~ x, data = quad_data, col = &quot;grey&quot;, pch = 20, cex = 1.5, main = &quot;Simulated Quadratic Data&quot;) curve(quad_fit$coef[1] + quad_fit$coef[2] * x + quad_fit$coef[3] * x ^ 2, from = -5, to = 30, add = TRUE, col = &quot;darkorange&quot;, lwd = 2) par(mfrow = c(1, 2)) plot(fitted(quad_fit), resid(quad_fit), col = &quot;grey&quot;, pch = 20, xlab = &quot;Fitted&quot;, ylab = &quot;Residuals&quot;, main = &quot;Fitted versus Residuals&quot;) abline(h = 0, col = &quot;darkorange&quot;, lwd = 2) qqnorm(resid(quad_fit), main = &quot;Normal Q-Q Plot&quot;, col = &quot;darkgrey&quot;) qqline(resid(quad_fit), col = &quot;dodgerblue&quot;, lwd = 2) 14.2.3 Overfitting and Extrapolation sim_for_perf = function() { x = seq(0, 10) y = 3 + x - 4 * x ^ 2 + rnorm(n = 11, mean = 0, sd = 25) data.frame(x, y) } set.seed(1234) data_for_perf = sim_for_perf() fit_correct = lm(y ~ x + I(x ^ 2), data = data_for_perf) fit_perfect = lm(y ~ x + I(x ^ 2) + I(x ^ 3) + I(x ^ 4) + I(x ^ 5) + I(x ^ 6) + I(x ^ 7) + I(x ^ 8) + I(x ^ 9) + I(x ^ 10), data = data_for_perf) x_plot = seq(-5, 15, by = 0.1) plot(y ~ x, data = data_for_perf, ylim = c(-450, 100), cex = 2, pch = 20) lines(x_plot, predict(fit_correct, newdata = data.frame(x = x_plot)), col = &quot;dodgerblue&quot;, lwd = 2, lty = 1) lines(x_plot, predict(fit_perfect, newdata = data.frame(x = x_plot)), col = &quot;darkorange&quot;, lwd = 2, lty = 2) 14.2.4 Comparing Polynomial Models sim_higher = function(sample_size = 250) { x = runif(n = sample_size, min = -1, max = 1) * 2 y = 3 + -6 * x ^ 2 + 1 * x ^ 4 + rnorm(n = sample_size, mean = 0, sd = 3) data.frame(x, y) } \\[ Y_i = \\beta_0 + \\beta_1 x_i + \\beta_2 x_i^2 + \\epsilon_i \\] \\[ Y_i = \\beta_0 + \\beta_1 x_i + \\beta_2 x_i^2 + \\beta_3 x_i^3 + \\beta_4 x_i^4 + \\epsilon_i \\] \\[ Y_i = \\beta_0 + \\beta_1 x_i + \\beta_2 x_i^2 + \\beta_3 x_i^3 + \\beta_4 x_i^4 + \\beta_5 x_i^5 + \\beta_6 x_i^6 + \\epsilon_i \\] set.seed(42) data_higher = sim_higher() plot(y ~ x, data = data_higher, col = &quot;grey&quot;, pch = 20, cex = 1.5, main = &quot;Simulated Quartic Data&quot;) fit_2 = lm(y ~ poly(x, 2), data = data_higher) fit_4 = lm(y ~ poly(x, 4), data = data_higher) plot(y ~ x, data = data_higher, col = &quot;grey&quot;, pch = 20, cex = 1.5, main = &quot;Simulated Quartic Data&quot;) x_plot = seq(-5, 5, by = 0.05) lines(x_plot, predict(fit_2, newdata = data.frame(x = x_plot)), col = &quot;dodgerblue&quot;, lwd = 2, lty = 1) lines(x_plot, predict(fit_4, newdata = data.frame(x = x_plot)), col = &quot;darkorange&quot;, lwd = 2, lty = 2) par(mfrow = c(1, 2)) plot(fitted(fit_2), resid(fit_2), col = &quot;grey&quot;, pch = 20, xlab = &quot;Fitted&quot;, ylab = &quot;Residuals&quot;, main = &quot;Fitted versus Residuals&quot;) abline(h = 0, col = &quot;darkorange&quot;, lwd = 2) qqnorm(resid(fit_2), main = &quot;Normal Q-Q Plot&quot;, col = &quot;darkgrey&quot;) qqline(resid(fit_2), col = &quot;dodgerblue&quot;, lwd = 2) par(mfrow = c(1, 2)) plot(fitted(fit_4), resid(fit_4), col = &quot;grey&quot;, pch = 20, xlab = &quot;Fitted&quot;, ylab = &quot;Residuals&quot;, main = &quot;Fitted versus Residuals&quot;) abline(h = 0, col = &quot;darkorange&quot;, lwd = 2) qqnorm(resid(fit_4), main = &quot;Normal Q-Q Plot&quot;, col = &quot;darkgrey&quot;) qqline(resid(fit_4), col = &quot;dodgerblue&quot;, lwd = 2) anova(fit_2, fit_4) ## Analysis of Variance Table ## ## Model 1: y ~ poly(x, 2) ## Model 2: y ~ poly(x, 4) ## Res.Df RSS Df Sum of Sq F Pr(&gt;F) ## 1 247 2334.1 ## 2 245 1912.6 2 421.51 26.997 2.536e-11 *** ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 fit_6 = lm(y ~ poly(x, 6), data = data_higher) anova(fit_4, fit_6) ## Analysis of Variance Table ## ## Model 1: y ~ poly(x, 4) ## Model 2: y ~ poly(x, 6) ## Res.Df RSS Df Sum of Sq F Pr(&gt;F) ## 1 245 1912.6 ## 2 243 1904.4 2 8.1889 0.5224 0.5937 14.2.5 poly() Function and Orthogonal Polynomials \\[ Y_i = \\beta_0 + \\beta_1 x_i + \\beta_2 x_i^2 + \\beta_3 x_i^3 + \\beta_4 x_i^4 + \\epsilon_i \\] fit_4a = lm(y ~ poly(x, degree = 4), data = data_higher) fit_4b = lm(y ~ poly(x, degree = 4, raw = TRUE), data = data_higher) fit_4c = lm(y ~ x + I(x^2) + I(x^3) + I(x^4), data = data_higher) coef(fit_4a) ## (Intercept) poly(x, degree = 4)1 poly(x, degree = 4)2 ## -1.980036 -2.053929 -49.344752 ## poly(x, degree = 4)3 poly(x, degree = 4)4 ## 0.669874 20.519759 coef(fit_4b) ## (Intercept) poly(x, degree = 4, raw = TRUE)1 ## 2.9996256 -0.3880250 ## poly(x, degree = 4, raw = TRUE)2 poly(x, degree = 4, raw = TRUE)3 ## -6.1511166 0.1269046 ## poly(x, degree = 4, raw = TRUE)4 ## 1.0282139 coef(fit_4c) ## (Intercept) x I(x^2) I(x^3) I(x^4) ## 2.9996256 -0.3880250 -6.1511166 0.1269046 1.0282139 unname(coef(fit_4a)) ## [1] -1.980036 -2.053929 -49.344752 0.669874 20.519759 unname(coef(fit_4b)) ## [1] 2.9996256 -0.3880250 -6.1511166 0.1269046 1.0282139 unname(coef(fit_4c)) ## [1] 2.9996256 -0.3880250 -6.1511166 0.1269046 1.0282139 all.equal(fitted(fit_4a), fitted(fit_4b)) ## [1] TRUE all.equal(resid(fit_4a), resid(fit_4b)) ## [1] TRUE summary(fit_4a) ## ## Call: ## lm(formula = y ~ poly(x, degree = 4), data = data_higher) ## ## Residuals: ## Min 1Q Median 3Q Max ## -7.6982 -2.0334 0.0042 1.9532 7.4626 ## ## Coefficients: ## Estimate Std. Error t value Pr(&gt;|t|) ## (Intercept) -1.9800 0.1767 -11.205 &lt; 2e-16 *** ## poly(x, degree = 4)1 -2.0539 2.7940 -0.735 0.463 ## poly(x, degree = 4)2 -49.3448 2.7940 -17.661 &lt; 2e-16 *** ## poly(x, degree = 4)3 0.6699 2.7940 0.240 0.811 ## poly(x, degree = 4)4 20.5198 2.7940 7.344 3.06e-12 *** ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 ## ## Residual standard error: 2.794 on 245 degrees of freedom ## Multiple R-squared: 0.5993, Adjusted R-squared: 0.5928 ## F-statistic: 91.61 on 4 and 245 DF, p-value: &lt; 2.2e-16 summary(fit_4c) ## ## Call: ## lm(formula = y ~ x + I(x^2) + I(x^3) + I(x^4), data = data_higher) ## ## Residuals: ## Min 1Q Median 3Q Max ## -7.6982 -2.0334 0.0042 1.9532 7.4626 ## ## Coefficients: ## Estimate Std. Error t value Pr(&gt;|t|) ## (Intercept) 2.9996 0.3315 9.048 &lt; 2e-16 *** ## x -0.3880 0.3828 -1.014 0.312 ## I(x^2) -6.1511 0.5049 -12.183 &lt; 2e-16 *** ## I(x^3) 0.1269 0.1456 0.871 0.384 ## I(x^4) 1.0282 0.1400 7.344 3.06e-12 *** ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 ## ## Residual standard error: 2.794 on 245 degrees of freedom ## Multiple R-squared: 0.5993, Adjusted R-squared: 0.5928 ## F-statistic: 91.61 on 4 and 245 DF, p-value: &lt; 2.2e-16 14.2.6 Inhibit Function coef(lm(y ~ x + x ^ 2, data = quad_data)) ## (Intercept) x ## -18.32715 24.87163 coef(lm(y ~ x + I(x ^ 2), data = quad_data)) ## (Intercept) x I(x^2) ## 3.0649446 -0.5108131 5.0739805 coef(lm(y ~ x + x:x, data = quad_data)) ## (Intercept) x ## -18.32715 24.87163 coef(lm(y ~ x * x, data = quad_data)) ## (Intercept) x ## -18.32715 24.87163 coef(lm(y ~ x ^ 2, data = quad_data)) ## (Intercept) x ## -18.32715 24.87163 coef(lm(y ~ x + x ^ 2, data = quad_data)) ## (Intercept) x ## -18.32715 24.87163 coef(lm(y ~ I(x + x), data = quad_data)) ## (Intercept) I(x + x) ## -18.32715 12.43582 coef(lm(y ~ x + x, data = quad_data)) ## (Intercept) x ## -18.32715 24.87163 14.2.7 Data Example pairs(autompg) mpg_hp = lm(mpg ~ hp, data = autompg) par(mfrow = c(1, 2)) plot(mpg ~ hp, data = autompg, col = &quot;dodgerblue&quot;, pch = 20, cex = 1.5) abline(mpg_hp, col = &quot;darkorange&quot;, lwd = 2) plot(fitted(mpg_hp), resid(mpg_hp), col = &quot;dodgerblue&quot;, pch = 20, cex = 1.5, xlab = &quot;Fitted&quot;, ylab = &quot;Residuals&quot;) abline(h = 0, lty = 2, col = &quot;darkorange&quot;, lwd = 2) mpg_hp_log = lm(mpg ~ hp + I(hp ^ 2), data = autompg) par(mfrow = c(1, 2)) plot(mpg ~ hp, data = autompg, col = &quot;dodgerblue&quot;, pch = 20, cex = 1.5) xplot = seq(min(autompg$hp), max(autompg$hp), by = 0.1) lines(xplot, predict(mpg_hp_log, newdata = data.frame(hp = xplot)), col = &quot;darkorange&quot;, lwd = 2, lty = 1) plot(fitted(mpg_hp_log), resid(mpg_hp_log), col = &quot;dodgerblue&quot;, pch = 20, cex = 1.5, xlab = &quot;Fitted&quot;, ylab = &quot;Residuals&quot;) abline(h = 0, lty = 2, col = &quot;darkorange&quot;, lwd = 2) mpg_hp_log = lm(log(mpg) ~ hp + I(hp ^ 2), data = autompg) par(mfrow = c(1, 2)) plot(log(mpg) ~ hp, data = autompg, col = &quot;dodgerblue&quot;, pch = 20, cex = 1.5) xplot = seq(min(autompg$hp), max(autompg$hp), by = 0.1) lines(xplot, predict(mpg_hp_log, newdata = data.frame(hp = xplot)), col = &quot;darkorange&quot;, lwd = 2, lty = 1) plot(fitted(mpg_hp_log), resid(mpg_hp_log), col = &quot;dodgerblue&quot;, pch = 20, cex = 1.5, xlab = &quot;Fitted&quot;, ylab = &quot;Residuals&quot;) abline(h = 0, lty = 2, col = &quot;darkorange&quot;, lwd = 2) mpg_hp_loglog = lm(log(mpg) ~ log(hp), data = autompg) par(mfrow = c(1, 2)) plot(log(mpg) ~ log(hp), data = autompg, col = &quot;dodgerblue&quot;, pch = 20, cex = 1.5) abline(mpg_hp_loglog, col = &quot;darkorange&quot;, lwd = 2) plot(fitted(mpg_hp_loglog), resid(mpg_hp_loglog), col = &quot;dodgerblue&quot;, pch = 20, cex = 1.5, xlab = &quot;Fitted&quot;, ylab = &quot;Residuals&quot;) abline(h = 0, lty = 2, col = &quot;darkorange&quot;, lwd = 2) big_model = lm(mpg ~ disp * hp * domestic, data = autompg) qqnorm(resid(big_model), col = &quot;darkgrey&quot;) qqline(resid(big_model), col = &quot;dodgerblue&quot;, lwd = 2) bigger_model = lm(log(mpg) ~ disp * hp * domestic + I(disp ^ 2) + I(hp ^ 2), data = autompg) summary(bigger_model) ## ## Call: ## lm(formula = log(mpg) ~ disp * hp * domestic + I(disp^2) + I(hp^2), ## data = autompg) ## ## Residuals: ## Min 1Q Median 3Q Max ## -0.40381 -0.08635 -0.01040 0.09995 0.71365 ## ## Coefficients: ## Estimate Std. Error t value Pr(&gt;|t|) ## (Intercept) 4.276e+00 2.564e-01 16.674 &lt;2e-16 *** ## disp -5.289e-03 2.565e-03 -2.062 0.0399 * ## hp -7.386e-03 3.309e-03 -2.232 0.0262 * ## domestic -2.496e-01 2.787e-01 -0.896 0.3710 ## I(disp^2) 8.552e-06 4.141e-06 2.065 0.0396 * ## I(hp^2) -1.565e-05 1.679e-05 -0.932 0.3519 ## disp:hp 2.685e-05 3.082e-05 0.871 0.3842 ## disp:domestic -1.101e-03 2.526e-03 -0.436 0.6631 ## hp:domestic 7.560e-03 3.689e-03 2.049 0.0411 * ## disp:hp:domestic -2.311e-05 2.662e-05 -0.868 0.3859 ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 ## ## Residual standard error: 0.1507 on 373 degrees of freedom ## Multiple R-squared: 0.8107, Adjusted R-squared: 0.8062 ## F-statistic: 177.5 on 9 and 373 DF, p-value: &lt; 2.2e-16 qqnorm(resid(bigger_model), col = &quot;darkgrey&quot;) qqline(resid(bigger_model), col = &quot;dodgerblue&quot;, lwd = 2) 14.3 R Markdown The R Markdown file for this chapter can be found here: transformations.Rmd The file was created using R version 4.3.2. "],["collinearity.html", "Chapter 15 Collinearity 15.1 Exact Collinearity 15.2 Collinearity 15.3 Simulation 15.4 R Markdown", " Chapter 15 Collinearity “If I look confused it is because I am thinking.” — Samuel Goldwyn After reading this chapter you will be able to: Identify collinearity in regression. Understand the effect of collinearity on regression models. 15.1 Exact Collinearity Let’s create a dataset where one of the predictors, \\(x_3\\), is a linear combination of the other predictors. gen_exact_collin_data = function(num_samples = 100) { x1 = rnorm(n = num_samples, mean = 80, sd = 10) x2 = rnorm(n = num_samples, mean = 70, sd = 5) x3 = 2 * x1 + 4 * x2 + 3 y = 3 + x1 + x2 + rnorm(n = num_samples, mean = 0, sd = 1) data.frame(y, x1, x2, x3) } Notice that the way we are generating this data, the response \\(y\\) only really depends on \\(x_1\\) and \\(x_2\\). set.seed(42) exact_collin_data = gen_exact_collin_data() head(exact_collin_data) ## y x1 x2 x3 ## 1 170.7135 93.70958 76.00483 494.4385 ## 2 152.9106 74.35302 75.22376 452.6011 ## 3 152.7866 83.63128 64.98396 430.1984 ## 4 170.6306 86.32863 79.24241 492.6269 ## 5 152.3320 84.04268 66.66613 437.7499 ## 6 151.3155 78.93875 70.52757 442.9878 What happens when we attempt to fit a regression model in R using all of the predictors? exact_collin_fit = lm(y ~ x1 + x2 + x3, data = exact_collin_data) summary(exact_collin_fit) ## ## Call: ## lm(formula = y ~ x1 + x2 + x3, data = exact_collin_data) ## ## Residuals: ## Min 1Q Median 3Q Max ## -2.57662 -0.66188 -0.08253 0.63706 2.52057 ## ## Coefficients: (1 not defined because of singularities) ## Estimate Std. Error t value Pr(&gt;|t|) ## (Intercept) 2.957336 1.735165 1.704 0.0915 . ## x1 0.985629 0.009788 100.702 &lt;2e-16 *** ## x2 1.017059 0.022545 45.112 &lt;2e-16 *** ## x3 NA NA NA NA ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 ## ## Residual standard error: 1.014 on 97 degrees of freedom ## Multiple R-squared: 0.9923, Adjusted R-squared: 0.9921 ## F-statistic: 6236 on 2 and 97 DF, p-value: &lt; 2.2e-16 We see that R simply decides to exclude a variable. Why is this happening? X = cbind(1, as.matrix(exact_collin_data[,-1])) solve(t(X) %*% X) If we attempt to find \\(\\boldsymbol{\\hat{\\beta}}\\) using \\(\\left( \\boldsymbol{X}^T \\boldsymbol{X} \\right)^{-1}\\), we see that this is not possible, due to the fact that the columns of \\(\\boldsymbol{X}\\) are linearly dependent. The previous lines of code were not run, because they produce an error! When this happens, we say there is exact collinearity in the dataset. As a result of this issue, R essentially chose to fit the model y ~ x1 + x2. However notice that two other models would accomplish exactly the same fit. fit1 = lm(y ~ x1 + x2, data = exact_collin_data) fit2 = lm(y ~ x1 + x3, data = exact_collin_data) fit3 = lm(y ~ x2 + x3, data = exact_collin_data) We see that the fitted values for each of the three models are exactly the same. This is a result of \\(x_3\\) containing all of the information from \\(x_1\\) and \\(x_2\\). As long as one of \\(x_1\\) or \\(x_2\\) is included in the model, \\(x_3\\) can be used to recover the information from the variable not included. all.equal(fitted(fit1), fitted(fit2)) ## [1] TRUE all.equal(fitted(fit2), fitted(fit3)) ## [1] TRUE While their fitted values are all the same, their estimated coefficients are wildly different. The sign of \\(x_2\\) is switched in two of the models! So only fit1 properly explains the relationship between the variables, fit2 and fit3 still predict as well as fit1, despite the coefficients having little to no meaning, a concept we will return to later. coef(fit1) ## (Intercept) x1 x2 ## 2.9573357 0.9856291 1.0170586 coef(fit2) ## (Intercept) x1 x3 ## 2.1945418 0.4770998 0.2542647 coef(fit3) ## (Intercept) x2 x3 ## 1.4788921 -0.9541995 0.4928145 15.2 Collinearity Exact collinearity is an extreme example of collinearity, which occurs in multiple regression when predictor variables are highly correlated. Collinearity is often called multicollinearity, since it is a phenomenon that really only occurs during multiple regression. Looking at the seatpos dataset from the faraway package, we will see an example of this concept. The predictors in this dataset are various attributes of car drivers, such as their height, weight and age. The response variable hipcenter measures the “horizontal distance of the midpoint of the hips from a fixed location in the car in mm.” Essentially, it measures the position of the seat for a given driver. This is potentially useful information for car manufacturers considering comfort and safety when designing vehicles. We will attempt to fit a model that predicts hipcenter. Two predictor variables are immediately interesting to us: HtShoes and Ht. We certainly expect a person’s height to be highly correlated to their height when wearing shoes. We’ll pay special attention to these two variables when fitting models. library(faraway) pairs(seatpos, col = &quot;dodgerblue&quot;) round(cor(seatpos), 2) ## Age Weight HtShoes Ht Seated Arm Thigh Leg hipcenter ## Age 1.00 0.08 -0.08 -0.09 -0.17 0.36 0.09 -0.04 0.21 ## Weight 0.08 1.00 0.83 0.83 0.78 0.70 0.57 0.78 -0.64 ## HtShoes -0.08 0.83 1.00 1.00 0.93 0.75 0.72 0.91 -0.80 ## Ht -0.09 0.83 1.00 1.00 0.93 0.75 0.73 0.91 -0.80 ## Seated -0.17 0.78 0.93 0.93 1.00 0.63 0.61 0.81 -0.73 ## Arm 0.36 0.70 0.75 0.75 0.63 1.00 0.67 0.75 -0.59 ## Thigh 0.09 0.57 0.72 0.73 0.61 0.67 1.00 0.65 -0.59 ## Leg -0.04 0.78 0.91 0.91 0.81 0.75 0.65 1.00 -0.79 ## hipcenter 0.21 -0.64 -0.80 -0.80 -0.73 -0.59 -0.59 -0.79 1.00 After loading the faraway package, we do some quick checks of correlation between the predictors. Visually, we can do this with the pairs() function, which plots all possible scatterplots between pairs of variables in the dataset. We can also do this numerically with the cor() function, which when applied to a dataset, returns all pairwise correlations. Notice this is a symmetric matrix. Recall that correlation measures strength and direction of the linear relationship between two variables. The correlation between Ht and HtShoes is extremely high. So high, that rounded to two decimal places, it appears to be 1! Unlike exact collinearity, here we can still fit a model with all of the predictors, but what effect does this have? hip_model = lm(hipcenter ~ ., data = seatpos) summary(hip_model) ## ## Call: ## lm(formula = hipcenter ~ ., data = seatpos) ## ## Residuals: ## Min 1Q Median 3Q Max ## -73.827 -22.833 -3.678 25.017 62.337 ## ## Coefficients: ## Estimate Std. Error t value Pr(&gt;|t|) ## (Intercept) 436.43213 166.57162 2.620 0.0138 * ## Age 0.77572 0.57033 1.360 0.1843 ## Weight 0.02631 0.33097 0.080 0.9372 ## HtShoes -2.69241 9.75304 -0.276 0.7845 ## Ht 0.60134 10.12987 0.059 0.9531 ## Seated 0.53375 3.76189 0.142 0.8882 ## Arm -1.32807 3.90020 -0.341 0.7359 ## Thigh -1.14312 2.66002 -0.430 0.6706 ## Leg -6.43905 4.71386 -1.366 0.1824 ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 ## ## Residual standard error: 37.72 on 29 degrees of freedom ## Multiple R-squared: 0.6866, Adjusted R-squared: 0.6001 ## F-statistic: 7.94 on 8 and 29 DF, p-value: 1.306e-05 One of the first things we should notice is that the \\(F\\)-test for the regression tells us that the regression is significant, however each individual predictor is not. Another interesting result is the opposite signs of the coefficients for Ht and HtShoes. This should seem rather counter-intuitive. Increasing Ht increases hipcenter, but increasing HtShoes decreases hipcenter? This happens as a result of the predictors being highly correlated. For example, the HtShoe variable explains a large amount of the variation in Ht. When they are both in the model, their effects on the response are lessened individually, but together they still explain a large portion of the variation of hipcenter. We define \\(R_j^2\\) to be the proportion of observed variation in the \\(j\\)-th predictor explained by the other predictors. In other words \\(R_j^2\\) is the multiple R-Squared for the regression of \\(x_j\\) on each of the other predictors. ht_shoes_model = lm(HtShoes ~ . - hipcenter, data = seatpos) summary(ht_shoes_model)$r.squared ## [1] 0.9967472 Here we see that the other predictors explain \\(99.67\\%\\) of the variation in HtShoe. When fitting this model, we removed hipcenter since it is not a predictor. 15.2.1 Variance Inflation Factor. Now note that the variance of \\(\\hat{\\beta_j}\\) can be written as \\[ \\text{Var}(\\hat{\\beta_j}) = \\sigma^2 C_{jj} = \\sigma^2 \\left( \\frac{1}{1 - R_j^2} \\right) \\frac{1}{S_{x_j x_j}} \\] where \\[ S_{x_j x_j} = \\sum(x_{ij}-\\bar{x}_j)^2. \\] This gives us a way to understand how collinearity affects our regression estimates. We will call, \\[ \\frac{1}{1 - R_j^2} \\] the variance inflation factor. The variance inflation factor quantifies the effect of collinearity on the variance of our regression estimates. When \\(R_j^2\\) is large, that is close to 1, \\(x_j\\) is well explained by the other predictors. With a large \\(R_j^2\\) the variance inflation factor becomes large. This tells us that when \\(x_j\\) is highly correlated with other predictors, our estimate of \\(\\beta_j\\) is highly variable. The vif function from the faraway package calculates the VIFs for each of the predictors of a model. vif(hip_model) ## Age Weight HtShoes Ht Seated Arm Thigh ## 1.997931 3.647030 307.429378 333.137832 8.951054 4.496368 2.762886 ## Leg ## 6.694291 In practice it is common to say that any VIF greater than \\(5\\) is cause for concern. So in this example we see there is a huge multicollinearity issue as many of the predictors have a VIF greater than 5. Let’s further investigate how the presence of collinearity actually affects a model. If we add a moderate amount of noise to the data, we see that the estimates of the coefficients change drastically. This is a rather undesirable effect. Adding random noise should not affect the coefficients of a model. set.seed(1337) noise = rnorm(n = nrow(seatpos), mean = 0, sd = 5) hip_model_noise = lm(hipcenter + noise ~ ., data = seatpos) Adding the noise had such a large effect, the sign of the coefficient for Ht has changed. coef(hip_model) ## (Intercept) Age Weight HtShoes Ht Seated ## 436.43212823 0.77571620 0.02631308 -2.69240774 0.60134458 0.53375170 ## Arm Thigh Leg ## -1.32806864 -1.14311888 -6.43904627 coef(hip_model_noise) ## (Intercept) Age Weight HtShoes Ht Seated ## 415.32909380 0.76578240 0.01910958 -2.90377584 -0.12068122 2.03241638 ## Arm Thigh Leg ## -1.02127944 -0.89034509 -5.61777220 This tells us that a model with collinearity is bad at explaining the relationship between the response and the predictors. We cannot even be confident in the direction of the relationship. However, does collinearity affect prediction? plot(fitted(hip_model), fitted(hip_model_noise), col = &quot;dodgerblue&quot;, pch = 20, xlab = &quot;Predicted, Without Noise&quot;, ylab = &quot;Predicted, With Noise&quot;, cex = 1.5) abline(a = 0, b = 1, col = &quot;darkorange&quot;, lwd = 2) We see that by plotting the predicted values using both models against each other, they are actually rather similar. Let’s now look at a smaller model, hip_model_small = lm(hipcenter ~ Age + Arm + Ht, data = seatpos) summary(hip_model_small) ## ## Call: ## lm(formula = hipcenter ~ Age + Arm + Ht, data = seatpos) ## ## Residuals: ## Min 1Q Median 3Q Max ## -82.347 -24.745 -0.094 23.555 58.314 ## ## Coefficients: ## Estimate Std. Error t value Pr(&gt;|t|) ## (Intercept) 493.2491 101.0724 4.880 2.46e-05 *** ## Age 0.7988 0.5111 1.563 0.12735 ## Arm -2.9385 3.5210 -0.835 0.40979 ## Ht -3.4991 0.9954 -3.515 0.00127 ** ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 ## ## Residual standard error: 36.12 on 34 degrees of freedom ## Multiple R-squared: 0.6631, Adjusted R-squared: 0.6333 ## F-statistic: 22.3 on 3 and 34 DF, p-value: 3.649e-08 vif(hip_model_small) ## Age Arm Ht ## 1.749943 3.996766 3.508693 Immediately we see that multicollinearity isn’t an issue here. anova(hip_model_small, hip_model) ## Analysis of Variance Table ## ## Model 1: hipcenter ~ Age + Arm + Ht ## Model 2: hipcenter ~ Age + Weight + HtShoes + Ht + Seated + Arm + Thigh + ## Leg ## Res.Df RSS Df Sum of Sq F Pr(&gt;F) ## 1 34 44354 ## 2 29 41262 5 3091.9 0.4346 0.8207 Also notice that using an \\(F\\)-test to compare the two models, we would prefer the smaller model. We now investigate the effect of adding another variable to this smaller model. Specifically we want to look at adding the variable HtShoes. So now our possible predictors are HtShoes, Age, Arm, and Ht. Our response is still hipcenter. To quantify this effect we will look at a variable added plot and a partial correlation coefficient. For both of these, we will look at the residuals of two models: Regressing the response (hipcenter) against all of the predictors except the predictor of interest (HtShoes). Regressing the predictor of interest (HtShoes) against the other predictors (Age, Arm, and Ht). ht_shoes_model_small = lm(HtShoes ~ Age + Arm + Ht, data = seatpos) So now, the residuals of hip_model_small give us the variation of hipcenter that is unexplained by Age, Arm, and Ht. Similarly, the residuals of ht_shoes_model_small give us the variation of HtShoes unexplained by Age, Arm, and Ht. The correlation of these two residuals gives us the partial correlation coefficient of HtShoes and hipcenter with the effects of Age, Arm, and Ht removed. cor(resid(ht_shoes_model_small), resid(hip_model_small)) ## [1] -0.03311061 Since this value is small, close to zero, it means that the variation of hipcenter that is unexplained by Age, Arm, and Ht shows very little correlation with the variation of HtShoes that is not explained by Age, Arm, and Ht. Thus adding HtShoes to the model would likely be of little benefit. Similarly a variable added plot visualizes these residuals against each other. It is also helpful to regress the residuals of the response against the residuals of the predictor and add the regression line to the plot. plot(resid(hip_model_small) ~ resid(ht_shoes_model_small), col = &quot;dodgerblue&quot;, pch = 20, xlab = &quot;Residuals, Added Predictor&quot;, ylab = &quot;Residuals, Original Model&quot;, main = &quot;Variable Added Plot&quot;) abline(h = 0, lty = 2) abline(v = 0, lty = 2) abline(lm(resid(hip_model_small) ~ resid(ht_shoes_model_small)), col = &quot;darkorange&quot;, lwd = 2) Here the variable added plot shows almost no linear relationship. This tells us that adding HtShoes to the model would probably not be worthwhile. Since its variation is largely explained by the other predictors, adding it to the model will not do much to improve the model. However it will increase the variation of the estimates and make the model much harder to interpret. Had there been a strong linear relationship here, thus a large partial correlation coefficient, it would likely have been useful to add the additional predictor to the model. This trade-off is mostly true in general. As a model gets more predictors, errors will get smaller and its prediction will be better, but it will be harder to interpret. This is why, if we are interested in explaining the relationship between the predictors and the response, we often want a model that fits well, but with a small number of predictors with little correlation. Next chapter we will learn about methods to find models that both fit well, but also have a small number of predictors. We will also discuss overfitting. Although, adding additional predictors will always make errors smaller, sometimes we will be “fitting the noise” and such a model will not generalize to additional observations well. 15.3 Simulation Here we simulate example data with and without collinearity. We will note the difference in the distribution of the estimates of the \\(\\beta\\) parameters, in particular their variance. However, we will also notice the similarity in their \\(MSE\\). We will use the model, \\[ Y = \\beta_0 + \\beta_1 x_1 + \\beta_2 x_2 + \\epsilon \\] where \\(\\epsilon \\sim N(\\mu = 0, \\sigma^2 = 25)\\) and the \\(\\beta\\) coefficients defined below. set.seed(42) beta_0 = 7 beta_1 = 3 beta_2 = 4 sigma = 5 We will use a sample size of 10, and 2500 simulations for both situations. sample_size = 10 num_sim = 2500 We’ll first consider the situation with a collinearity issue, so we manually create the two predictor variables. x1 = c(1, 2, 3, 4, 5, 6, 7, 8, 9, 10) x2 = c(1, 2, 3, 4, 5, 7, 6, 10, 9, 8) c(sd(x1), sd(x2)) ## [1] 3.02765 3.02765 cor(x1, x2) ## [1] 0.9393939 Notice that they have extremely high correlation. true_line_bad = beta_0 + beta_1 * x1 + beta_2 * x2 beta_hat_bad = matrix(0, num_sim, 2) mse_bad = rep(0, num_sim) We perform the simulation 2500 times, each time fitting a regression model, and storing the estimated coefficients and the MSE. for (s in 1:num_sim) { y = true_line_bad + rnorm(n = sample_size, mean = 0, sd = sigma) reg_out = lm(y ~ x1 + x2) beta_hat_bad[s, ] = coef(reg_out)[-1] mse_bad[s] = mean(resid(reg_out) ^ 2) } Now we move to the situation without a collinearity issue, so we again manually create the two predictor variables. z1 = c(1, 2, 3, 4, 5, 6, 7, 8, 9, 10) z2 = c(9, 2, 7, 4, 5, 6, 3, 8, 1, 10) Notice that the standard deviations of each are the same as before, however, now the correlation is extremely close to 0. c(sd(z1), sd(z2)) ## [1] 3.02765 3.02765 cor(z1, z2) ## [1] 0.03030303 true_line_good = beta_0 + beta_1 * z1 + beta_2 * z2 beta_hat_good = matrix(0, num_sim, 2) mse_good = rep(0, num_sim) We then perform simulations and store the same results. for (s in 1:num_sim) { y = true_line_good + rnorm(n = sample_size, mean = 0, sd = sigma) reg_out = lm(y ~ z1 + z2) beta_hat_good[s, ] = coef(reg_out)[-1] mse_good[s] = mean(resid(reg_out) ^ 2) } We’ll now investigate the differences. par(mfrow = c(1, 2)) hist(beta_hat_bad[, 1], col = &quot;darkorange&quot;, border = &quot;dodgerblue&quot;, main = expression(&quot;Histogram of &quot; *hat(beta)[1]* &quot; with Collinearity&quot;), xlab = expression(hat(beta)[1]), breaks = 20) hist(beta_hat_good[, 1], col = &quot;darkorange&quot;, border = &quot;dodgerblue&quot;, main = expression(&quot;Histogram of &quot; *hat(beta)[1]* &quot; without Collinearity&quot;), xlab = expression(hat(beta)[1]), breaks = 20) First, for \\(\\beta_1\\), which has a true value of \\(3\\), we see that both with and without collinearity, the simulated values are centered near \\(3\\). mean(beta_hat_bad[, 1]) ## [1] 2.963325 mean(beta_hat_good[, 1]) ## [1] 3.013414 The way the predictors were created, the \\(S_{x_j x_j}\\) portion of the variance is the same for the predictors in both cases, but the variance is still much larger in the simulations performed with collinearity. The variance is so large in the collinear case, that sometimes the estimated coefficient for \\(\\beta_1\\) is negative! sd(beta_hat_bad[, 1]) ## [1] 1.633294 sd(beta_hat_good[, 1]) ## [1] 0.5484684 par(mfrow = c(1, 2)) hist(beta_hat_bad[, 2], col = &quot;darkorange&quot;, border = &quot;dodgerblue&quot;, main = expression(&quot;Histogram of &quot; *hat(beta)[2]* &quot; with Collinearity&quot;), xlab = expression(hat(beta)[2]), breaks = 20) hist(beta_hat_good[, 2], col = &quot;darkorange&quot;, border = &quot;dodgerblue&quot;, main = expression(&quot;Histogram of &quot; *hat(beta)[2]* &quot; without Collinearity&quot;), xlab = expression(hat(beta)[2]), breaks = 20) We see the same issues with \\(\\beta_2\\). On average the estimates are correct, but the variance is again much larger with collinearity. mean(beta_hat_bad[, 2]) ## [1] 4.025059 mean(beta_hat_good[, 2]) ## [1] 4.004913 sd(beta_hat_bad[, 2]) ## [1] 1.642592 sd(beta_hat_good[, 2]) ## [1] 0.5470381 par(mfrow = c(1, 2)) hist(mse_bad, col = &quot;darkorange&quot;, border = &quot;dodgerblue&quot;, main = &quot;MSE, with Collinearity&quot;, xlab = &quot;MSE&quot;) hist(mse_good, col = &quot;darkorange&quot;, border = &quot;dodgerblue&quot;, main = &quot;MSE, without Collinearity&quot;, xlab = &quot;MSE&quot;) Interestingly, in both cases, the MSE is roughly the same on average. Again, this is because collinearity affects a model’s ability to explain, but not predict. mean(mse_bad) ## [1] 17.7186 mean(mse_good) ## [1] 17.70513 15.4 R Markdown The R Markdown file for this chapter can be found here: collinearity.Rmd The file was created using R version 4.3.2. "],["variable-selection-and-model-building.html", "Chapter 16 Variable Selection and Model Building 16.1 Quality Criterion 16.2 Selection Procedures 16.3 Higher Order Terms 16.4 Explanation versus Prediction 16.5 R Markdown", " Chapter 16 Variable Selection and Model Building “Choose well. Your choice is brief, and yet endless.” — Johann Wolfgang von Goethe After reading this chapter you will be able to: Understand the trade-off between goodness-of-fit and model complexity. Use variable selection procedures to find a good model from a set of possible models. Understand the two uses of models: explanation and prediction. Last chapter we saw how correlation between predictor variables can have undesirable effects on models. We used variance inflation factors to assess the severity of the collinearity issues caused by these correlations. We also saw how fitting a smaller model, leaving out some of the correlated predictors, results in a model which no longer suffers from collinearity issues. But how should we choose this smaller model? This chapter, we will discuss several criteria and procedures for choosing a “good” model from among a choice of many. 16.1 Quality Criterion So far, we have seen criteria such as \\(R^2\\) and \\(\\text{RMSE}\\) for assessing quality of fit. However, both of these have a fatal flaw. By increasing the size of a model, that is adding predictors, that can at worst not improve. It is impossible to add a predictor to a model and make \\(R^2\\) or \\(\\text{RMSE}\\) worse. That means, if we were to use either of these to choose between models, we would always simply choose the larger model. Eventually we would simply be fitting to noise. This suggests that we need a quality criterion that takes into account the size of the model, since our preference is for small models that still fit well. We are willing to sacrifice a small amount of “goodness-of-fit” for obtaining a smaller model. (Here we use “goodness-of-fit” to simply mean how far the data is from the model, the smaller the errors the better. Often in statistics, goodness-of-fit can have a more precise meaning.) We will look at three criteria that do this explicitly: \\(\\text{AIC}\\), \\(\\text{BIC}\\), and Adjusted \\(R^2\\). We will also look at one, Cross-Validated \\(\\text{RMSE}\\), which implicitly considers the size of the model. 16.1.1 Akaike Information Criterion The first criteria we will discuss is the Akaike Information Criterion, or \\(\\text{AIC}\\) for short. (Note that, when Akaike first introduced this metric, it was simply called An Information Criterion. The A has changed meaning over the years.) Recall, the maximized log-likelihood of a regression model can be written as \\[ \\log L(\\boldsymbol{\\hat{\\beta}}, \\hat{\\sigma}^2) = -\\frac{n}{2}\\log(2\\pi) - \\frac{n}{2}\\log\\left(\\frac{\\text{RSS}}{n}\\right) - \\frac{n}{2}, \\] where \\(\\text{RSS} = \\sum_{i=1}^n (y_i - \\hat{y}_i) ^ 2\\) and \\(\\boldsymbol{\\hat{\\beta}}\\) and \\(\\hat{\\sigma}^2\\) were chosen to maximize the likelihood. Then we can define \\(\\text{AIC}\\) as \\[ \\text{AIC} = -2 \\log L(\\boldsymbol{\\hat{\\beta}}, \\hat{\\sigma}^2) + 2p = n + n \\log(2\\pi) + n \\log\\left(\\frac{\\text{RSS}}{n}\\right) + 2p, \\] which is a measure of quality of the model. The smaller the \\(\\text{AIC}\\), the better. To see why, let’s talk about the two main components of \\(\\text{AIC}\\), the likelihood (which measures “goodness-of-fit”) and the penalty (which is a function of the size of the model). The likelihood portion of \\(\\text{AIC}\\) is given by \\[ -2 \\log L(\\boldsymbol{\\hat{\\beta}}, \\hat{\\sigma}^2) = n + n \\log(2\\pi) + n \\log\\left(\\frac{\\text{RSS}}{n}\\right). \\] For the sake of comparing models, the only term here that will change is \\(n \\log\\left(\\frac{\\text{RSS}}{n}\\right)\\), which is a function of \\(\\text{RSS}\\). The \\[ n + n \\log(2\\pi) \\] terms will be constant across all models applied to the same data. So, when a model fits well, that is, has a low \\(\\text{RSS}\\), then this likelihood component will be small. Similarly, we can discuss the penalty component of \\(\\text{AIC}\\) which is, \\[ 2p, \\] where \\(p\\) is the number of \\(\\beta\\) parameters in the model. We call this a penalty, because it is large when \\(p\\) is large, but we are seeking to find a small \\(\\text{AIC}\\) Thus, a good model, that is one with a small \\(\\text{AIC}\\), will have a good balance between fitting well, and using a small number of parameters. For comparing models \\[ \\text{AIC} = n\\log\\left(\\frac{\\text{RSS}}{n}\\right) + 2p \\] is a sufficient expression, as \\(n + n \\log(2\\pi)\\) is the same across all models for any particular dataset. 16.1.2 Bayesian Information Criterion The Bayesian Information Criterion, or \\(\\text{BIC}\\), is similar to \\(\\text{AIC}\\), but has a larger penalty. \\(\\text{BIC}\\) also quantifies the trade-off between a model which fits well and the number of model parameters, however for a reasonable sample size, generally picks a smaller model than \\(\\text{AIC}\\). Again, for model selection use the model with the smallest \\(\\text{BIC}\\). \\[ \\text{BIC} = -2 \\log L(\\boldsymbol{\\hat{\\beta}}, \\hat{\\sigma}^2) + \\log(n) p = n + n\\log(2\\pi) + n\\log\\left(\\frac{\\text{RSS}}{n}\\right) + \\log(n)p. \\] Notice that the \\(\\text{AIC}\\) penalty was \\[ 2p, \\] whereas for \\(\\text{BIC}\\), the penalty is \\[ \\log(n) p. \\] So, for any dataset where \\(log(n) &gt; 2\\) the \\(\\text{BIC}\\) penalty will be larger than the \\(\\text{AIC}\\) penalty, thus \\(\\text{BIC}\\) will likely prefer a smaller model. Note that sometimes the penalty is considered a general expression of the form \\[ k \\cdot p. \\] Then, for \\(\\text{AIC}\\) \\(k = 2\\), and for \\(\\text{BIC}\\) \\(k = \\log(n)\\). For comparing models \\[ \\text{BIC} = n\\log\\left(\\frac{\\text{RSS}}{n}\\right) + \\log(n)p \\] is again a sufficient expression, as \\(n + n \\log(2\\pi)\\) is the same across all models for any particular dataset. 16.1.3 Adjusted R-Squared Recall, \\[ R^2 = 1 - \\frac{\\text{SSE}}{\\text{SST}} = 1 - \\frac{\\sum_{i=1}^{n}(y_i - \\hat{y}_i)^2}{\\sum_{i=1}^{n}(y_i - \\bar{y})^2}. \\] We now define \\[ R_a^2 = 1 - \\frac{\\text{SSE}/(n-p)}{\\text{SST}/(n-1)} = 1 - \\left( \\frac{n-1}{n-p} \\right)(1-R^2) \\] which we call the Adjusted \\(R^2\\). Unlike \\(R^2\\) which can never become smaller with added predictors, Adjusted \\(R^2\\) effectively penalizes for additional predictors, and can decrease with added predictors. Like \\(R^2\\), larger is still better. 16.1.4 Cross-Validated RMSE Each of the previous three metrics explicitly used \\(p\\), the number of parameters, in their calculations. Thus, they all explicitly limit the size of models chosen when used to compare models. We’ll now briefly introduce overfitting and cross-validation. make_poly_data = function(sample_size = 11) { x = seq(0, 10) y = 3 + x + 4 * x ^ 2 + rnorm(n = sample_size, mean = 0, sd = 20) data.frame(x, y) } set.seed(1234) poly_data = make_poly_data() Here we have generated data where the mean of \\(Y\\) is a quadratic function of a single predictor \\(x\\), specifically, \\[ Y = 3 + x + 4 x ^ 2 + \\epsilon. \\] We’ll now fit two models to this data, one which has the correct form, quadratic, and one that is large, which includes terms up to and including an eighth degree. fit_quad = lm(y ~ poly(x, degree = 2), data = poly_data) fit_big = lm(y ~ poly(x, degree = 8), data = poly_data) We then plot the data and the results of the two models. plot(y ~ x, data = poly_data, ylim = c(-100, 400), cex = 2, pch = 20) xplot = seq(0, 10, by = 0.1) lines(xplot, predict(fit_quad, newdata = data.frame(x = xplot)), col = &quot;dodgerblue&quot;, lwd = 2, lty = 1) lines(xplot, predict(fit_big, newdata = data.frame(x = xplot)), col = &quot;darkorange&quot;, lwd = 2, lty = 2) We can see that the solid blue curve models this data rather nicely. The dashed orange curve fits the points better, making smaller errors, however it is unlikely that it is correctly modeling the true relationship between \\(x\\) and \\(y\\). It is fitting the random noise. This is an example of overfitting. We see that the larger model indeed has a lower \\(\\text{RMSE}\\). sqrt(mean(resid(fit_quad) ^ 2)) ## [1] 17.61812 sqrt(mean(resid(fit_big) ^ 2)) ## [1] 10.4197 To correct for this, we will introduce cross-validation. We define the leave-one-out cross-validated RMSE to be \\[ \\text{RMSE}_{\\text{LOOCV}} = \\sqrt{\\frac{1}{n} \\sum_{i=1}^n e_{[i]}^2}. \\] The \\(e_{[i]}\\) are the residual for the \\(i\\)th observation, when that observation is not used to fit the model. \\[ e_{[i]} = y_{i} - \\hat{y}_{[i]} \\] That is, the fitted value is calculated as \\[ \\hat{y}_{[i]} = \\boldsymbol{x}_i ^ \\top \\hat{\\beta}_{[i]} \\] where \\(\\hat{\\beta}_{[i]}\\) are the estimated coefficients when the \\(i\\)th observation is removed from the dataset. In general, to perform this calculation, we would be required to fit the model \\(n\\) times, once with each possible observation removed. However, for leave-one-out cross-validation and linear models, the equation can be rewritten as \\[ \\text{RMSE}_{\\text{LOOCV}} = \\sqrt{\\frac{1}{n}\\sum_{i=1}^n \\left(\\frac{e_{i}}{1-h_{i}}\\right)^2}, \\] where \\(h_i\\) are the leverages and \\(e_i\\) are the usual residuals. This is great, because now we can obtain the LOOCV \\(\\text{RMSE}\\) by fitting only one model! In practice 5 or 10 fold cross-validation are much more popular. For example, in 5-fold cross-validation, the model is fit 5 times, each time leaving out a fifth of the data, then predicting on those values. We’ll leave in-depth examination of cross-validation to a machine learning course, and simply use LOOCV here. Let’s calculate LOOCV \\(\\text{RMSE}\\) for both models, then discuss why we want to do so. We first write a function which calculates the LOOCV \\(\\text{RMSE}\\) as defined using the shortcut formula for linear models. calc_loocv_rmse = function(model) { sqrt(mean((resid(model) / (1 - hatvalues(model))) ^ 2)) } Then calculate the metric for both models. calc_loocv_rmse(fit_quad) ## [1] 23.57189 calc_loocv_rmse(fit_big) ## [1] 1334.357 Now we see that the quadratic model has a much smaller LOOCV \\(\\text{RMSE}\\), so we would prefer this quadratic model. This is because the large model has severely over-fit the data. By leaving a single data point out and fitting the large model, the resulting fit is much different than the fit using all of the data. For example, let’s leave out the third data point and fit both models, then plot the result. fit_quad_removed = lm(y ~ poly(x, degree = 2), data = poly_data[-3, ]) fit_big_removed = lm(y ~ poly(x, degree = 8), data = poly_data[-3, ]) plot(y ~ x, data = poly_data, ylim = c(-100, 400), cex = 2, pch = 20) xplot = seq(0, 10, by = 0.1) lines(xplot, predict(fit_quad_removed, newdata = data.frame(x = xplot)), col = &quot;dodgerblue&quot;, lwd = 2, lty = 1) lines(xplot, predict(fit_big_removed, newdata = data.frame(x = xplot)), col = &quot;darkorange&quot;, lwd = 2, lty = 2) We see that on average, the solid blue line for the quadratic model has similar errors as before. It has changed very slightly. However, the dashed orange line for the large model, has a huge error at the point that was removed and is much different than the previous fit. This is the purpose of cross-validation. By assessing how the model fits points that were not used to perform the regression, we get an idea of how well the model will work for future observations. It assesses how well the model works in general, not simply on the observed data. 16.2 Selection Procedures We’ve now seen a number of model quality criteria, but now we need to address which models to consider. Model selection involves both a quality criterion, plus a search procedure. library(faraway) hipcenter_mod = lm(hipcenter ~ ., data = seatpos) coef(hipcenter_mod) ## (Intercept) Age Weight HtShoes Ht Seated ## 436.43212823 0.77571620 0.02631308 -2.69240774 0.60134458 0.53375170 ## Arm Thigh Leg ## -1.32806864 -1.14311888 -6.43904627 Let’s return to the seatpos data from the faraway package. Now, let’s consider only models with first order terms, thus no interactions and no polynomials. There are eight predictors in this model. So if we consider all possible models, ranging from using 0 predictors, to all eight predictors, there are \\[ \\sum_{k = 0}^{p - 1} {{p - 1} \\choose {k}} = 2 ^ {p - 1} = 2 ^ 8 = 256 \\] possible models. If we had 10 or more predictors, we would already be considering over 1000 models! For this reason, we often search through possible models in an intelligent way, bypassing some models that are unlikely to be considered good. We will consider three search procedures: backwards, forwards, and stepwise. 16.2.1 Backward Search Backward selection procedures start with all possible predictors in the model, then consider how deleting a single predictor will affect a chosen metric. Let’s try this on the seatpos data. We will use the step() function in R which by default uses \\(\\text{AIC}\\) as its metric of choice. hipcenter_mod_back_aic = step(hipcenter_mod, direction = &quot;backward&quot;) ## Start: AIC=283.62 ## hipcenter ~ Age + Weight + HtShoes + Ht + Seated + Arm + Thigh + ## Leg ## ## Df Sum of Sq RSS AIC ## - Ht 1 5.01 41267 281.63 ## - Weight 1 8.99 41271 281.63 ## - Seated 1 28.64 41290 281.65 ## - HtShoes 1 108.43 41370 281.72 ## - Arm 1 164.97 41427 281.78 ## - Thigh 1 262.76 41525 281.87 ## &lt;none&gt; 41262 283.62 ## - Age 1 2632.12 43894 283.97 ## - Leg 1 2654.85 43917 283.99 ## ## Step: AIC=281.63 ## hipcenter ~ Age + Weight + HtShoes + Seated + Arm + Thigh + Leg ## ## Df Sum of Sq RSS AIC ## - Weight 1 11.10 41278 279.64 ## - Seated 1 30.52 41297 279.66 ## - Arm 1 160.50 41427 279.78 ## - Thigh 1 269.08 41536 279.88 ## - HtShoes 1 971.84 42239 280.51 ## &lt;none&gt; 41267 281.63 ## - Leg 1 2664.65 43931 282.01 ## - Age 1 2808.52 44075 282.13 ## ## Step: AIC=279.64 ## hipcenter ~ Age + HtShoes + Seated + Arm + Thigh + Leg ## ## Df Sum of Sq RSS AIC ## - Seated 1 35.10 41313 277.67 ## - Arm 1 156.47 41434 277.78 ## - Thigh 1 285.16 41563 277.90 ## - HtShoes 1 975.48 42253 278.53 ## &lt;none&gt; 41278 279.64 ## - Leg 1 2661.39 43939 280.01 ## - Age 1 3011.86 44290 280.31 ## ## Step: AIC=277.67 ## hipcenter ~ Age + HtShoes + Arm + Thigh + Leg ## ## Df Sum of Sq RSS AIC ## - Arm 1 172.02 41485 275.83 ## - Thigh 1 344.61 41658 275.99 ## - HtShoes 1 1853.43 43166 277.34 ## &lt;none&gt; 41313 277.67 ## - Leg 1 2871.07 44184 278.22 ## - Age 1 2976.77 44290 278.31 ## ## Step: AIC=275.83 ## hipcenter ~ Age + HtShoes + Thigh + Leg ## ## Df Sum of Sq RSS AIC ## - Thigh 1 472.8 41958 274.26 ## &lt;none&gt; 41485 275.83 ## - HtShoes 1 2340.7 43826 275.92 ## - Age 1 3501.0 44986 276.91 ## - Leg 1 3591.7 45077 276.98 ## ## Step: AIC=274.26 ## hipcenter ~ Age + HtShoes + Leg ## ## Df Sum of Sq RSS AIC ## &lt;none&gt; 41958 274.26 ## - Age 1 3108.8 45067 274.98 ## - Leg 1 3476.3 45434 275.28 ## - HtShoes 1 4218.6 46176 275.90 We start with the model hipcenter ~ ., which is otherwise known as hipcenter ~ Age + Weight + HtShoes + Ht + Seated + Arm + Thigh + Leg. R will then repeatedly attempt to delete a predictor until it stops, or reaches the model hipcenter ~ 1, which contains no predictors. At each “step,” R reports the current model, its \\(\\text{AIC}\\), and the possible steps with their \\(\\text{RSS}\\) and more importantly \\(\\text{AIC}\\). In this example, at the first step, the current model is hipcenter ~ Age + Weight + HtShoes + Ht + Seated + Arm + Thigh + Leg which has an AIC of 283.62. Note that when R is calculating this value, it is using extractAIC(), which uses the expression \\[ \\text{AIC} = n\\log\\left(\\frac{\\text{RSS}}{n}\\right) + 2p, \\] which we quickly verify. extractAIC(hipcenter_mod) # returns both p and AIC ## [1] 9.000 283.624 n = length(resid(hipcenter_mod)) (p = length(coef(hipcenter_mod))) ## [1] 9 n * log(mean(resid(hipcenter_mod) ^ 2)) + 2 * p ## [1] 283.624 Returning to the first step, R then gives us a row which shows the effect of deleting each of the current predictors. The - signs at the beginning of each row indicate we are considering removing a predictor. There is also a row with &lt;none&gt; which is a row for keeping the current model. Notice that this row has the smallest \\(\\text{RSS}\\), as it is the largest model. We see that every row above &lt;none&gt; has a smaller \\(\\text{AIC}\\) than the row for &lt;none&gt; with the one at the top, Ht, giving the lowest \\(\\text{AIC}\\). Thus we remove Ht from the model, and continue the process. Notice, in the second step, we start with the model hipcenter ~ Age + Weight + HtShoes + Seated + Arm + Thigh + Leg and the variable Ht is no longer considered. We continue the process until we reach the model hipcenter ~ Age + HtShoes + Leg. At this step, the row for &lt;none&gt; tops the list, as removing any additional variable will not improve the \\(\\text{AIC}\\) This is the model which is stored in hipcenter_mod_back_aic. coef(hipcenter_mod_back_aic) ## (Intercept) Age HtShoes Leg ## 456.2136538 0.5998327 -2.3022555 -6.8297461 We could also search through the possible models in a backwards fashion using \\(\\text{BIC}\\). To do so, we again use the step() function, but now specify k = log(n), where n stores the number of observations in the data. n = length(resid(hipcenter_mod)) hipcenter_mod_back_bic = step(hipcenter_mod, direction = &quot;backward&quot;, k = log(n)) ## Start: AIC=298.36 ## hipcenter ~ Age + Weight + HtShoes + Ht + Seated + Arm + Thigh + ## Leg ## ## Df Sum of Sq RSS AIC ## - Ht 1 5.01 41267 294.73 ## - Weight 1 8.99 41271 294.73 ## - Seated 1 28.64 41290 294.75 ## - HtShoes 1 108.43 41370 294.82 ## - Arm 1 164.97 41427 294.88 ## - Thigh 1 262.76 41525 294.97 ## - Age 1 2632.12 43894 297.07 ## - Leg 1 2654.85 43917 297.09 ## &lt;none&gt; 41262 298.36 ## ## Step: AIC=294.73 ## hipcenter ~ Age + Weight + HtShoes + Seated + Arm + Thigh + Leg ## ## Df Sum of Sq RSS AIC ## - Weight 1 11.10 41278 291.10 ## - Seated 1 30.52 41297 291.12 ## - Arm 1 160.50 41427 291.24 ## - Thigh 1 269.08 41536 291.34 ## - HtShoes 1 971.84 42239 291.98 ## - Leg 1 2664.65 43931 293.47 ## - Age 1 2808.52 44075 293.59 ## &lt;none&gt; 41267 294.73 ## ## Step: AIC=291.1 ## hipcenter ~ Age + HtShoes + Seated + Arm + Thigh + Leg ## ## Df Sum of Sq RSS AIC ## - Seated 1 35.10 41313 287.50 ## - Arm 1 156.47 41434 287.61 ## - Thigh 1 285.16 41563 287.73 ## - HtShoes 1 975.48 42253 288.35 ## - Leg 1 2661.39 43939 289.84 ## - Age 1 3011.86 44290 290.14 ## &lt;none&gt; 41278 291.10 ## ## Step: AIC=287.5 ## hipcenter ~ Age + HtShoes + Arm + Thigh + Leg ## ## Df Sum of Sq RSS AIC ## - Arm 1 172.02 41485 284.02 ## - Thigh 1 344.61 41658 284.18 ## - HtShoes 1 1853.43 43166 285.53 ## - Leg 1 2871.07 44184 286.41 ## - Age 1 2976.77 44290 286.50 ## &lt;none&gt; 41313 287.50 ## ## Step: AIC=284.02 ## hipcenter ~ Age + HtShoes + Thigh + Leg ## ## Df Sum of Sq RSS AIC ## - Thigh 1 472.8 41958 280.81 ## - HtShoes 1 2340.7 43826 282.46 ## - Age 1 3501.0 44986 283.46 ## - Leg 1 3591.7 45077 283.54 ## &lt;none&gt; 41485 284.02 ## ## Step: AIC=280.81 ## hipcenter ~ Age + HtShoes + Leg ## ## Df Sum of Sq RSS AIC ## - Age 1 3108.8 45067 279.89 ## - Leg 1 3476.3 45434 280.20 ## &lt;none&gt; 41958 280.81 ## - HtShoes 1 4218.6 46176 280.81 ## ## Step: AIC=279.89 ## hipcenter ~ HtShoes + Leg ## ## Df Sum of Sq RSS AIC ## - Leg 1 3038.8 48105 278.73 ## &lt;none&gt; 45067 279.89 ## - HtShoes 1 5004.4 50071 280.25 ## ## Step: AIC=278.73 ## hipcenter ~ HtShoes ## ## Df Sum of Sq RSS AIC ## &lt;none&gt; 48105 278.73 ## - HtShoes 1 83534 131639 313.35 The procedure is exactly the same, except at each step we look to improve the \\(\\text{BIC}\\), which R still labels \\(\\text{AIC}\\) in the output. The variable hipcenter_mod_back_bic stores the model chosen by this procedure. coef(hipcenter_mod_back_bic) ## (Intercept) HtShoes ## 565.592659 -4.262091 We note that this model is smaller, has fewer predictors, than the model chosen by \\(\\text{AIC}\\), which is what we would expect. Also note that while both models are different, neither uses both Ht and HtShoes which are extremely correlated. We can use information from the summary() function to compare their Adjusted \\(R^2\\) values. Note that either selected model performs better than the original full model. summary(hipcenter_mod)$adj.r.squared ## [1] 0.6000855 summary(hipcenter_mod_back_aic)$adj.r.squared ## [1] 0.6531427 summary(hipcenter_mod_back_bic)$adj.r.squared ## [1] 0.6244149 We can also calculate the LOOCV \\(\\text{RMSE}\\) for both selected models, as well as the full model. calc_loocv_rmse(hipcenter_mod) ## [1] 44.44564 calc_loocv_rmse(hipcenter_mod_back_aic) ## [1] 37.58473 calc_loocv_rmse(hipcenter_mod_back_bic) ## [1] 37.40564 We see that we would prefer the model chosen via \\(\\text{BIC}\\) if using LOOCV \\(\\text{RMSE}\\) as our metric. 16.2.2 Forward Search Forward selection is the exact opposite of backwards selection. Here we tell R to start with a model using no predictors, that is hipcenter ~ 1, then at each step R will attempt to add a predictor until it finds a good model or reaches hipcenter ~ Age + Weight + HtShoes + Ht + Seated + Arm + Thigh + Leg. hipcenter_mod_start = lm(hipcenter ~ 1, data = seatpos) hipcenter_mod_forw_aic = step( hipcenter_mod_start, scope = hipcenter ~ Age + Weight + HtShoes + Ht + Seated + Arm + Thigh + Leg, direction = &quot;forward&quot;) ## Start: AIC=311.71 ## hipcenter ~ 1 ## ## Df Sum of Sq RSS AIC ## + Ht 1 84023 47616 275.07 ## + HtShoes 1 83534 48105 275.45 ## + Leg 1 81568 50071 276.98 ## + Seated 1 70392 61247 284.63 ## + Weight 1 53975 77664 293.66 ## + Thigh 1 46010 85629 297.37 ## + Arm 1 45065 86574 297.78 ## &lt;none&gt; 131639 311.71 ## + Age 1 5541 126098 312.07 ## ## Step: AIC=275.07 ## hipcenter ~ Ht ## ## Df Sum of Sq RSS AIC ## + Leg 1 2781.10 44835 274.78 ## &lt;none&gt; 47616 275.07 ## + Age 1 2353.51 45262 275.14 ## + Weight 1 195.86 47420 276.91 ## + Seated 1 101.56 47514 276.99 ## + Arm 1 75.78 47540 277.01 ## + HtShoes 1 25.76 47590 277.05 ## + Thigh 1 4.63 47611 277.06 ## ## Step: AIC=274.78 ## hipcenter ~ Ht + Leg ## ## Df Sum of Sq RSS AIC ## + Age 1 2896.60 41938 274.24 ## &lt;none&gt; 44835 274.78 ## + Arm 1 522.72 44312 276.33 ## + Weight 1 445.10 44390 276.40 ## + HtShoes 1 34.11 44801 276.75 ## + Thigh 1 32.96 44802 276.75 ## + Seated 1 1.12 44834 276.78 ## ## Step: AIC=274.24 ## hipcenter ~ Ht + Leg + Age ## ## Df Sum of Sq RSS AIC ## &lt;none&gt; 41938 274.24 ## + Thigh 1 372.71 41565 275.90 ## + Arm 1 257.09 41681 276.01 ## + Seated 1 121.26 41817 276.13 ## + Weight 1 46.83 41891 276.20 ## + HtShoes 1 13.38 41925 276.23 Again, by default R uses \\(\\text{AIC}\\) as its quality metric when using the step() function. Also note that now the rows begin with a + which indicates addition of predictors to the current model from any step. hipcenter_mod_forw_bic = step( hipcenter_mod_start, scope = hipcenter ~ Age + Weight + HtShoes + Ht + Seated + Arm + Thigh + Leg, direction = &quot;forward&quot;, k = log(n)) ## Start: AIC=313.35 ## hipcenter ~ 1 ## ## Df Sum of Sq RSS AIC ## + Ht 1 84023 47616 278.34 ## + HtShoes 1 83534 48105 278.73 ## + Leg 1 81568 50071 280.25 ## + Seated 1 70392 61247 287.91 ## + Weight 1 53975 77664 296.93 ## + Thigh 1 46010 85629 300.64 ## + Arm 1 45065 86574 301.06 ## &lt;none&gt; 131639 313.35 ## + Age 1 5541 126098 315.35 ## ## Step: AIC=278.34 ## hipcenter ~ Ht ## ## Df Sum of Sq RSS AIC ## &lt;none&gt; 47616 278.34 ## + Leg 1 2781.10 44835 279.69 ## + Age 1 2353.51 45262 280.05 ## + Weight 1 195.86 47420 281.82 ## + Seated 1 101.56 47514 281.90 ## + Arm 1 75.78 47540 281.92 ## + HtShoes 1 25.76 47590 281.96 ## + Thigh 1 4.63 47611 281.98 We can make the same modification as last time to instead use \\(\\text{BIC}\\) with forward selection. summary(hipcenter_mod)$adj.r.squared ## [1] 0.6000855 summary(hipcenter_mod_forw_aic)$adj.r.squared ## [1] 0.6533055 summary(hipcenter_mod_forw_bic)$adj.r.squared ## [1] 0.6282374 We can compare the two selected models’ Adjusted \\(R^2\\) as well as their LOOCV \\(\\text{RMSE}\\) The results are very similar to those using backwards selection, although the models are not exactly the same. calc_loocv_rmse(hipcenter_mod) ## [1] 44.44564 calc_loocv_rmse(hipcenter_mod_forw_aic) ## [1] 37.62516 calc_loocv_rmse(hipcenter_mod_forw_bic) ## [1] 37.2511 16.2.3 Stepwise Search Stepwise search checks going both backwards and forwards at every step. It considers the addition of any variable not currently in the model, as well as the removal of any variable currently in the model. Here we perform stepwise search using \\(\\text{AIC}\\) as our metric. We start with the model hipcenter ~ 1 and search up to hipcenter ~ Age + Weight + HtShoes + Ht + Seated + Arm + Thigh + Leg. Notice that at many of the steps, some row begin with -, while others begin with +. hipcenter_mod_both_aic = step( hipcenter_mod_start, scope = hipcenter ~ Age + Weight + HtShoes + Ht + Seated + Arm + Thigh + Leg, direction = &quot;both&quot;) ## Start: AIC=311.71 ## hipcenter ~ 1 ## ## Df Sum of Sq RSS AIC ## + Ht 1 84023 47616 275.07 ## + HtShoes 1 83534 48105 275.45 ## + Leg 1 81568 50071 276.98 ## + Seated 1 70392 61247 284.63 ## + Weight 1 53975 77664 293.66 ## + Thigh 1 46010 85629 297.37 ## + Arm 1 45065 86574 297.78 ## &lt;none&gt; 131639 311.71 ## + Age 1 5541 126098 312.07 ## ## Step: AIC=275.07 ## hipcenter ~ Ht ## ## Df Sum of Sq RSS AIC ## + Leg 1 2781 44835 274.78 ## &lt;none&gt; 47616 275.07 ## + Age 1 2354 45262 275.14 ## + Weight 1 196 47420 276.91 ## + Seated 1 102 47514 276.99 ## + Arm 1 76 47540 277.01 ## + HtShoes 1 26 47590 277.05 ## + Thigh 1 5 47611 277.06 ## - Ht 1 84023 131639 311.71 ## ## Step: AIC=274.78 ## hipcenter ~ Ht + Leg ## ## Df Sum of Sq RSS AIC ## + Age 1 2896.6 41938 274.24 ## &lt;none&gt; 44835 274.78 ## - Leg 1 2781.1 47616 275.07 ## + Arm 1 522.7 44312 276.33 ## + Weight 1 445.1 44390 276.40 ## + HtShoes 1 34.1 44801 276.75 ## + Thigh 1 33.0 44802 276.75 ## + Seated 1 1.1 44834 276.78 ## - Ht 1 5236.3 50071 276.98 ## ## Step: AIC=274.24 ## hipcenter ~ Ht + Leg + Age ## ## Df Sum of Sq RSS AIC ## &lt;none&gt; 41938 274.24 ## - Age 1 2896.6 44835 274.78 ## - Leg 1 3324.2 45262 275.14 ## - Ht 1 4238.3 46176 275.90 ## + Thigh 1 372.7 41565 275.90 ## + Arm 1 257.1 41681 276.01 ## + Seated 1 121.3 41817 276.13 ## + Weight 1 46.8 41891 276.20 ## + HtShoes 1 13.4 41925 276.23 We could again instead use \\(\\text{BIC}\\) as our metric. hipcenter_mod_both_bic = step( hipcenter_mod_start, scope = hipcenter ~ Age + Weight + HtShoes + Ht + Seated + Arm + Thigh + Leg, direction = &quot;both&quot;, k = log(n)) ## Start: AIC=313.35 ## hipcenter ~ 1 ## ## Df Sum of Sq RSS AIC ## + Ht 1 84023 47616 278.34 ## + HtShoes 1 83534 48105 278.73 ## + Leg 1 81568 50071 280.25 ## + Seated 1 70392 61247 287.91 ## + Weight 1 53975 77664 296.93 ## + Thigh 1 46010 85629 300.64 ## + Arm 1 45065 86574 301.06 ## &lt;none&gt; 131639 313.35 ## + Age 1 5541 126098 315.35 ## ## Step: AIC=278.34 ## hipcenter ~ Ht ## ## Df Sum of Sq RSS AIC ## &lt;none&gt; 47616 278.34 ## + Leg 1 2781 44835 279.69 ## + Age 1 2354 45262 280.05 ## + Weight 1 196 47420 281.82 ## + Seated 1 102 47514 281.90 ## + Arm 1 76 47540 281.92 ## + HtShoes 1 26 47590 281.96 ## + Thigh 1 5 47611 281.98 ## - Ht 1 84023 131639 313.35 Adjusted \\(R^2\\) and LOOCV \\(\\text{RMSE}\\) comparisons are similar to backwards and forwards, which is not at all surprising, as some of the models selected are the same as before. summary(hipcenter_mod)$adj.r.squared ## [1] 0.6000855 summary(hipcenter_mod_both_aic)$adj.r.squared ## [1] 0.6533055 summary(hipcenter_mod_both_bic)$adj.r.squared ## [1] 0.6282374 calc_loocv_rmse(hipcenter_mod) ## [1] 44.44564 calc_loocv_rmse(hipcenter_mod_both_aic) ## [1] 37.62516 calc_loocv_rmse(hipcenter_mod_both_bic) ## [1] 37.2511 16.2.4 Exhaustive Search Backward, forward, and stepwise search are all useful, but do have an obvious issue. By not checking every possible model, sometimes they will miss the best possible model. With an extremely large number of predictors, sometimes this is necessary since checking every possible model would be rather time consuming, even with current computers. However, with a reasonably sized dataset, it isn’t too difficult to check all possible models. To do so, we will use the regsubsets() function in the R package leaps. library(leaps) all_hipcenter_mod = summary(regsubsets(hipcenter ~ ., data = seatpos)) A few points about this line of code. First, note that we immediately use summary() and store those results. That is simply the intended use of regsubsets(). Second, inside of regsubsets() we specify the model hipcenter ~ .. This will be the largest model considered, that is the model using all first-order predictors, and R will check all possible subsets. Third, regsubsets() has an argument of nvmax to set the maximum size of subsets to examine with \\(8\\) as default. You need to change it if you investigate the subsets with more than \\(8\\) predictors, although increasing nvmax will significantly increase computation time. We’ll now look at the information stored in all_hipcenter_mod. all_hipcenter_mod$which ## (Intercept) Age Weight HtShoes Ht Seated Arm Thigh Leg ## 1 TRUE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE ## 2 TRUE FALSE FALSE FALSE TRUE FALSE FALSE FALSE TRUE ## 3 TRUE TRUE FALSE FALSE TRUE FALSE FALSE FALSE TRUE ## 4 TRUE TRUE FALSE TRUE FALSE FALSE FALSE TRUE TRUE ## 5 TRUE TRUE FALSE TRUE FALSE FALSE TRUE TRUE TRUE ## 6 TRUE TRUE FALSE TRUE FALSE TRUE TRUE TRUE TRUE ## 7 TRUE TRUE TRUE TRUE FALSE TRUE TRUE TRUE TRUE ## 8 TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE Using $which gives us the best model, according to \\(\\text{RSS}\\), for a model of each possible size, in this case ranging from one to eight predictors. For example the best model with four predictors (\\(p = 5\\)) would use Age, HtShoes, Thigh, and Leg. all_hipcenter_mod$rss ## [1] 47615.79 44834.69 41938.09 41485.01 41313.00 41277.90 41266.80 41261.78 We can obtain the \\(\\text{RSS}\\) for each of these models using $rss. Notice that these are decreasing since the models range from small to large. Now that we have the \\(\\text{RSS}\\) for each of these models, it is rather easy to obtain \\(\\text{AIC}\\), \\(\\text{BIC}\\), and Adjusted \\(R^2\\) since they are all a function of \\(\\text{RSS}\\) Also, since we have the models with the best \\(\\text{RSS}\\) for each size, they will result in the models with the best \\(\\text{AIC}\\), \\(\\text{BIC}\\), and Adjusted \\(R^2\\) for each size. Then by picking from those, we can find the overall best \\(\\text{AIC}\\), \\(\\text{BIC}\\), and Adjusted \\(R^2\\). Conveniently, Adjusted \\(R^2\\) is automatically calculated. all_hipcenter_mod$adjr2 ## [1] 0.6282374 0.6399496 0.6533055 0.6466586 0.6371276 0.6257403 0.6133690 ## [8] 0.6000855 To find which model has the highest Adjusted \\(R^2\\) we can use the which.max() function. (best_r2_ind = which.max(all_hipcenter_mod$adjr2)) ## [1] 3 We can then extract the predictors of that model. all_hipcenter_mod$which[best_r2_ind, ] ## (Intercept) Age Weight HtShoes Ht Seated ## TRUE TRUE FALSE FALSE TRUE FALSE ## Arm Thigh Leg ## FALSE FALSE TRUE We’ll now calculate \\(\\text{AIC}\\) and \\(\\text{BIC}\\) for each of the models with the best \\(\\text{RSS}\\). To do so, we will need both \\(n\\) and the \\(p\\) for the largest possible model. p = length(coef(hipcenter_mod)) n = length(resid(hipcenter_mod)) We’ll use the form of \\(\\text{AIC}\\) which leaves out the constant term that is equal across all models. \\[ \\text{AIC} = n\\log\\left(\\frac{\\text{RSS}}{n}\\right) + 2p. \\] Since we have the \\(\\text{RSS}\\) of each model stored, this is easy to calculate. hipcenter_mod_aic = n * log(all_hipcenter_mod$rss / n) + 2 * (2:p) We can then extract the predictors of the model with the best \\(\\text{AIC}\\). best_aic_ind = which.min(hipcenter_mod_aic) all_hipcenter_mod$which[best_aic_ind,] ## (Intercept) Age Weight HtShoes Ht Seated ## TRUE TRUE FALSE FALSE TRUE FALSE ## Arm Thigh Leg ## FALSE FALSE TRUE Let’s fit this model so we can compare to our previously chosen models using \\(\\text{AIC}\\) and search procedures. hipcenter_mod_best_aic = lm(hipcenter ~ Age + Ht + Leg, data = seatpos) The extractAIC() function will calculate the \\(\\text{AIC}\\) defined above for a fitted model. extractAIC(hipcenter_mod_best_aic) ## [1] 4.0000 274.2418 extractAIC(hipcenter_mod_back_aic) ## [1] 4.0000 274.2597 extractAIC(hipcenter_mod_forw_aic) ## [1] 4.0000 274.2418 extractAIC(hipcenter_mod_both_aic) ## [1] 4.0000 274.2418 We see that two of the models chosen by search procedures have the best possible \\(\\text{AIC}\\), as they are the same model. This is however never guaranteed. We see that the model chosen using backwards selection does not achieve the smallest possible \\(\\text{AIC}\\). plot(hipcenter_mod_aic ~ I(2:p), ylab = &quot;AIC&quot;, xlab = &quot;p, number of parameters&quot;, pch = 20, col = &quot;dodgerblue&quot;, type = &quot;b&quot;, cex = 2, main = &quot;AIC vs Model Complexity&quot;) We could easily repeat this process for \\(\\text{BIC}\\). \\[ \\text{BIC} = n\\log\\left(\\frac{\\text{RSS}}{n}\\right) + \\log(n)p. \\] hipcenter_mod_bic = n * log(all_hipcenter_mod$rss / n) + log(n) * (2:p) which.min(hipcenter_mod_bic) ## [1] 1 all_hipcenter_mod$which[1,] ## (Intercept) Age Weight HtShoes Ht Seated ## TRUE FALSE FALSE FALSE TRUE FALSE ## Arm Thigh Leg ## FALSE FALSE FALSE hipcenter_mod_best_bic = lm(hipcenter ~ Ht, data = seatpos) extractAIC(hipcenter_mod_best_bic, k = log(n)) ## [1] 2.0000 278.3418 extractAIC(hipcenter_mod_back_bic, k = log(n)) ## [1] 2.0000 278.7306 extractAIC(hipcenter_mod_forw_bic, k = log(n)) ## [1] 2.0000 278.3418 extractAIC(hipcenter_mod_both_bic, k = log(n)) ## [1] 2.0000 278.3418 16.3 Higher Order Terms So far we have only allowed first-order terms in our models. Let’s return to the autompg dataset to explore higher-order terms. autompg = read.table( &quot;http://archive.ics.uci.edu/ml/machine-learning-databases/auto-mpg/auto-mpg.data&quot;, quote = &quot;\\&quot;&quot;, comment.char = &quot;&quot;, stringsAsFactors = FALSE) colnames(autompg) = c(&quot;mpg&quot;, &quot;cyl&quot;, &quot;disp&quot;, &quot;hp&quot;, &quot;wt&quot;, &quot;acc&quot;, &quot;year&quot;, &quot;origin&quot;, &quot;name&quot;) autompg = subset(autompg, autompg$hp != &quot;?&quot;) autompg = subset(autompg, autompg$name != &quot;plymouth reliant&quot;) rownames(autompg) = paste(autompg$cyl, &quot;cylinder&quot;, autompg$year, autompg$name) autompg$hp = as.numeric(autompg$hp) autompg$domestic = as.numeric(autompg$origin == 1) autompg = autompg[autompg$cyl != 5,] autompg = autompg[autompg$cyl != 3,] autompg$cyl = as.factor(autompg$cyl) autompg$domestic = as.factor(autompg$domestic) autompg = subset(autompg, select = c(&quot;mpg&quot;, &quot;cyl&quot;, &quot;disp&quot;, &quot;hp&quot;, &quot;wt&quot;, &quot;acc&quot;, &quot;year&quot;, &quot;domestic&quot;)) str(autompg) ## &#39;data.frame&#39;: 383 obs. of 8 variables: ## $ mpg : num 18 15 18 16 17 15 14 14 14 15 ... ## $ cyl : Factor w/ 3 levels &quot;4&quot;,&quot;6&quot;,&quot;8&quot;: 3 3 3 3 3 3 3 3 3 3 ... ## $ disp : num 307 350 318 304 302 429 454 440 455 390 ... ## $ hp : num 130 165 150 150 140 198 220 215 225 190 ... ## $ wt : num 3504 3693 3436 3433 3449 ... ## $ acc : num 12 11.5 11 12 10.5 10 9 8.5 10 8.5 ... ## $ year : int 70 70 70 70 70 70 70 70 70 70 ... ## $ domestic: Factor w/ 2 levels &quot;0&quot;,&quot;1&quot;: 2 2 2 2 2 2 2 2 2 2 ... Recall that we have two factor variables, cyl and domestic. The cyl variable has three levels, while the domestic variable has only two. Thus the cyl variable will be coded using two dummy variables, while the domestic variable will only need one. We will pay attention to this later. pairs(autompg, col = &quot;dodgerblue&quot;) We’ll use the pairs() plot to determine which variables may benefit from a quadratic relationship with the response. We’ll also consider all possible two-way interactions. We won’t consider any three-order or higher. For example, we won’t consider the interaction between first-order terms and the added quadratic terms. So now, we’ll fit this rather large model. We’ll use a log-transformed response. Notice that log(mpg) ~ . ^ 2 will automatically consider all first-order terms, as well as all two-way interactions. We use I(var_name ^ 2) to add quadratic terms for some variables. This generally works better than using poly() when performing variable selection. autompg_big_mod = lm( log(mpg) ~ . ^ 2 + I(disp ^ 2) + I(hp ^ 2) + I(wt ^ 2) + I(acc ^ 2), data = autompg) We think it is rather unlikely that we truly need all of these terms. There are quite a few! length(coef(autompg_big_mod)) ## [1] 40 We’ll try backwards search with both \\(\\text{AIC}\\) and \\(\\text{BIC}\\) to attempt to find a smaller, more reasonable model. autompg_mod_back_aic = step(autompg_big_mod, direction = &quot;backward&quot;, trace = 0) Notice that we used trace = 0 in the function call. This suppresses the output for each step, and simply stores the chosen model. This is useful, as this code would otherwise create a large amount of output. If we had viewed the output, which you can try on your own by removing trace = 0, we would see that R only considers the cyl variable as a single variable, despite the fact that it is coded using two dummy variables. So removing cyl would actually remove two parameters from the resulting model. You should also notice that R respects hierarchy when attempting to remove variables. That is, for example, R will not consider removing hp if hp:disp or I(hp ^ 2) are currently in the model. We also use \\(\\text{BIC}\\). n = length(resid(autompg_big_mod)) autompg_mod_back_bic = step(autompg_big_mod, direction = &quot;backward&quot;, k = log(n), trace = 0) Looking at the coefficients of the two chosen models, we see they are still rather large. coef(autompg_mod_back_aic) ## (Intercept) cyl6 cyl8 disp hp ## 3.671884e+00 -1.602563e-01 -8.581644e-01 -9.371971e-03 2.293534e-02 ## wt acc year domestic1 I(hp^2) ## -3.064497e-04 -1.393888e-01 -1.966361e-03 9.369324e-01 -1.497669e-05 ## cyl6:acc cyl8:acc disp:wt disp:year hp:acc ## 7.220298e-03 5.041915e-02 5.797816e-07 9.493770e-05 -5.062295e-04 ## hp:year acc:year acc:domestic1 year:domestic1 ## -1.838985e-04 2.345625e-03 -2.372468e-02 -7.332725e-03 coef(autompg_mod_back_bic) ## (Intercept) cyl6 cyl8 disp hp ## 4.657847e+00 -1.086165e-01 -7.611631e-01 -1.609316e-03 2.621266e-03 ## wt acc year domestic1 cyl6:acc ## -2.635972e-04 -1.670601e-01 -1.045646e-02 3.341579e-01 4.315493e-03 ## cyl8:acc disp:wt hp:acc acc:year acc:domestic1 ## 4.610095e-02 4.102804e-07 -3.386261e-04 2.500137e-03 -2.193294e-02 However, they are much smaller than the original full model. Also notice that the resulting models respect hierarchy. length(coef(autompg_big_mod)) ## [1] 40 length(coef(autompg_mod_back_aic)) ## [1] 19 length(coef(autompg_mod_back_bic)) ## [1] 15 Calculating the LOOCV \\(\\text{RMSE}\\) for each, we see that the model chosen using \\(\\text{BIC}\\) performs the best. That means that it is both the best model for prediction, since it achieves the best LOOCV \\(\\text{RMSE}\\), but also the best model for explanation, as it is also the smallest. calc_loocv_rmse(autompg_big_mod) ## [1] 0.1112024 calc_loocv_rmse(autompg_mod_back_aic) ## [1] 0.1032888 calc_loocv_rmse(autompg_mod_back_bic) ## [1] 0.103134 16.4 Explanation versus Prediction Throughout this chapter, we have attempted to find reasonably “small” models, which are good at explaining the relationship between the response and the predictors, that also have small errors which are thus good for making predictions. We’ll further discuss the model autompg_mod_back_bic to better explain the difference between using models for explaining and predicting. This is the model fit to the autompg data that was chosen using Backwards Search and \\(\\text{BIC}\\), which obtained the lowest LOOCV \\(\\text{RMSE}\\) of the models we considered. autompg_mod_back_bic ## ## Call: ## lm(formula = log(mpg) ~ cyl + disp + hp + wt + acc + year + domestic + ## cyl:acc + disp:wt + hp:acc + acc:year + acc:domestic, data = autompg) ## ## Coefficients: ## (Intercept) cyl6 cyl8 disp hp ## 4.658e+00 -1.086e-01 -7.612e-01 -1.609e-03 2.621e-03 ## wt acc year domestic1 cyl6:acc ## -2.636e-04 -1.671e-01 -1.046e-02 3.342e-01 4.315e-03 ## cyl8:acc disp:wt hp:acc acc:year acc:domestic1 ## 4.610e-02 4.103e-07 -3.386e-04 2.500e-03 -2.193e-02 Notice this is a somewhat “large” model, which uses 15 parameters, including several interaction terms. Do we care that this is a “large” model? The answer is, it depends. 16.4.1 Explanation Suppose we would like to use this model for explanation. Perhaps we are a car manufacturer trying to engineer a fuel efficient vehicle. If this is the case, we are interested in both what predictor variables are useful for explaining the car’s fuel efficiency, as well as how those variables affect fuel efficiency. By understanding this relationship, we can use this knowledge to our advantage when designing a car. To explain a relationship, we are interested in keeping models as small as possible, since smaller models are easy to interpret. The fewer predictors the fewer considerations we need to make in our design process. Also the fewer interactions and polynomial terms, the easier it is to interpret any one parameter, since the parameter interpretations are conditional on which parameters are in the model. Note that linear models are rather interpretable to begin with. Later in your data analysis careers, you will see more complicated models that may fit data better, but are much harder, if not impossible to interpret. These models aren’t very useful for explaining a relationship. To find small and interpretable models, we would use selection criteria that explicitly penalize larger models, such as AIC and BIC. In this case we still obtained a somewhat large model, but much smaller than the model we used to start the selection process. 16.4.1.1 Correlation and Causation A word of caution when using a model to explain a relationship. There are two terms often used to describe a relationship between two variables: causation and correlation. Correlation is often also referred to as association. Just because two variables are correlated does not necessarily mean that one causes the other. For example, considering modeling mpg as only a function of hp. plot(mpg ~ hp, data = autompg, col = &quot;dodgerblue&quot;, pch = 20, cex = 1.5) Does an increase in horsepower cause a drop in fuel efficiency? Or, perhaps the causality is reversed and an increase in fuel efficiency causes a decrease in horsepower. Or, perhaps there is a third variable that explains both! The issue here is that we have observational data. With observational data, we can only detect associations. To speak with confidence about causality, we would need to run experiments. This is a concept that you should encounter often in your statistics education. For some further reading, and some related fallacies, see: Wikipedia: Correlation does not imply causation. 16.4.2 Prediction Suppose now instead of the manufacturer who would like to build a car, we are a consumer who wishes to purchase a new car. However this particular car is so new, it has not been rigorously tested, so we are unsure of what fuel efficiency to expect. (And, as skeptics, we don’t trust what the manufacturer is telling us.) In this case, we would like to use the model to help predict the fuel efficiency of this car based on its attributes, which are the predictors of the model. The smaller the errors the model makes, the more confident we are in its prediction. Thus, to find models for prediction, we would use selection criteria that implicitly penalize larger models, such as LOOCV \\(\\text{RMSE}\\). So long as the model does not over-fit, we do not actually care how large the model becomes. Explaining the relationship between the variables is not our goal here, we simply want to know what kind of fuel efficiency we should expect! If we only care about prediction, we don’t need to worry about correlation vs causation, and we don’t need to worry about model assumptions. If a variable is correlated with the response, it doesn’t actually matter if it causes an effect on the response, it can still be useful for prediction. For example, in elementary school aged children their shoe size certainly doesn’t cause them to read at a higher level, however we could very easily use shoe size to make a prediction about a child’s reading ability. The larger their shoe size, the better they read. There’s a lurking variable here though, their age! (Don’t send your kids to school with size 14 shoes, it won’t make them read better!) We also don’t care about model assumptions. Least squares is least squares. For a specified model, it will find the values of the parameters which will minimize the squared error loss. Your results might be largely uninterpretable and useless for inference, but for prediction none of that matters. 16.5 R Markdown The R Markdown file for this chapter can be found here: selection.Rmd The file was created using R version 4.3.2. "],["logistic-regression.html", "Chapter 17 Logistic Regression 17.1 Generalized Linear Models 17.2 Binary Response 17.3 Working with Logistic Regression 17.4 Classification 17.5 R Markdown", " Chapter 17 Logistic Regression Note to current readers: This chapter is slightly less tested than previous chapters. Please do not hesitate to report any errors, or suggest sections that need better explanation! Also, as a result, this material is more likely to receive edits. After reading this chapter you will be able to: Understand how generalized linear models are a generalization of ordinary linear models. Use logistic regression to model a binary response. Apply concepts learned for ordinary linear models to logistic regression. Use logistic regression to perform classification. So far we have only considered models for numeric response variables. What about response variables that only take integer values? What about a response variable that is categorical? Can we use linear models in these situations? Yes! The model that we have been using, which we will call ordinary linear regression, is actually a specific case of the more general, generalized linear model. (Aren’t statisticians great at naming things?) 17.1 Generalized Linear Models So far, we’ve had response variables that, conditioned on the predictors, were modeled using a normal distribution with a mean that is some linear combination of the predictors. This linear combination is what made a linear model “linear.” \\[ Y \\mid {\\bf X} = {\\bf x} \\sim N(\\beta_0 + \\beta_1x_1 + \\ldots + \\beta_{p - 1}x_{p - 1}, \\ \\sigma^2) \\] Now we’ll allow for two modifications of this situation, which will let us use linear models in many more situations. Instead of using a normal distribution for the response conditioned on the predictors, we’ll allow for other distributions. Also, instead of the conditional mean being a linear combination of the predictors, it can be some function of a linear combination of the predictors. In general, a generalized linear model has three parts: A distribution of the response conditioned on the predictors. (Technically this distribution needs to be from the exponential family of distributions.) A linear combination of the \\(p - 1\\) predictors, \\(\\beta_0 + \\beta_1 x_1 + \\beta_2 x_2 + \\ldots + \\beta_{p - 1} x_{p - 1}\\), which we write as \\(\\eta({\\bf x})\\). That is, \\[\\eta({\\bf x}) = \\beta_0 + \\beta_1 x_1 + \\beta_2 x_2 + \\ldots + \\beta_{p - 1} x_{p - 1}\\] A link function, \\(g()\\), that defines how \\(\\eta({\\bf x})\\), the linear combination of the predictors, is related to the mean of the response conditioned on the predictors, \\(\\text{E}[Y \\mid {\\bf X} = {\\bf x}]\\). \\[ \\eta({\\bf x}) = g\\left(\\text{E}[Y \\mid {\\bf X} = {\\bf x}]\\right). \\] The following table summarizes three examples of a generalized linear model: Linear Regression Poisson Regression Logistic Regression \\(Y \\mid {\\bf X} = {\\bf x}\\) \\(N(\\mu({\\bf x}), \\sigma^2)\\) \\(\\text{Pois}(\\lambda({\\bf x}))\\) \\(\\text{Bern}(p({\\bf x}))\\) Distribution Name Normal Poisson Bernoulli (Binomial) \\(\\text{E}[Y \\mid {\\bf X} = {\\bf x}]\\) \\(\\mu({\\bf x})\\) \\(\\lambda({\\bf x})\\) \\(p({\\bf x})\\) Support Real: \\((-\\infty, \\infty)\\) Integer: \\(0, 1, 2, \\ldots\\) Integer: \\(0, 1\\) Usage Numeric Data Count (Integer) Data Binary (Class ) Data Link Name Identity Log Logit Link Function \\(\\eta({\\bf x}) = \\mu({\\bf x})\\) \\(\\eta({\\bf x}) = \\log(\\lambda({\\bf x}))\\) \\(\\eta({\\bf x}) = \\log \\left(\\frac{p({\\bf x})}{1 - p({\\bf x})} \\right)\\) Mean Function \\(\\mu({\\bf x}) = \\eta({\\bf x})\\) \\(\\lambda({\\bf x}) = e^{\\eta({\\bf x})}\\) \\(p({\\bf x}) = \\frac{e^{\\eta({\\bf x})}}{1 + e^{\\eta({\\bf x})}} = \\frac{1}{1 + e^{-\\eta({\\bf x})}}\\) Like ordinary linear regression, we will seek to “fit” the model by estimating the \\(\\beta\\) parameters. To do so, we will use the method of maximum likelihood. Note that a Bernoulli distribution is a specific case of a binomial distribution where the \\(n\\) parameter of a binomial is \\(1\\). Binomial regression is also possible, but we’ll focus on the much more popular Bernoulli case. So, in general, GLMs relate the mean of the response to a linear combination of the predictors, \\(\\eta({\\bf x})\\), through the use of a link function, \\(g()\\). That is, \\[ \\eta({\\bf x}) = g\\left(\\text{E}[Y \\mid {\\bf X} = {\\bf x}]\\right). \\] The mean is then \\[ \\text{E}[Y \\mid {\\bf X} = {\\bf x}] = g^{-1}(\\eta({\\bf x})). \\] 17.2 Binary Response To illustrate the use of a GLM we’ll focus on the case of binary responses variable coded using \\(0\\) and \\(1\\). In practice, these \\(0\\) and \\(1\\)s will code for two classes such as yes/no, cat/dog, sick/healthy, etc. \\[ Y = \\begin{cases} 1 &amp; \\text{yes} \\\\ 0 &amp; \\text{no} \\end{cases} \\] First, we define some notation that we will use throughout. \\[ p({\\bf x}) = P[Y = 1 \\mid {\\bf X} = {\\bf x}] \\] With a binary (Bernoulli) response, we’ll mostly focus on the case when \\(Y = 1\\), since with only two possibilities, it is trivial to obtain probabilities when \\(Y = 0\\). \\[ P[Y = 0 \\mid {\\bf X} = {\\bf x}] + P[Y = 1 \\mid {\\bf X} = {\\bf x}] = 1 \\] \\[ P[Y = 0 \\mid {\\bf X} = {\\bf x}] = 1 - p({\\bf x}) \\] We now define the logistic regression model. \\[ \\log\\left(\\frac{p({\\bf x})}{1 - p({\\bf x})}\\right) = \\beta_0 + \\beta_1 x_1 + \\ldots + \\beta_{p - 1} x_{p - 1} \\] Immediately we notice some similarities to ordinary linear regression, in particular, the right-hand side. This is our usual linear combination of the predictors. We have our usual \\(p - 1\\) predictors for a total of \\(p\\) \\(\\beta\\) parameters. (Note, many more machine learning focused texts will use \\(p\\) as the number of predictors. This is an arbitrary choice, but you should be aware of it.) The left-hand side is called the log odds, which is the log of the odds. The odds are the probability for a positive event \\((Y = 1)\\) divided by the probability of a negative event \\((Y = 0)\\). So when the odds are \\(1\\), the two events have equal probability. Odds greater than \\(1\\) favor a positive event. The opposite is true when the odds are less than \\(1\\). \\[ \\frac{p({\\bf x})}{1 - p({\\bf x})} = \\frac{P[Y = 1 \\mid {\\bf X} = {\\bf x}]}{P[Y = 0 \\mid {\\bf X} = {\\bf x}]} \\] Essentially, the log odds are the logit transform applied to \\(p({\\bf x})\\). \\[ \\text{logit}(\\xi) = \\log\\left(\\frac{\\xi}{1 - \\xi}\\right) \\] It will also be useful to define the inverse logit, otherwise known as the “logistic” or sigmoid function. \\[ \\text{logit}^{-1}(\\xi) = \\frac{e^\\xi}{1 + e^{\\xi}} = \\frac{1}{1 + e^{-\\xi}} \\] Note that for \\(x \\in (-\\infty, \\infty))\\), this function outputs values between 0 and 1. Students often ask, where is the error term? The answer is that it’s something that is specific to the normal model. First notice that the model with the error term, \\[ Y = \\beta_0 + \\beta_1x_1 + \\ldots + \\beta_qx_q + \\epsilon, \\ \\ \\epsilon \\sim N(0, \\sigma^2) \\] can instead be written as \\[ Y \\mid {\\bf X} = {\\bf x} \\sim N(\\beta_0 + \\beta_1x_1 + \\ldots + \\beta_qx_q, \\ \\sigma^2). \\] While our main focus is on estimating the mean, \\(\\beta_0 + \\beta_1x_1 + \\ldots + \\beta_qx_q\\), there is also another parameter, \\(\\sigma^2\\) which needs to be estimated. This is the result of the normal distribution having two parameters. With logistic regression, which uses the Bernoulli distribution, we only need to estimate the Bernoulli distribution’s single parameter \\(p({\\bf x})\\), which happens to be its mean. \\[ \\log\\left(\\frac{p({\\bf x})}{1 - p({\\bf x})}\\right) = \\beta_0 + \\beta_1 x_1 + \\ldots + \\beta_{q} x_{q} \\] So even though we introduced ordinary linear regression first, in some ways, logistic regression is actually simpler. Note that applying the inverse logit transformation allow us to obtain an expression for \\(p({\\bf x})\\). \\[ p({\\bf x}) = P[Y = 1 \\mid {\\bf X} = {\\bf x}] = \\frac{e^{\\beta_0 + \\beta_1 x_{1} + \\cdots + \\beta_{p-1} x_{(p-1)}}}{1 + e^{\\beta_0 + \\beta_1 x_{1} + \\cdots + \\beta_{p-1} x_{(p-1)}}} \\] 17.2.1 Fitting Logistic Regression With \\(n\\) observations, we write the model indexed with \\(i\\) to note that it is being applied to each observation. \\[ \\log\\left(\\frac{p({\\bf x_i})}{1 - p({\\bf x_i})}\\right) = \\beta_0 + \\beta_1 x_{i1} + \\cdots + \\beta_{p-1} x_{i(p-1)} \\] We can apply the inverse logit transformation to obtain \\(P[Y_i = 1 \\mid {\\bf X_i} = {\\bf x_i}]\\) for each observation. Since these are probabilities, it’s good that we used a function that returns values between \\(0\\) and \\(1\\). \\[ p({\\bf x_i}) = P[Y_i = 1 \\mid {\\bf X_i} = {\\bf x_i}] = \\frac{e^{\\beta_0 + \\beta_1 x_{i1} + \\cdots + \\beta_{p-1} x_{i(p-1)}}}{1 + e^{\\beta_0 + \\beta_1 x_{i1} + \\cdots + \\beta_{p-1} x_{i(p-1)}}} \\] \\[ 1 - p({\\bf x_i}) = P[Y_i = 0 \\mid {\\bf X} = {\\bf x_i}] = \\frac{1}{1 + e^{\\beta_0 + \\beta_1 x_{i1} + \\cdots + \\beta_{p-1} x_{i(p-1)}}} \\] To “fit” this model, that is estimate the \\(\\beta\\) parameters, we will use maximum likelihood. \\[ \\boldsymbol{{\\beta}} = [\\beta_0, \\beta_1, \\beta_2, \\beta_3, \\ldots, \\beta_{p - 1}] \\] We first write the likelihood given the observed data. \\[ L(\\boldsymbol{{\\beta}}) = \\prod_{i = 1}^{n} P[Y_i = y_i \\mid {\\bf X_i} = {\\bf x_i}] \\] This is already technically a function of the \\(\\beta\\) parameters, but we’ll do some rearrangement to make this more explicit. \\[ L(\\boldsymbol{{\\beta}}) = \\prod_{i = 1}^{n} p({\\bf x_i})^{y_i} (1 - p({\\bf x_i}))^{(1 - y_i)} \\] \\[ L(\\boldsymbol{{\\beta}}) = \\prod_{i : y_i = 1}^{n} p({\\bf x_i}) \\prod_{j : y_j = 0}^{n} (1 - p({\\bf x_j})) \\] \\[ L(\\boldsymbol{{\\beta}}) = \\prod_{i : y_i = 1}^{} \\frac{e^{\\beta_0 + \\beta_1 x_{i1} + \\cdots + \\beta_{p-1} x_{i(p-1)}}}{1 + e^{\\beta_0 + \\beta_1 x_{i1} + \\cdots + \\beta_{p-1} x_{i(p-1)}}} \\prod_{j : y_j = 0}^{} \\frac{1}{1 + e^{\\beta_0 + \\beta_1 x_{j1} + \\cdots + \\beta_{p-1} x_{j(p-1)}}} \\] Unfortunately, unlike ordinary linear regression, there is no analytical solution for this maximization problem. Instead, it will need to be solved numerically. Fortunately, R will take care of this for us using an iteratively reweighted least squares algorithm. (We’ll leave the details for a machine learning or optimization course, which would likely also discuss alternative optimization strategies.) 17.2.2 Fitting Issues We should note that, if there exists some \\(\\beta^*\\) such that \\[ {\\bf x_i}^{\\top} \\boldsymbol{{\\beta}^*} &gt; 0 \\implies y_i = 1 \\] and \\[ {\\bf x_i}^{\\top} \\boldsymbol{{\\beta}^*} &lt; 0 \\implies y_i = 0 \\] for all observations, then the MLE is not unique. Such data is said to be separable. This, and similar numeric issues related to estimated probabilities near 0 or 1, will return a warning in R: ## Warning: glm.fit: fitted probabilities numerically 0 or 1 occurred When this happens, the model is still “fit,” but there are consequences, namely, the estimated coefficients are highly suspect. This is an issue when then trying to interpret the model. When this happens, the model will often still be useful for creating a classifier, which will be discussed later. However, it is still subject to the usual evaluations for classifiers to determine how well it is performing. For details, see Modern Applied Statistics with S-PLUS, Chapter 7. 17.2.3 Simulation Examples sim_logistic_data = function(sample_size = 25, beta_0 = -2, beta_1 = 3) { x = rnorm(n = sample_size) eta = beta_0 + beta_1 * x p = 1 / (1 + exp(-eta)) y = rbinom(n = sample_size, size = 1, prob = p) data.frame(y, x) } You might think, why not simply use ordinary linear regression? Even with a binary response, our goal is still to model (some function of) \\(\\text{E}[Y \\mid {\\bf X} = {\\bf x}]\\). However, with a binary response coded as \\(0\\) and \\(1\\), \\(\\text{E}[Y \\mid {\\bf X} = {\\bf x}] = P[Y = 1 \\mid {\\bf X} = {\\bf x}]\\) since \\[ \\begin{aligned} \\text{E}[Y \\mid {\\bf X} = {\\bf x}] &amp;= 1 \\cdot P[Y = 1 \\mid {\\bf X} = {\\bf x}] + 0 \\cdot P[Y = 0 \\mid {\\bf X} = {\\bf x}] \\\\ &amp;= P[Y = 1 \\mid {\\bf X} = {\\bf x}] \\end{aligned} \\] Then why can’t we just use ordinary linear regression to estimate \\(\\text{E}[Y \\mid {\\bf X} = {\\bf x}]\\), and thus \\(P[Y = 1 \\mid {\\bf X} = {\\bf x}]\\)? To investigate, let’s simulate data from the following model: \\[ \\log\\left(\\frac{p({\\bf x})}{1 - p({\\bf x})}\\right) = -2 + 3 x \\] Another way to write this, which better matches the function we’re using to simulate the data: \\[ \\begin{aligned} Y_i \\mid {\\bf X_i} = {\\bf x_i} &amp;\\sim \\text{Bern}(p_i) \\\\ p_i &amp;= p({\\bf x_i}) = \\frac{1}{1 + e^{-\\eta({\\bf x_i})}} \\\\ \\eta({\\bf x_i}) &amp;= -2 + 3 x_i \\end{aligned} \\] set.seed(1) example_data = sim_logistic_data() head(example_data) ## y x ## 1 0 -0.6264538 ## 2 1 0.1836433 ## 3 0 -0.8356286 ## 4 1 1.5952808 ## 5 0 0.3295078 ## 6 0 -0.8204684 After simulating a dataset, we’ll then fit both ordinary linear regression and logistic regression. Notice that currently the responses variable y is a numeric variable that only takes values 0 and 1. Later we’ll see that we can also fit logistic regression when the response is a factor variable with only two levels. (Generally, having a factor response is preferred, but having a dummy response allows us to make the comparison to using ordinary linear regression.) # ordinary linear regression fit_lm = lm(y ~ x, data = example_data) # logistic regression fit_glm = glm(y ~ x, data = example_data, family = binomial) Notice that the syntax is extremely similar. What’s changed? lm() has become glm() We’ve added family = binomial argument In a lot of ways, lm() is just a more specific version of glm(). For example glm(y ~ x, data = example_data) would actually fit the ordinary linear regression that we have seen in the past. By default, glm() uses family = gaussian argument. That is, we’re fitting a GLM with a normally distributed response and the identity function as the link. The family argument to glm() actually specifies both the distribution and the link function. If not made explicit, the link function is chosen to be the canonical link function, which is essentially the most mathematical convenient link function. See ?glm and ?family for details. For example, the following code explicitly specifies the link function which was previously used by default. # more detailed call to glm for logistic regression fit_glm = glm(y ~ x, data = example_data, family = binomial(link = &quot;logit&quot;)) Making predictions with an object of type glm is slightly different than making predictions after fitting with lm(). In the case of logistic regression, with family = binomial, we have: type Returned \"link\" [default] \\(\\hat{\\eta}({\\bf x}) = \\log\\left(\\frac{\\hat{p}({\\bf x})}{1 - \\hat{p}({\\bf x})}\\right)\\) \"response\" \\(\\hat{p}({\\bf x}) = \\frac{e^{\\hat{\\eta}({\\bf x})}}{1 + e^{\\hat{\\eta}({\\bf x})}} = \\frac{1}{1 + e^{-\\hat{\\eta}({\\bf x})}}\\) That is, type = \"link\" will get you the log odds, while type = \"response\" will return the estimated mean, in this case, \\(P[Y = 1 \\mid {\\bf X} = {\\bf x}]\\) for each observation. predict(fit_glm, data.frame(x = 1.2)) # type = &quot;link&quot; as default ## 1 ## 2.076166 predict(fit_glm, data.frame(x = 1.2), type = &quot;link&quot;) ## 1 ## 2.076166 predict(fit_glm, data.frame(x = 1.2), type = &quot;response&quot;) ## 1 ## 0.888565 plot(y ~ x, data = example_data, pch = 20, ylab = &quot;Estimated Probability&quot;, main = &quot;Ordinary vs Logistic Regression&quot;) grid() abline(fit_lm, col = &quot;darkorange&quot;) curve(predict(fit_glm, data.frame(x), type = &quot;response&quot;), add = TRUE, col = &quot;dodgerblue&quot;, lty = 2) legend(&quot;topleft&quot;, c(&quot;Ordinary&quot;, &quot;Logistic&quot;, &quot;Data&quot;), lty = c(1, 2, 0), pch = c(NA, NA, 20), lwd = 2, col = c(&quot;darkorange&quot;, &quot;dodgerblue&quot;, &quot;black&quot;)) Since we only have a single predictor variable, we are able to graphically show this situation. First, note that the data, is plotted using black dots. The response y only takes values 0 and 1. Next, we need to discuss the two added lines to the plot. The first, the solid orange line, is the fitted ordinary linear regression. The dashed blue curve is the estimated logistic regression. It is helpful to realize that we are not plotting an estimate of \\(Y\\) for either. (Sometimes it might seem that way with ordinary linear regression, but that isn’t what is happening.) For both, we are plotting \\(\\hat{\\text{E}}[Y \\mid {\\bf X} = {\\bf x}]\\), the estimated mean, which for a binary response happens to be an estimate of \\(P[Y = 1 \\mid {\\bf X} = {\\bf x}]\\). We immediately see why ordinary linear regression is not a good idea. While it is estimating the mean, we see that it produces estimates that are less than 0! (And in other situations could produce estimates greater than 1!) If the mean is a probability, we don’t want probabilities less than 0 or greater than 1. Enter logistic regression. Since the output of the inverse logit function is restricted to be between 0 and 1, our estimates make much more sense as probabilities. Let’s look at our estimated coefficients. (With a lot of rounding, for simplicity.) round(coef(fit_glm), 1) ## (Intercept) x ## -2.3 3.7 Our estimated model is then: \\[ \\log\\left(\\frac{\\hat{p}({\\bf x})}{1 - \\hat{p}({\\bf x})}\\right) = -2.3 + 3.7 x \\] Because we’re not directly estimating the mean, but instead a function of the mean, we need to be careful with our interpretation of \\(\\hat{\\beta}_1 = 3.7\\). This means that, for a one unit increase in \\(x\\), the log odds change (in this case increase) by \\(3.7\\). Also, since \\(\\hat{\\beta}_1\\) is positive, as we increase \\(x\\) we also increase \\(\\hat{p}({\\bf x})\\). To see how much, we have to consider the inverse logistic function. For example, we have: \\[ \\hat{P}[Y = 1 \\mid X = -0.5] = \\frac{e^{-2.3 + 3.7 \\cdot (-0.5)}}{1 + e^{-2.3 + 3.7 \\cdot (-0.5)}} \\approx 0.016 \\] \\[ \\hat{P}[Y = 1 \\mid X = 0] = \\frac{e^{-2.3 + 3.7 \\cdot (0)}}{1 + e^{-2.3 + 3.7 \\cdot (0)}} \\approx 0.09112296 \\] \\[ \\hat{P}[Y = 1 \\mid X = 1] = \\frac{e^{-2.3 + 3.7 \\cdot (1)}}{1 + e^{-2.3 + 3.7 \\cdot (1)}} \\approx 0.8021839 \\] Now that we know we should use logistic regression, and not ordinary linear regression, let’s consider another example. This time, let’s consider the model \\[ \\log\\left(\\frac{p({\\bf x})}{1 - p({\\bf x})}\\right) = 1 + -4 x. \\] Again, we could re-write this to better match the function we’re using to simulate the data: \\[ \\begin{aligned} Y_i \\mid {\\bf X_i} = {\\bf x_i} &amp;\\sim \\text{Bern}(p_i) \\\\ p_i &amp;= p({\\bf x_i}) = \\frac{1}{1 + e^{-\\eta({\\bf x_i})}} \\\\ \\eta({\\bf x_i}) &amp;= 1 + -4 x_i \\end{aligned} \\] In this model, as \\(x\\) increases, the log odds decrease. set.seed(1) example_data = sim_logistic_data(sample_size = 50, beta_0 = 1, beta_1 = -4) We again simulate some observations form this model, then fit logistic regression. fit_glm = glm(y ~ x, data = example_data, family = binomial) plot(y ~ x, data = example_data, pch = 20, ylab = &quot;Estimated Probability&quot;, main = &quot;Logistic Regression, Decreasing Probability&quot;) grid() curve(predict(fit_glm, data.frame(x), type = &quot;response&quot;), add = TRUE, col = &quot;dodgerblue&quot;, lty = 2) curve(boot::inv.logit(1 - 4 * x), add = TRUE, col = &quot;darkorange&quot;, lty = 1) legend(&quot;bottomleft&quot;, c(&quot;True Probability&quot;, &quot;Estimated Probability&quot;, &quot;Data&quot;), lty = c(1, 2, 0), pch = c(NA, NA, 20), lwd = 2, col = c(&quot;darkorange&quot;, &quot;dodgerblue&quot;, &quot;black&quot;)) We see that this time, as \\(x\\) increases, \\(\\hat{p}({\\bf x})\\) decreases. Now let’s look at an example where the estimated probability doesn’t always simply increase or decrease. Much like ordinary linear regression, the linear combination of predictors can contain transformations of predictors (in this case a quadratic term) and interactions. sim_quadratic_logistic_data = function(sample_size = 25) { x = rnorm(n = sample_size) eta = -1.5 + 0.5 * x + x ^ 2 p = 1 / (1 + exp(-eta)) y = rbinom(n = sample_size, size = 1, prob = p) data.frame(y, x) } \\[ \\log\\left(\\frac{p({\\bf x})}{1 - p({\\bf x})}\\right) = -1.5 + 0.5x + x^2. \\] Again, we could re-write this to better match the function we’re using to simulate the data: \\[ \\begin{aligned} Y_i \\mid {\\bf X_i} = {\\bf x_i} &amp;\\sim \\text{Bern}(p_i) \\\\ p_i &amp;= p({\\bf x_i}) = \\frac{1}{1 + e^{-\\eta({\\bf x_i})}} \\\\ \\eta({\\bf x_i}) &amp;= -1.5 + 0.5x_i + x_i^2 \\end{aligned} \\] set.seed(42) example_data = sim_quadratic_logistic_data(sample_size = 50) fit_glm = glm(y ~ x + I(x^2), data = example_data, family = binomial) plot(y ~ x, data = example_data, pch = 20, ylab = &quot;Estimated Probability&quot;, main = &quot;Logistic Regression, Quadratic Relationship&quot;) grid() curve(predict(fit_glm, data.frame(x), type = &quot;response&quot;), add = TRUE, col = &quot;dodgerblue&quot;, lty = 2) curve(boot::inv.logit(-1.5 + 0.5 * x + x ^ 2), add = TRUE, col = &quot;darkorange&quot;, lty = 1) legend(&quot;bottomleft&quot;, c(&quot;True Probability&quot;, &quot;Estimated Probability&quot;, &quot;Data&quot;), lty = c(1, 2, 0), pch = c(NA, NA, 20), lwd = 2, col = c(&quot;darkorange&quot;, &quot;dodgerblue&quot;, &quot;black&quot;)) 17.3 Working with Logistic Regression While the logistic regression model isn’t exactly the same as the ordinary linear regression model, because they both use a linear combination of the predictors \\[ \\eta({\\bf x}) = \\beta_0 + \\beta_1 x_1 + \\beta_2 x_2 + \\ldots + \\beta_{p - 1} x_{p - 1} \\] working with logistic regression is very similar. Many of the things we did with ordinary linear regression can be done with logistic regression in a very similar fashion. For example, Testing for a single \\(\\beta\\) parameter Testing for a set of \\(\\beta\\) parameters Formula specification in R Interpreting parameters and estimates Confidence intervals for parameters Confidence intervals for mean response Variable selection After some introduction to the new tests, we’ll demonstrate each of these using an example. 17.3.1 Testing with GLMs Like ordinary linear regression, we’ll want to be able to perform hypothesis testing. We’ll again want both single parameter, and multiple parameter tests. 17.3.2 Wald Test In ordinary linear regression, we performed the test of \\[ H_0: \\beta_j = 0 \\quad \\text{vs} \\quad H_1: \\beta_j \\neq 0 \\] using a \\(t\\)-test. For the logistic regression model, \\[ \\log\\left(\\frac{p({\\bf x})}{1 - p({\\bf x})}\\right) = \\beta_0 + \\beta_1 x_1 + \\ldots + \\beta_{p - 1} x_{p - 1} \\] we can again perform a test of \\[ H_0: \\beta_j = 0 \\quad \\text{vs} \\quad H_1: \\beta_j \\neq 0 \\] however, the test statistic and its distribution are no longer \\(t\\). We see that the test statistic takes the same form \\[ z = \\frac{\\hat{\\beta}_j - \\beta_j}{\\text{SE}[\\hat{\\beta}_j]} \\overset{\\text{approx}}{\\sim} N(0, 1) \\] but now we are performing a \\(z\\)-test, as the test statistic is approximated by a standard normal distribution, provided we have a large enough sample. (The \\(t\\)-test for ordinary linear regression, assuming the assumptions were correct, had an exact distribution for any sample size.) We’ll skip some of the exact details of the calculations, as R will obtain the standard error for us. The use of this test will be extremely similar to the \\(t\\)-test for ordinary linear regression. Essentially the only thing that changes is the distribution of the test statistic. 17.3.3 Likelihood-Ratio Test Consider the following full model, \\[ \\log\\left(\\frac{p({\\bf x_i})}{1 - p({\\bf x_i})}\\right) = \\beta_0 + \\beta_1 x_{i1} + \\beta_2 x_{i2} + \\cdots + \\beta_{(p-1)} x_{i(p-1)} \\] This model has \\(p - 1\\) predictors, for a total of \\(p\\) \\(\\beta\\)-parameters. We will denote the MLE of these \\(\\beta\\)-parameters as \\(\\hat{\\beta}_{\\text{Full}}\\) Now consider a null (or reduced) model, \\[ \\log\\left(\\frac{p({\\bf x_i})}{1 - p({\\bf x_i})}\\right) = \\beta_0 + \\beta_1 x_{i1} + \\beta_2 x_{i2} + \\cdots + \\beta_{(q-1)} x_{i(q-1)} \\] where \\(q &lt; p\\). This model has \\(q - 1\\) predictors, for a total of \\(q\\) \\(\\beta\\)-parameters. We will denote the MLE of these \\(\\beta\\)-parameters as \\(\\hat{\\beta}_{\\text{Null}}\\) The difference between these two models can be codified by the null hypothesis of a test. \\[ H_0: \\beta_q = \\beta_{q+1} = \\cdots = \\beta_{p - 1} = 0. \\] This implies that the reduced model is nested inside the full model. We then define a test statistic, \\(D\\), \\[ D = -2 \\log \\left( \\frac{L(\\boldsymbol{\\hat{\\beta}_{\\text{Null}}})} {L(\\boldsymbol{\\hat{\\beta}_{\\text{Full}}})} \\right) = 2 \\log \\left( \\frac{L(\\boldsymbol{\\hat{\\beta}_{\\text{Full}}})} {L(\\boldsymbol{\\hat{\\beta}_{\\text{Null}}})} \\right) = 2 \\left( \\ell(\\hat{\\beta}_{\\text{Full}}) - \\ell(\\hat{\\beta}_{\\text{Null}})\\right) \\] where \\(L\\) denotes a likelihood and \\(\\ell\\) denotes a log-likelihood. For a large enough sample, this test statistic has an approximate Chi-square distribution \\[ D \\overset{\\text{approx}}{\\sim} \\chi^2_{k} \\] where \\(k = p - q\\), the difference in number of parameters of the two models. This test, which we will call the Likelihood-Ratio Test, will be the analogue to the ANOVA \\(F\\)-test for logistic regression. Interestingly, to perform the Likelihood-Ratio Test, we’ll actually again use the anova() function in R!. The Likelihood-Ratio Test is actually a rather general test, however, here we have presented a specific application to nested logistic regression models. 17.3.4 SAheart Example To illustrate the use of logistic regression, we will use the SAheart dataset from the bestglm package. # &quot;leaps&quot; is required for &quot;bestglm&quot; if (!require(&quot;leaps&quot;)) install.packages(&quot;leaps&quot;) if (!require(&quot;bestglm&quot;)) install.packages(&quot;bestglm&quot;) library(leaps) library(bestglm) data(&quot;SAheart&quot;) sbp tobacco ldl adiposity famhist typea obesity alcohol age chd 160 12.00 5.73 23.11 Present 49 25.30 97.20 52 1 144 0.01 4.41 28.61 Absent 55 28.87 2.06 63 1 118 0.08 3.48 32.28 Present 52 29.14 3.81 46 0 170 7.50 6.41 38.03 Present 51 31.99 24.26 58 1 134 13.60 3.50 27.78 Present 60 25.99 57.34 49 1 132 6.20 6.47 36.21 Present 62 30.77 14.14 45 0 This data comes from a retrospective sample of males in a heart-disease high-risk region of the Western Cape, South Africa. The chd variable, which we will use as a response, indicates whether or not coronary heart disease is present in an individual. Note that this is coded as a numeric 0 / 1 variable. Using this as a response with glm() it is important to indicate family = binomial, otherwise ordinary linear regression will be fit. Later, we will see the use of a factor variable response, which is actually preferred, as you cannot accidentally fit ordinary linear regression. The predictors are various measurements for each individual, many related to heart health. For example sbp, systolic blood pressure, and ldl, low density lipoprotein cholesterol. For full details, use ?SAheart. We’ll begin by attempting to model the probability of coronary heart disease based on low density lipoprotein cholesterol. That is, we will fit the model \\[ \\log\\left(\\frac{P[\\texttt{chd} = 1]}{1 - P[\\texttt{chd} = 1]}\\right) = \\beta_0 + \\beta_{\\texttt{ldl}} x_{\\texttt{ldl}} \\] chd_mod_ldl = glm(chd ~ ldl, data = SAheart, family = binomial) plot(jitter(chd, factor = 0.1) ~ ldl, data = SAheart, pch = 20, ylab = &quot;Probability of CHD&quot;, xlab = &quot;Low Density Lipoprotein Cholesterol&quot;) grid() curve(predict(chd_mod_ldl, data.frame(ldl = x), type = &quot;response&quot;), add = TRUE, col = &quot;dodgerblue&quot;, lty = 2) As before, we plot the data in addition to the estimated probabilities. Note that we have “jittered” the data to make it easier to visualize, but the data do only take values 0 and 1. As we would expect, this plot indicates that as ldl increases, so does the probability of chd. coef(summary(chd_mod_ldl)) ## Estimate Std. Error z value Pr(&gt;|z|) ## (Intercept) -1.9686681 0.27307908 -7.209150 5.630207e-13 ## ldl 0.2746613 0.05163983 5.318787 1.044615e-07 To perform the test \\[ H_0: \\beta_{\\texttt{ldl}} = 0 \\] we use the summary() function as we have done so many times before. Like the \\(t\\)-test for ordinary linear regression, this returns the estimate of the parameter, its standard error, the relevant test statistic (\\(z\\)), and its p-value. Here we have an incredibly low p-value, so we reject the null hypothesis. The ldl variable appears to be a significant predictor. When fitting logistic regression, we can use the same formula syntax as ordinary linear regression. So, to fit an additive model using all available predictors, we use: chd_mod_additive = glm(chd ~ ., data = SAheart, family = binomial) We can then use the likelihood-ratio test to compare the two models. Specifically, we are testing \\[ H_0: \\beta_{\\texttt{sbp}} = \\beta_{\\texttt{tobacco}} = \\beta_{\\texttt{adiposity}} = \\beta_{\\texttt{famhist}} = \\beta_{\\texttt{typea}} = \\beta_{\\texttt{obesity}} = \\beta_{\\texttt{alcohol}} = \\beta_{\\texttt{age}} = 0 \\] We could manually calculate the test statistic, -2 * as.numeric(logLik(chd_mod_ldl) - logLik(chd_mod_additive)) ## [1] 92.13879 Or we could utilize the anova() function. By specifying test = \"LRT\", R will use the likelihood-ratio test to compare the two models. anova(chd_mod_ldl, chd_mod_additive, test = &quot;LRT&quot;) ## Analysis of Deviance Table ## ## Model 1: chd ~ ldl ## Model 2: chd ~ sbp + tobacco + ldl + adiposity + famhist + typea + obesity + ## alcohol + age ## Resid. Df Resid. Dev Df Deviance Pr(&gt;Chi) ## 1 460 564.28 ## 2 452 472.14 8 92.139 &lt; 2.2e-16 *** ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 We see that the test statistic that we had just calculated appears in the output. The very small p-value suggests that we prefer the larger model. While we prefer the additive model compared to the model with only a single predictor, do we actually need all of the predictors in the additive model? To select a subset of predictors, we can use a stepwise procedure as we did with ordinary linear regression. Recall that AIC and BIC were defined in terms of likelihoods. Here we demonstrate using AIC with a backwards selection procedure. chd_mod_selected = step(chd_mod_additive, trace = 0) coef(chd_mod_selected) ## (Intercept) tobacco ldl famhistPresent typea ## -6.44644451 0.08037533 0.16199164 0.90817526 0.03711521 ## age ## 0.05046038 We could again compare this model to the additive models. \\[ H_0: \\beta_{\\texttt{sbp}} = \\beta_{\\texttt{adiposity}} = \\beta_{\\texttt{obesity}} = \\beta_{\\texttt{alcohol}} = 0 \\] anova(chd_mod_selected, chd_mod_additive, test = &quot;LRT&quot;) ## Analysis of Deviance Table ## ## Model 1: chd ~ tobacco + ldl + famhist + typea + age ## Model 2: chd ~ sbp + tobacco + ldl + adiposity + famhist + typea + obesity + ## alcohol + age ## Resid. Df Resid. Dev Df Deviance Pr(&gt;Chi) ## 1 456 475.69 ## 2 452 472.14 4 3.5455 0.471 Here it seems that we would prefer the selected model. 17.3.5 Confidence Intervals We can create confidence intervals for the \\(\\beta\\) parameters using the confint() function as we did with ordinary linear regression. confint(chd_mod_selected, level = 0.99) ## Waiting for profiling to be done... ## 0.5 % 99.5 % ## (Intercept) -8.941825274 -4.18278990 ## tobacco 0.015704975 0.14986616 ## ldl 0.022923610 0.30784590 ## famhistPresent 0.330033483 1.49603366 ## typea 0.006408724 0.06932612 ## age 0.024847330 0.07764277 Note that we could create intervals by rearranging the results of the Wald test to obtain the Wald confidence interval. This would be given by \\[ \\hat{\\beta}_j \\pm z_{\\alpha/2} \\cdot \\text{SE}[\\hat{\\beta}_j]. \\] However, R is using a slightly different approach based on a concept called the profile likelihood. (The details of which we will omit.) Ultimately the intervals reported will be similar, but the method used by R is more common in practice, probably at least partially because it is the default approach in R. Check to see how intervals using the formula above compare to those from the output of confint(). (Or, note that using confint.default() will return the results of calculating the Wald confidence interval.) 17.3.6 Confidence Intervals for Mean Response Confidence intervals for the mean response require some additional thought. With a “large enough” sample, we have \\[ \\frac{\\hat{\\eta}({\\bf x}) - \\eta({\\bf x})}{\\text{SE}[\\hat{\\eta}({\\bf x})]} \\overset{\\text{approx}}{\\sim} N(0, 1) \\] Then we can create an approximate \\((1 - \\alpha)\\%\\) confidence intervals for \\(\\eta({\\bf x})\\) using \\[ \\hat{\\eta}({\\bf x}) \\pm z_{\\alpha/2} \\cdot \\text{SE}[\\hat{\\eta}({\\bf x})] \\] where \\(z_{\\alpha/2}\\) is the critical value such that \\(P(Z &gt; z_{\\alpha/2}) = \\alpha/2\\). This isn’t a particularly interesting interval. Instead, what we really want is an interval for the mean response, \\(p({\\bf x})\\). To obtain an interval for \\(p({\\bf x})\\), we simply apply the inverse logit transform to the endpoints of the interval for \\(\\eta.\\) \\[ \\left(\\text{logit}^{-1}(\\hat{\\eta}({\\bf x}) - z_{\\alpha/2} \\cdot \\text{SE}[\\hat{\\eta}({\\bf x})] ), \\ \\text{logit}^{-1}(\\hat{\\eta}({\\bf x}) + z_{\\alpha/2} \\cdot \\text{SE}[\\hat{\\eta}({\\bf x})])\\right) \\] To demonstrate creating these intervals, we’ll consider a new observation. new_obs = data.frame( sbp = 148.0, tobacco = 5, ldl = 12, adiposity = 31.23, famhist = &quot;Present&quot;, typea = 47, obesity = 28.50, alcohol = 23.89, age = 60 ) First, we’ll use the predict() function to obtain \\(\\hat{\\eta}({\\bf x})\\) for this observation. eta_hat = predict(chd_mod_selected, new_obs, se.fit = TRUE, type = &quot;link&quot;) eta_hat ## $fit ## 1 ## 1.579545 ## ## $se.fit ## [1] 0.4114796 ## ## $residual.scale ## [1] 1 By setting se.fit = TRUE, R also computes \\(\\text{SE}[\\hat{\\eta}({\\bf x})]\\). Note that we used type = \"link\", but this is actually a default value. We added it here to stress that the output from predict() will be the value of the link function. z_crit = qnorm(0.975) round(z_crit, 2) ## [1] 1.96 After obtaining the correct critical value, we can easily create a \\(95\\%\\) confidence interval for \\(\\eta({\\bf x})\\). eta_hat$fit + c(-1, 1) * z_crit * eta_hat$se.fit ## [1] 0.7730599 2.3860302 Now we simply need to apply the correct transformation to make this a confidence interval for \\(p({\\bf x})\\), the probability of coronary heart disease for this observation. Note that the boot package contains functions logit() and inv.logit() which are the logit and inverse logit transformations, respectively. boot::inv.logit(eta_hat$fit + c(-1, 1) * z_crit * eta_hat$se.fit) ## [1] 0.6841824 0.9157558 Notice, as we would expect, the bounds of this interval are both between 0 and 1. Also, since both bounds of the interval for \\(\\eta({\\bf x})\\) are positive, both bounds of the interval for \\(p({\\bf x})\\) are greater than 0.5. 17.3.7 Formula Syntax Without really thinking about it, we’ve been using our previous knowledge of R’s model formula syntax to fit logistic regression. 17.3.7.1 Interactions Let’s add an interaction between LDL and family history for the model we selected. chd_mod_interaction = glm(chd ~ alcohol + ldl + famhist + typea + age + ldl:famhist, data = SAheart, family = binomial) summary(chd_mod_interaction) ## ## Call: ## glm(formula = chd ~ alcohol + ldl + famhist + typea + age + ldl:famhist, ## family = binomial, data = SAheart) ## ## Coefficients: ## Estimate Std. Error z value Pr(&gt;|z|) ## (Intercept) -6.043472 0.937186 -6.449 1.13e-10 *** ## alcohol 0.003800 0.004332 0.877 0.38033 ## ldl 0.035593 0.071448 0.498 0.61837 ## famhistPresent -0.733836 0.618131 -1.187 0.23515 ## typea 0.036253 0.012172 2.978 0.00290 ** ## age 0.062416 0.009723 6.419 1.37e-10 *** ## ldl:famhistPresent 0.314311 0.114922 2.735 0.00624 ** ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 ## ## (Dispersion parameter for binomial family taken to be 1) ## ## Null deviance: 596.11 on 461 degrees of freedom ## Residual deviance: 477.46 on 455 degrees of freedom ## AIC: 491.46 ## ## Number of Fisher Scoring iterations: 5 Based on the \\(z\\)-test seen in the above summary, this interaction is significant. The effect of LDL on the probability of CHD is different depending on family history. 17.3.7.2 Polynomial Terms Let’s take the previous model, and now add a polynomial term. chd_mod_int_quad = glm(chd ~ alcohol + ldl + famhist + typea + age + ldl:famhist + I(ldl^2), data = SAheart, family = binomial) summary(chd_mod_int_quad) ## ## Call: ## glm(formula = chd ~ alcohol + ldl + famhist + typea + age + ldl:famhist + ## I(ldl^2), family = binomial, data = SAheart) ## ## Coefficients: ## Estimate Std. Error z value Pr(&gt;|z|) ## (Intercept) -6.096747 1.065952 -5.720 1.07e-08 *** ## alcohol 0.003842 0.004350 0.883 0.37716 ## ldl 0.056876 0.214420 0.265 0.79081 ## famhistPresent -0.723769 0.625167 -1.158 0.24698 ## typea 0.036248 0.012171 2.978 0.00290 ** ## age 0.062299 0.009788 6.365 1.95e-10 *** ## I(ldl^2) -0.001587 0.015076 -0.105 0.91617 ## ldl:famhistPresent 0.311615 0.117559 2.651 0.00803 ** ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 ## ## (Dispersion parameter for binomial family taken to be 1) ## ## Null deviance: 596.11 on 461 degrees of freedom ## Residual deviance: 477.45 on 454 degrees of freedom ## AIC: 493.45 ## ## Number of Fisher Scoring iterations: 5 Unsurprisingly, since this additional transformed variable wasn’t intelligently chosen, it is not significant. However, this does allow us to stress the fact that the syntax notation that we had been using with lm() works basically exactly the same for glm(), however now we understand that this is specifying the linear combination of predictions, \\(\\eta({\\bf x})\\). That is, the above fits the model \\[ \\log\\left(\\frac{p({\\bf x})}{1 - p({\\bf x})}\\right) = \\beta_0 + \\beta_{1}x_{\\texttt{alcohol}} + \\beta_{2}x_{\\texttt{ldl}} + \\beta_{3}x_{\\texttt{famhist}} + \\beta_{4}x_{\\texttt{typea}} + \\beta_{5}x_{\\texttt{age}} + \\beta_{6}x_{\\texttt{ldl}}x_{\\texttt{famhist}} + \\beta_{7}x_{\\texttt{ldl}}^2 \\] You may have realized this before we actually explicitly wrote it down! 17.3.8 Deviance You have probably noticed that the output from summary() is also very similar to that of ordinary linear regression. One difference, is the “deviance” being reported. The Null deviance is the deviance for the null model, that is, a model with no predictors. The Residual deviance is the deviance for the model that was fit. Deviance compares the model to a saturated model. (Without repeated observations, a saturated model is a model that fits perfectly, using a parameter for each observation.) Essentially, deviance is a generalized residual sum of squares for GLMs. Like RSS, deviance decreases as the model complexity increases. deviance(chd_mod_ldl) ## [1] 564.2788 deviance(chd_mod_selected) ## [1] 475.6856 deviance(chd_mod_additive) ## [1] 472.14 Note that these are nested, and we see that deviance does decrease as the model size becomes larger. So while a lower deviance is better, if the model becomes too big, it may be overfitting. Note that R also outputs AIC in the summary, which will penalize according to model size, to prevent overfitting. 17.4 Classification So far we’ve mostly used logistic regression to estimate class probabilities. The somewhat obvious next step is to use these probabilities to make “predictions,” which in this context, we would call classifications. Based on the values of the predictors, should an observation be classified as \\(Y = 1\\) or as \\(Y = 0\\)? Suppose we didn’t need to estimate probabilities from data, and instead, we actually knew both \\[ p({\\bf x}) = P[Y = 1 \\mid {\\bf X} = {\\bf x}] \\] and \\[ 1 - p({\\bf x}) = P[Y = 0 \\mid {\\bf X} = {\\bf x}]. \\] With this information, classifying observations based on the values of the predictors is actually extremely easy. Simply classify an observation to the class (\\(0\\) or \\(1\\)) with the larger probability. In general, this result is called the Bayes Classifier, \\[ C^B({\\bf x}) = \\underset{k}{\\mathrm{argmax}} \\ P[Y = k \\mid {\\bf X = x}]. \\] For a binary response, that is, \\[ \\hat{C}(\\bf x) = \\begin{cases} 1 &amp; p({\\bf x}) &gt; 0.5 \\\\ 0 &amp; p({\\bf x}) \\leq 0.5 \\end{cases} \\] Simply put, the Bayes classifier (not to be confused with the Naive Bayes Classifier) minimizes the probability of misclassification by classifying each observation to the class with the highest probability. Unfortunately, in practice, we won’t know the necessary probabilities to directly use the Bayes classifier. Instead we’ll have to use estimated probabilities. So to create a classifier that seeks to minimize misclassifications, we would use, \\[ \\hat{C}({\\bf x}) = \\underset{k}{\\mathrm{argmax}} \\ \\hat{P}[Y = k \\mid {\\bf X = x}]. \\] In the case of a binary response since \\(\\hat{p}({\\bf x}) = 1 - \\hat{p}({\\bf x})\\), this becomes \\[ \\hat{C}(\\bf x) = \\begin{cases} 1 &amp; \\hat{p}({\\bf x}) &gt; 0.5 \\\\ 0 &amp; \\hat{p}({\\bf x}) \\leq 0.5 \\end{cases} \\] Using this simple classification rule, we can turn logistic regression into a classifier. To use logistic regression for classification, we first use logistic regression to obtain estimated probabilities, \\(\\hat{p}({\\bf x})\\), then use these in conjunction with the above classification rule. Logistic regression is just one of many ways that these probabilities could be estimated. In a course completely focused on machine learning, you’ll learn many additional ways to do this, as well as methods to directly make classifications without needing to first estimate probabilities. But since we had already introduced logistic regression, it makes sense to discuss it in the context of classification. 17.4.1 spam Example To illustrate the use of logistic regression as a classifier, we will use the spam dataset from the kernlab package. # install.packages(&quot;kernlab&quot;) library(kernlab) data(&quot;spam&quot;) tibble::as.tibble(spam) ## Warning: `as.tibble()` was deprecated in tibble 2.0.0. ## ℹ Please use `as_tibble()` instead. ## ℹ The signature and semantics have changed, see `?as_tibble`. ## This warning is displayed once every 8 hours. ## Call `lifecycle::last_lifecycle_warnings()` to see where this warning was ## generated. ## # A tibble: 4,601 × 58 ## make address all num3d our over remove internet order mail receive ## &lt;dbl&gt; &lt;dbl&gt; &lt;dbl&gt; &lt;dbl&gt; &lt;dbl&gt; &lt;dbl&gt; &lt;dbl&gt; &lt;dbl&gt; &lt;dbl&gt; &lt;dbl&gt; &lt;dbl&gt; ## 1 0 0.64 0.64 0 0.32 0 0 0 0 0 0 ## 2 0.21 0.28 0.5 0 0.14 0.28 0.21 0.07 0 0.94 0.21 ## 3 0.06 0 0.71 0 1.23 0.19 0.19 0.12 0.64 0.25 0.38 ## 4 0 0 0 0 0.63 0 0.31 0.63 0.31 0.63 0.31 ## 5 0 0 0 0 0.63 0 0.31 0.63 0.31 0.63 0.31 ## 6 0 0 0 0 1.85 0 0 1.85 0 0 0 ## 7 0 0 0 0 1.92 0 0 0 0 0.64 0.96 ## 8 0 0 0 0 1.88 0 0 1.88 0 0 0 ## 9 0.15 0 0.46 0 0.61 0 0.3 0 0.92 0.76 0.76 ## 10 0.06 0.12 0.77 0 0.19 0.32 0.38 0 0.06 0 0 ## # ℹ 4,591 more rows ## # ℹ 47 more variables: will &lt;dbl&gt;, people &lt;dbl&gt;, report &lt;dbl&gt;, addresses &lt;dbl&gt;, ## # free &lt;dbl&gt;, business &lt;dbl&gt;, email &lt;dbl&gt;, you &lt;dbl&gt;, credit &lt;dbl&gt;, ## # your &lt;dbl&gt;, font &lt;dbl&gt;, num000 &lt;dbl&gt;, money &lt;dbl&gt;, hp &lt;dbl&gt;, hpl &lt;dbl&gt;, ## # george &lt;dbl&gt;, num650 &lt;dbl&gt;, lab &lt;dbl&gt;, labs &lt;dbl&gt;, telnet &lt;dbl&gt;, ## # num857 &lt;dbl&gt;, data &lt;dbl&gt;, num415 &lt;dbl&gt;, num85 &lt;dbl&gt;, technology &lt;dbl&gt;, ## # num1999 &lt;dbl&gt;, parts &lt;dbl&gt;, pm &lt;dbl&gt;, direct &lt;dbl&gt;, cs &lt;dbl&gt;, … This dataset, created in the late 1990s at Hewlett-Packard Labs, contains 4601 emails, of which 1813 are considered spam. The remaining are not spam. (Which for simplicity, we might call, ham.) Additional details can be obtained by using ?spam or by visiting the UCI Machine Learning Repository. The response variable, type, is a factor with levels that label each email as spam or nonspam. When fitting models, nonspam will be the reference level, \\(Y = 0\\), as it comes first alphabetically. is.factor(spam$type) ## [1] TRUE levels(spam$type) ## [1] &quot;nonspam&quot; &quot;spam&quot; Many of the predictors (often called features in machine learning) are engineered based on the emails. For example, charDollar is the number of times an email contains the $ character. Some variables are highly specific to this dataset, for example george and num650. (The name and area code for one of the researchers whose emails were used.) We should keep in mind that this dataset was created based on emails sent to academic type researchers in the 1990s. Any results we derive probably won’t generalize to modern emails for the general public. To get started, we’ll first test-train split the data. set.seed(42) # spam_idx = sample(nrow(spam), round(nrow(spam) / 2)) spam_idx = sample(nrow(spam), 1000) spam_trn = spam[spam_idx, ] spam_tst = spam[-spam_idx, ] We’ve used a somewhat small train set relative to the total size of the dataset. In practice it should likely be larger, but this is simply to keep training time low for illustration and rendering of this document. fit_caps = glm(type ~ capitalTotal, data = spam_trn, family = binomial) fit_selected = glm(type ~ edu + money + capitalTotal + charDollar, data = spam_trn, family = binomial) fit_additive = glm(type ~ ., data = spam_trn, family = binomial) fit_over = glm(type ~ capitalTotal * (.), data = spam_trn, family = binomial, maxit = 75) We’ll fit four logistic regressions, each more complex than the previous. Note that we’re suppressing two warnings. The first we briefly mentioned previously. ## Warning: glm.fit: fitted probabilities numerically 0 or 1 occurred Note that, when we receive this warning, we should be highly suspicious of the parameter estimates. coef(fit_selected) ## (Intercept) edu money capitalTotal charDollar ## -1.1199744712 -1.9837988840 0.9784675298 0.0007757011 11.5772904667 However, the model can still be used to create a classifier, and we will evaluate that classifier on its own merits. We also, “suppressed” the warning: ## Warning: glm.fit: algorithm did not converge In reality, we didn’t actually suppress it, but instead changed maxit to 75, when fitting the model fit_over. This was enough additional iterations to allow the iteratively reweighted least squares algorithm to converge when fitting the model. 17.4.2 Evaluating Classifiers The metric we’ll be most interested in for evaluating the overall performance of a classifier is the misclassification rate. (Sometimes, instead accuracy is reported, which is instead the proportion of correct classifications, so both metrics serve the same purpose.) \\[ \\text{Misclass}(\\hat{C}, \\text{Data}) = \\frac{1}{n}\\sum_{i = 1}^{n}I(y_i \\neq \\hat{C}({\\bf x_i})) \\] \\[ I(y_i \\neq \\hat{C}({\\bf x_i})) = \\begin{cases} 0 &amp; y_i = \\hat{C}({\\bf x_i}) \\\\ 1 &amp; y_i \\neq \\hat{C}({\\bf x_i}) \\\\ \\end{cases} \\] When using this metric on the training data, it will have the same issues as RSS did for ordinary linear regression, that is, it will only go down. # training misclassification rate mean(ifelse(predict(fit_caps) &gt; 0, &quot;spam&quot;, &quot;nonspam&quot;) != spam_trn$type) ## [1] 0.339 mean(ifelse(predict(fit_selected) &gt; 0, &quot;spam&quot;, &quot;nonspam&quot;) != spam_trn$type) ## [1] 0.224 mean(ifelse(predict(fit_additive) &gt; 0, &quot;spam&quot;, &quot;nonspam&quot;) != spam_trn$type) ## [1] 0.066 mean(ifelse(predict(fit_over) &gt; 0, &quot;spam&quot;, &quot;nonspam&quot;) != spam_trn$type) ## [1] 0.118 Because of this, training data isn’t useful for evaluating, as it would suggest that we should always use the largest possible model, when in reality, that model is likely overfitting. Recall, a model that is too complex will overfit. A model that is too simple will underfit. (We’re looking for something in the middle.) To overcome this, we’ll use cross-validation as we did with ordinary linear regression, but this time we’ll cross-validate the misclassification rate. To do so, we’ll use the cv.glm() function from the boot library. It takes arguments for the data (in this case training), a model fit via glm(), and K, the number of folds. It returns delta, a vector of the average mean-squared error (MSE) and the bias-corrected MSE. See ?cv.glm for details. Previously, for cross-validating RMSE in ordinary linear regression, we used LOOCV. We certainly could do that here. However, with logistic regression, we no longer have the clever trick that would allow us to obtain a LOOCV metric without needing to fit the model \\(n\\) times. So instead, we’ll use 5-fold cross-validation. (5 and 10 fold are the most common in practice.) Instead of leaving a single observation out repeatedly, we’ll leave out a fifth of the data. Essentially we’ll repeat the following process 5 times: Randomly set aside a fifth of the data (each observation will only be held-out once) Train model on remaining data Evaluate misclassification rate on held-out data The 5-fold cross-validated misclassification rate will be the average of these misclassification rates. By only needing to refit the model 5 times, instead of \\(n\\) times, we will save a lot of computation time. library(boot) set.seed(1) cv.glm(spam_trn, fit_caps, K = 5)$delta[1] ## [1] 0.2166961 cv.glm(spam_trn, fit_selected, K = 5)$delta[1] ## [1] 0.1587043 cv.glm(spam_trn, fit_additive, K = 5)$delta[1] ## [1] 0.08684467 cv.glm(spam_trn, fit_over, K = 5)$delta[1] ## [1] 0.146 Note that we’re suppressing warnings again here. (Now there would be a lot more, since were fitting a total of 20 models.) Based on these results, fit_caps and fit_selected are underfitting relative to fit_additive. Similarly, fit_over is overfitting relative to fit_additive. Thus, based on these results, we prefer the classifier created based on the logistic regression fit and stored in fit_additive. Going forward, to evaluate and report on the efficacy of this classifier, we’ll use the test dataset. We’re going to take the position that the test data set should never be used in training, which is why we used cross-validation within the training dataset to select a model. Even though cross-validation uses hold-out sets to generate metrics, at some point all of the data is used for training. To quickly summarize how well this classifier works, we’ll create a confusion matrix. Confusion Matrix It further breaks down the classification errors into false positives and false negatives. make_conf_mat = function(predicted, actual) { table(predicted = predicted, actual = actual) } Let’s explicitly store the predicted values of our classifier on the test dataset. spam_tst_pred = ifelse(predict(fit_additive, spam_tst) &gt; 0, &quot;spam&quot;, &quot;nonspam&quot;) spam_tst_pred = ifelse(predict(fit_additive, spam_tst, type = &quot;response&quot;) &gt; 0.5, &quot;spam&quot;, &quot;nonspam&quot;) The previous two lines of code produce the same output, that is the same predictions, since \\[ \\eta({\\bf x}) = 0 \\iff p({\\bf x}) = 0.5 \\] Now we’ll use these predictions to create a confusion matrix. (conf_mat_50 = make_conf_mat(predicted = spam_tst_pred, actual = spam_tst$type)) ## actual ## predicted nonspam spam ## nonspam 2057 157 ## spam 127 1260 \\[ \\text{Prev} = \\frac{\\text{P}}{\\text{Total Obs}}= \\frac{\\text{TP + FN}}{\\text{Total Obs}} \\] table(spam_tst$type) / nrow(spam_tst) ## ## nonspam spam ## 0.6064982 0.3935018 First, note that to be a reasonable classifier, it needs to outperform the obvious classifier of simply classifying all observations to the majority class. In this case, classifying everything as non-spam for a test misclassification rate of 0.3935018 Next, we can see that using the classifier created from fit_additive, only a total of \\(127 + 157 = 284\\) from the total of 3601 emails in the test set are misclassified. Overall, the accuracy in the test set it mean(spam_tst_pred == spam_tst$type) ## [1] 0.921133 In other words, the test misclassification is mean(spam_tst_pred != spam_tst$type) ## [1] 0.07886698 This seems like a decent classifier… However, are all errors created equal? In this case, absolutely not. The 127 non-spam emails that were marked as spam (false positives) are a problem. We can’t allow important information, say, a job offer, to miss our inbox and get sent to the spam folder. On the other hand, the 157 spam email that would make it to an inbox (false negatives) are easily dealt with, just delete them. Instead of simply evaluating a classifier based on its misclassification rate (or accuracy), we’ll define two additional metrics, sensitivity and specificity. Note that these are simply two of many more metrics that can be considered. The Wikipedia page for sensitivity and specificity details a large number of metrics that can be derived from a confusion matrix. Sensitivity is essentially the true positive rate. So when sensitivity is high, the number of false negatives is low. \\[ \\text{Sens} = \\text{True Positive Rate} = \\frac{\\text{TP}}{\\text{P}} = \\frac{\\text{TP}}{\\text{TP + FN}} \\] Here we have an R function to calculate the sensitivity based on the confusion matrix. Note that this function is good for illustrative purposes, but is easily broken. (Think about what happens if there are no “positives” predicted.) get_sens = function(conf_mat) { conf_mat[2, 2] / sum(conf_mat[, 2]) } Specificity is essentially the true negative rate. So when specificity is high, the number of false positives is low. \\[ \\text{Spec} = \\text{True Negative Rate} = \\frac{\\text{TN}}{\\text{N}} = \\frac{\\text{TN}}{\\text{TN + FP}} \\] get_spec = function(conf_mat) { conf_mat[1, 1] / sum(conf_mat[, 1]) } We calculate both based on the confusion matrix we had created for our classifier. get_sens(conf_mat_50) ## [1] 0.8892025 get_spec(conf_mat_50) ## [1] 0.9418498 Recall that we had created this classifier using a probability of \\(0.5\\) as a “cutoff” for how observations should be classified. Now we’ll modify this cutoff. We’ll see that by modifying the cutoff, \\(c\\), we can improve sensitivity or specificity at the expense of the overall accuracy (misclassification rate). \\[ \\hat{C}(\\bf x) = \\begin{cases} 1 &amp; \\hat{p}({\\bf x}) &gt; c \\\\ 0 &amp; \\hat{p}({\\bf x}) \\leq c \\end{cases} \\] Additionally, if we change the cutoff to improve sensitivity, we’ll decrease specificity, and vice versa. First let’s see what happens when we lower the cutoff from \\(0.5\\) to \\(0.1\\) to create a new classifier, and thus new predictions. spam_tst_pred_10 = ifelse(predict(fit_additive, spam_tst, type = &quot;response&quot;) &gt; 0.1, &quot;spam&quot;, &quot;nonspam&quot;) This is essentially decreasing the threshold for an email to be labeled as spam, so far more emails will be labeled as spam. We see that in the following confusion matrix. (conf_mat_10 = make_conf_mat(predicted = spam_tst_pred_10, actual = spam_tst$type)) ## actual ## predicted nonspam spam ## nonspam 1583 29 ## spam 601 1388 Unfortunately, while this does greatly reduce false negatives, false positives have almost quadrupled. We see this reflected in the sensitivity and specificity. get_sens(conf_mat_10) ## [1] 0.9795342 get_spec(conf_mat_10) ## [1] 0.7248168 This classifier, using \\(0.1\\) instead of \\(0.5\\) has a higher sensitivity, but a much lower specificity. Clearly, we should have moved the cutoff in the other direction. Let’s try \\(0.9\\). spam_tst_pred_90 = ifelse(predict(fit_additive, spam_tst, type = &quot;response&quot;) &gt; 0.9, &quot;spam&quot;, &quot;nonspam&quot;) This is essentially increasing the threshold for an email to be labeled as spam, so far fewer emails will be labeled as spam. Again, we see that in the following confusion matrix. (conf_mat_90 = make_conf_mat(predicted = spam_tst_pred_90, actual = spam_tst$type)) ## actual ## predicted nonspam spam ## nonspam 2136 537 ## spam 48 880 This is the result we’re looking for. We have far fewer false positives. While sensitivity is greatly reduced, specificity has gone up. get_sens(conf_mat_90) ## [1] 0.6210303 get_spec(conf_mat_90) ## [1] 0.978022 While this is far fewer false positives, is it acceptable though? Still probably not. Also, don’t forget, this would actually be a terrible spam detector today since this is based on data from a very different era of the internet, for a very specific set of people. Spam has changed a lot since the 90s! (Ironically, machine learning is probably partially to blame.) This chapter has provided a rather quick introduction to classification, and thus, machine learning. For a more complete coverage of machine learning, An Introduction to Statistical Learning is a highly recommended resource. Additionally, R for Statistical Learning has been written as a supplement which provides additional detail on how to perform these methods using R. The classification and logistic regression chapters might be useful. We should note that the code to perform classification using logistic regression is presented in a way that illustrates the concepts to the reader. In practice, you may prefer to use a more general machine learning pipeline such as caret in R. This will streamline processes for creating predictions and generating evaluation metrics. 17.5 R Markdown The R Markdown file for this chapter can be found here: logistic.Rmd The file was created using R version 4.3.2. "],["beyond.html", "Chapter 18 Beyond 18.1 What’s Next 18.2 RStudio 18.3 Tidy Data 18.4 Visualization 18.5 Web Applications 18.6 Experimental Design 18.7 Machine Learning 18.8 Time Series 18.9 Bayesianism 18.10 High Performance Computing 18.11 Further R Resources", " Chapter 18 Beyond “End? No, the journey doesn’t end here.” — J.R.R. Tolkien After reading this chapter you will be able to: Understand the roadmap to continued education about models and the R programming language. 18.1 What’s Next So you’ve completed STAT 420, where do you go from here? Now that you understand the basics of linear modeling, there is a wide world of applied statistics waiting to be explored. We’ll briefly detail some resources and discuss how they relate to what you have learned in STAT 420. 18.2 RStudio RStudio has recently released version 1.0! This is exciting for a number of reasons, especially the release of R Notebooks. R Notebooks combine the RMarkdown you have already learned with the ability to work interactively. 18.3 Tidy Data In this textbook, much of the data we have seen has been nice and tidy. It was rectangular where each row was an observation and each column was a variable. This is not always the case! Many packages have been developed to deal with data, and force it into a nice format, which is called tidy data, that we can then use for modeling. Often during analysis, this is where a large portion of your time will be spent. The R community has started to call this collection of packages the Tidyverse. It was once called the Hadleyverse, as Hadley Wickham has authored so many of the packages. Hadley is writing a book called R for Data Science which describes the use of many of these packages. (And also how to use some to make the modeling process better!) This book is a great starting point for diving deeper into the R community. The two main packages are dplyr and tidyr both of which are used internally in RStudio. 18.4 Visualization In this course, we have mostly used the base plotting methods in R. When working with tidy data, many users prefer to use the ggplot2 package, also developed by Hadley Wickham. RStudio provides a rather detailed “cheat sheet” for working with ggplot2. The community maintains a graph gallery of examples. Use of the manipulate package with RStudio gives the ability to quickly change a static graphic to become interactive. 18.5 Web Applications RStudio has made it incredibly easy to create data products through the use of Shiny, which allows for the creation of web applications with R. RStudio maintains an ever-growing tutorial and gallery of examples. 18.6 Experimental Design In the ANOVA chapter, we briefly discussed experimental design. This topic could easily be its own class, and is currently an area of revitalized interest with the rise of A/B testing. Two more classic statistical references include Statistics for Experimenters by Box, Hunter, and Hunter as well as Design and Analysis of Experiments by Douglas Montgomery. There are several R packages for design of experiments, listed in the CRAN Task View. 18.7 Machine Learning Using models for prediction is the key focus of machine learning. There are many methods, each with its own package, however R has a wonderful package called caret, Classification And REgression Training, which provides a unified interface to training these models. It also contains various utilities for data processing and visualization that are useful for predictive modeling. Applied Predictive Modeling by Max Kuhn, the author of the caret package, is a good general resource for predictive modeling, which obviously utilizes R. An Introduction to Statistical Learning by James, Witten, Hastie, and Tibshirani is a gentle introduction to machine learning from a statistical perspective which uses R and picks up right where this course stops. This is based on the often referenced The Elements of Statistical Learning by Hastie, Tibshirani, and Friedman. Both are freely available online. 18.7.1 Deep Learning While it probably isn’t the best tool for the job, R now has the ability to train deep neural networks via TensorFlow. 18.8 Time Series In this class we have only considered independent data. What if data is dependent? Time Series is the area of statistics which deals with this issue, and could easily span multiple courses. The primary textbook for STAT 429: Time Series Analysis at the University of Illinois that is free is: Time Series Analysis and Its Applications: With R Examples by Shumway and Stoffer Some tutorials: Little Book of R for Time Series Quick R: Time Series and Forecasting TSA: Start to Finish Examples When performing time series analysis in R you should be aware of the many packages that are useful for analysis. It should be hard to avoid the forecast and zoo packages. Often the most difficult part will be dealing with time and date data. Make sure you are utilizing one of the many packages that help with this. 18.9 Bayesianism In this class, we have worked within the frequentist view of statistics. There is an entire alternative universe of Bayesian statistics. Doing Bayesian Data Analysis: A Tutorial with R, JAGS, and Stan by John Kruschke is a great introduction to the topic. It introduces the world of probabilistic programming, in particular Stan, which can be used in both R and Python. 18.10 High Performance Computing Often R will be called a “slow” language, for two reasons. One, because many do not understand R. Two, because sometimes it really is. Luckily, it is easy to extend R via the Rcpp package to allow for faster code. Many modern R packages utilize Rcpp to achieve better performance. 18.11 Further R Resources Also, don’t forget that previously in this book we have outlined a large number of R resources. Now that you’ve gotten started with R many of these will be much more useful. If any of these topics interest you, and you would like more information, please don’t hesitate to start a discussion on the forums! :) "],["appendix.html", "Chapter 19 Appendix", " Chapter 19 Appendix "],["404.html", "Page not found", " Page not found The page you requested cannot be found (perhaps it was moved or renamed). You may want to try searching to find the page's new location, or use the table of contents to find the page you are looking for. "]]
+[["index.html", "Applied Statistics with R Chapter 1 Introduction 1.1 About This Book 1.2 Conventions 1.3 Acknowledgements 1.4 License", " Applied Statistics with R David Dalpiaz Chapter 1 Introduction Welcome to Applied Statistics with R! 1.1 About This Book This book was originally (and currently) designed for use with STAT 420, Methods of Applied Statistics, at the University of Illinois Urbana-Champaign. It may certainly be used elsewhere, but any references to “this course” in this book specifically refer to STAT 420. This book is under active development. When possible, it would be best to always access the text online to be sure you are using the most up-to-date version. Also, the html version provides additional features such as changing text size, font, and colors. If you are in need of a local copy, a pdf version is continuously maintained, however, because a pdf uses pages, the formatting may not be as functional. (In other words, the author needs to go back and spend some time working on the pdf formatting.) Since this book is under active development you may encounter errors ranging from typos, to broken code, to poorly explained topics. If you do, please let us know! Simply send an email and we will make the changes as soon as possible. (dalpiaz2 AT illinois DOT edu) Or, if you know RMarkdown and are familiar with GitHub, make a pull request and fix an issue yourself! This process is partially automated by the edit button in the top-left corner of the html version. If your suggestion or fix becomes part of the book, you will be added to the list at the end of this chapter. We’ll also link to your GitHub account, or personal website upon request. This text uses MathJax to render mathematical notation for the web. Occasionally, but rarely, a JavaScript error will prevent MathJax from rendering correctly. In this case, you will see the “code” instead of the expected mathematical equations. From experience, this is almost always fixed by simply refreshing the page. You’ll also notice that if you right-click any equation you can obtain the MathML Code (for copying into Microsoft Word) or the TeX command used to generate the equation. \\[ a^2 + b^2 = c^2 \\] 1.2 Conventions R code will be typeset using a monospace font which is syntax highlighted. a = 3 b = 4 sqrt(a^2 + b^2) R output lines, which would appear in the console will begin with ##. They will generally not be syntax highlighted. ## [1] 5 We use the quantity \\(p\\) to refer to the number of \\(\\beta\\) parameters in a linear model, not the number of predictors. Don’t worry if you don’t know what this means yet! 1.3 Acknowledgements Material in this book was heavily influenced by: Alex Stepanov Longtime instructor of STAT 420 at the University of Illinois at Urbana-Champaign. The author of this book actually took Alex’s STAT 420 class many years ago! Alex provided or inspired many of the examples in the text. David Unger Another STAT 420 instructor at the University of Illinois at Urbana-Champaign. Co-taught with the author during the summer of 2016 while this book was first being developed. Provided endless hours of copy editing and countless suggestions. James Balamuta Current graduate student at the University of Illinois at Urbana-Champaign. Provided the initial push to write this book by introducing the author to the bookdown package in R. Also a frequent contributor via GitHub. Your name could be here! Suggest an edit! Correct a typo! If you submit a correction and would like to be listed below, please provide your name as you would like it to appear, as well as a link to a GitHub, LinkedIn, or personal website. Daniel McQuillan Mason Rubenstein Yuhang Wang Zhao Liu Jinfeng Xiao Somu Palaniappan Michael Hung-Yiu Chan Eloise Rosen Kiomars Nassiri Jeff Gerlach Brandon Ching Ray Fix Tyler Kim Yeongho Kim Elmar Langholz Thai Duy Cuong Nguyen Junyoung Kim Sezgin Kucukcoban Tony Ma Radu Manolescu Dileep Pasumarthi Sihun Wang Joseph Wilson Yingkui Lin Andy Siddall Nishant Balepur Durga Krovi Raj Krishnan Ed Pureza Siddharth Singh Schillaci Mcinnis Ivan Valdes Castillo Tony Mu Salman Yousaf Yutaro Nishiyama Regina Sahani Goonetilleke Paul Zuradzki Will Tsai Ellen Veomett David Newman Yixing Zheng Ossama Sybesma 1.4 License This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License. "],["introduction-to-r.html", "Chapter 2 Introduction to R 2.1 Getting Started 2.2 Basic Calculations 2.3 Getting Help 2.4 Installing Packages", " Chapter 2 Introduction to R 2.1 Getting Started R is both a programming language and software environment for statistical computing, which is free and open-source. To get started, you will need to install two pieces of software: R, the actual programming language. Chose your operating system, and select the most recent version, 4.4.1. RStudio, an excellent IDE for working with R. Note, you must have R installed to use RStudio. RStudio is simply an interface used to interact with R. The popularity of R is on the rise, and every day it becomes a better tool for statistical analysis. It even generated this book! (A skill you will learn in this course.) There are many good resources for learning R. The following few chapters will serve as a whirlwind introduction to R. They are by no means meant to be a complete reference for the R language, but simply an introduction to the basics that we will need along the way. Several of the more important topics will be re-stressed as they are actually needed for analyses. These introductory R chapters may feel like an overwhelming amount of information. You are not expected to pick up everything the first time through. You should try all of the code from these chapters, then return to them a number of times as you return to the concepts when performing analyses. R is used both for software development and data analysis. We will operate in a grey area, somewhere between these two tasks. Our main goal will be to analyze data, but we will also perform programming exercises that help illustrate certain concepts. RStudio has a large number of useful keyboard shortcuts. A list of these can be found using a keyboard shortcut – the keyboard shortcut to rule them all: On Windows: Alt + Shift + K On Mac: Option + Shift + K The RStudio team has developed a number of “cheatsheets” for working with both R and RStudio. This particular cheatsheet for “Base” R will summarize many of the concepts in this document. (“Base” R is a name used to differentiate the practice of using built-in R functions, as opposed to using functions from outside packages, in particular, those from the tidyverse. More on this later.) When programming, it is often a good practice to follow a style guide. (Where do spaces go? Tabs or spaces? Underscores or CamelCase when naming variables?) No style guide is “correct,” but it helps to be aware of what others do. The more important thing is to be consistent within your own code. Hadley Wickham Style Guide from Advanced R Google Style Guide For this course, our main deviation from these two guides is the use of = in place of &lt;-. (More on that later.) 2.2 Basic Calculations To get started, we’ll use R like a simple calculator. Addition, Subtraction, Multiplication, and Division Math R Result \\(3 + 2\\) 3 + 2 5 \\(3 - 2\\) 3 - 2 1 \\(3 \\cdot2\\) 3 * 2 6 \\(3 / 2\\) 3 / 2 1.5 Exponents Math R Result \\(3^2\\) 3 ^ 2 9 \\(2^{(-3)}\\) 2 ^ (-3) 0.125 \\(100^{1/2}\\) 100 ^ (1 / 2) 10 \\(\\sqrt{100}\\) sqrt(100) 10 Mathematical Constants Math R Result \\(\\pi\\) pi 3.1415927 \\(e\\) exp(1) 2.7182818 Logarithms Note that we will use \\(\\ln\\) and \\(\\log\\) interchangeably to mean the natural logarithm. There is no ln() in R, instead it uses log() to mean the natural logarithm. Math R Result \\(\\log(e)\\) log(exp(1)) 1 \\(\\log_{10}(1000)\\) log10(1000) 3 \\(\\log_{2}(8)\\) log2(8) 3 \\(\\log_{4}(16)\\) log(16, base = 4) 2 Trigonometry Math R Result \\(\\sin(\\pi / 2)\\) sin(pi / 2) 1 \\(\\cos(0)\\) cos(0) 1 2.3 Getting Help In using R as a calculator, we have seen a number of functions: sqrt(), exp(), log() and sin(). To get documentation about a function in R, simply put a question mark in front of the function name and RStudio will display the documentation, for example: ?log ?sin ?paste ?lm Frequently one of the most difficult things to do when learning R is asking for help. First, you need to decide to ask for help, then you need to know how to ask for help. Your very first line of defense should be to Google your error message or a short description of your issue. (The ability to solve problems using this method is quickly becoming an extremely valuable skill.) If that fails, and it eventually will, you should ask for help. There are a number of things you should include when emailing an instructor, or posting to a help website such as Stack Exchange. Describe what you expect the code to do. State the end goal you are trying to achieve. (Sometimes what you expect the code to do, is not what you want to actually do.) Provide the full text of any errors you have received. Provide enough code to recreate the error. Often for the purpose of this course, you could simply email your entire .R or .Rmd file. Sometimes it is also helpful to include a screenshot of your entire RStudio window when the error occurs. If you follow these steps, you will get your issue resolved much quicker, and possibly learn more in the process. Do not be discouraged by running into errors and difficulties when learning R. (Or any technical skill.) It is simply part of the learning process. 2.4 Installing Packages R comes with a number of built-in functions and datasets, but one of the main strengths of R as an open-source project is its package system. Packages add additional functions and data. Frequently if you want to do something in R, and it is not available by default, there is a good chance that there is a package that will fulfill your needs. To install a package, use the install.packages() function. Think of this as buying a recipe book from the store, bringing it home, and putting it on your shelf. install.packages(&quot;ggplot2&quot;) Once a package is installed, it must be loaded into your current R session before being used. Think of this as taking the book off of the shelf and opening it up to read. library(ggplot2) Once you close R, all the packages are closed and put back on the imaginary shelf. The next time you open R, you do not have to install the package again, but you do have to load any packages you intend to use by invoking library(). "],["data-and-programming.html", "Chapter 3 Data and Programming 3.1 Data Types 3.2 Data Structures 3.3 Programming Basics", " Chapter 3 Data and Programming 3.1 Data Types R has a number of basic data types. Numeric Also known as Double. The default type when dealing with numbers. Examples: 1, 1.0, 42.5 Integer Examples: 1L, 2L, 42L Complex Example: 4 + 2i Logical Two possible values: TRUE and FALSE You can also use T and F, but this is not recommended. NA is also considered logical. Character Examples: \"a\", \"Statistics\", \"1 plus 2.\" 3.2 Data Structures R also has a number of basic data structures. A data structure is either homogeneous (all elements are of the same data type) or heterogeneous (elements can be of more than one data type). Dimension Homogeneous Heterogeneous 1 Vector List 2 Matrix Data Frame 3+ Array 3.2.1 Vectors Many operations in R make heavy use of vectors. Vectors in R are indexed starting at 1. That is what the [1] in the output is indicating, that the first element of the row being displayed is the first element of the vector. Larger vectors will start additional rows with [*] where * is the index of the first element of the row. Possibly the most common way to create a vector in R is using the c() function, which is short for “combine.” As the name suggests, it combines a list of elements separated by commas. c(1, 3, 5, 7, 8, 9) ## [1] 1 3 5 7 8 9 Here R simply outputs this vector. If we would like to store this vector in a variable, we can do so with the assignment operator =. In this case the variable x now holds the vector we just created, and we can access the vector by typing x. x = c(1, 3, 5, 7, 8, 9) x ## [1] 1 3 5 7 8 9 As an aside, there is a long history of the assignment operator in R, partially due to the keys available on the keyboards of the creators of the S language. (Which preceded R.) For simplicity we will use =, but know that often you will see &lt;- as the assignment operator. The pros and cons of these two are well beyond the scope of this book, but know that for our purposes you will have no issue if you simply use =. If you are interested in the weird cases where the difference matters, check out The R Inferno. If you wish to use &lt;-, you will still need to use =, however only for argument passing. Some users like to keep assignment (&lt;-) and argument passing (=) separate. No matter what you choose, the more important thing is that you stay consistent. Also, if working on a larger collaborative project, you should use whatever style is already in place. Because vectors must contain elements that are all the same type, R will automatically coerce to a single type when attempting to create a vector that combines multiple types. c(42, &quot;Statistics&quot;, TRUE) ## [1] &quot;42&quot; &quot;Statistics&quot; &quot;TRUE&quot; c(42, TRUE) ## [1] 42 1 Frequently you may wish to create a vector based on a sequence of numbers. The quickest and easiest way to do this is with the : operator, which creates a sequence of integers between two specified integers. (y = 1:100) ## [1] 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 ## [19] 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 ## [37] 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 ## [55] 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 ## [73] 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 ## [91] 91 92 93 94 95 96 97 98 99 100 Here we see R labeling the rows after the first since this is a large vector. Also, we see that by putting parentheses around the assignment, R both stores the vector in a variable called y and automatically outputs y to the console. Note that scalars do not exist in R. They are simply vectors of length 1. 2 ## [1] 2 To create a sequence that is not limited to consecutive integers, use the seq() function to define a sequence by its start, end, and increment. seq(from = 1.5, to = 4.2, by = 0.1) ## [1] 1.5 1.6 1.7 1.8 1.9 2.0 2.1 2.2 2.3 2.4 2.5 2.6 2.7 2.8 2.9 3.0 3.1 3.2 3.3 ## [20] 3.4 3.5 3.6 3.7 3.8 3.9 4.0 4.1 4.2 We will discuss functions in detail later, but note here that the input labels from, to, and by are optional. seq(1.5, 4.2, 0.1) ## [1] 1.5 1.6 1.7 1.8 1.9 2.0 2.1 2.2 2.3 2.4 2.5 2.6 2.7 2.8 2.9 3.0 3.1 3.2 3.3 ## [20] 3.4 3.5 3.6 3.7 3.8 3.9 4.0 4.1 4.2 Another common operation to create a vector is rep(), which can repeat a single value a number of times. rep(&quot;A&quot;, times = 10) ## [1] &quot;A&quot; &quot;A&quot; &quot;A&quot; &quot;A&quot; &quot;A&quot; &quot;A&quot; &quot;A&quot; &quot;A&quot; &quot;A&quot; &quot;A&quot; The rep() function can be used to repeat a vector some number of times. rep(x, times = 3) ## [1] 1 3 5 7 8 9 1 3 5 7 8 9 1 3 5 7 8 9 We have now seen four different ways to create vectors: c() : seq() rep() So far we have mostly used them in isolation, but they are often used together. c(x, rep(seq(1, 9, 2), 3), c(1, 2, 3), 42, 2:4) ## [1] 1 3 5 7 8 9 1 3 5 7 9 1 3 5 7 9 1 3 5 7 9 1 2 3 42 ## [26] 2 3 4 The length of a vector can be obtained with the length() function. length(x) ## [1] 6 length(y) ## [1] 100 3.2.1.1 Subsetting To subset a vector, we use square brackets, []. x ## [1] 1 3 5 7 8 9 x[1] ## [1] 1 x[3] ## [1] 5 We see that x[1] returns the first element, and x[3] returns the third element. x[-2] ## [1] 1 5 7 8 9 We can also exclude certain indexes, in this case the second element. x[1:3] ## [1] 1 3 5 x[c(1,3,4)] ## [1] 1 5 7 Lastly we see that we can subset based on a vector of indices. All of the above are subsetting a vector using a vector of indexes. (Remember a single number is still a vector.) We could instead use a vector of logical values. z = c(TRUE, TRUE, FALSE, TRUE, TRUE, FALSE) z ## [1] TRUE TRUE FALSE TRUE TRUE FALSE x[z] ## [1] 1 3 7 8 3.2.2 Vectorization One of the biggest strengths of R is its use of vectorized operations. (Frequently the lack of understanding of this concept leads to a belief that R is slow. R is not the fastest language, but it has a reputation for being slower than it really is.) x = 1:10 x + 1 ## [1] 2 3 4 5 6 7 8 9 10 11 2 * x ## [1] 2 4 6 8 10 12 14 16 18 20 2 ^ x ## [1] 2 4 8 16 32 64 128 256 512 1024 sqrt(x) ## [1] 1.000000 1.414214 1.732051 2.000000 2.236068 2.449490 2.645751 2.828427 ## [9] 3.000000 3.162278 log(x) ## [1] 0.0000000 0.6931472 1.0986123 1.3862944 1.6094379 1.7917595 1.9459101 ## [8] 2.0794415 2.1972246 2.3025851 We see that when a function like log() is called on a vector x, a vector is returned which has applied the function to each element of the vector x. 3.2.3 Logical Operators Operator Summary Example Result x &lt; y x less than y 3 &lt; 42 TRUE x &gt; y x greater than y 3 &gt; 42 FALSE x &lt;= y x less than or equal to y 3 &lt;= 42 TRUE x &gt;= y x greater than or equal to y 3 &gt;= 42 FALSE x == y xequal to y 3 == 42 FALSE x != y x not equal to y 3 != 42 TRUE !x not x !(3 &gt; 42) TRUE x | y x or y (3 &gt; 42) | TRUE TRUE x &amp; y x and y (3 &lt; 4) &amp; ( 42 &gt; 13) TRUE In R, logical operators are vectorized. x = c(1, 3, 5, 7, 8, 9) x &gt; 3 ## [1] FALSE FALSE TRUE TRUE TRUE TRUE x &lt; 3 ## [1] TRUE FALSE FALSE FALSE FALSE FALSE x == 3 ## [1] FALSE TRUE FALSE FALSE FALSE FALSE x != 3 ## [1] TRUE FALSE TRUE TRUE TRUE TRUE x == 3 &amp; x != 3 ## [1] FALSE FALSE FALSE FALSE FALSE FALSE x == 3 | x != 3 ## [1] TRUE TRUE TRUE TRUE TRUE TRUE This is extremely useful for subsetting. x[x &gt; 3] ## [1] 5 7 8 9 x[x != 3] ## [1] 1 5 7 8 9 sum(x &gt; 3) ## [1] 4 as.numeric(x &gt; 3) ## [1] 0 0 1 1 1 1 Here we see that using the sum() function on a vector of logical TRUE and FALSE values that is the result of x &gt; 3 results in a numeric result. R is first automatically coercing the logical to numeric where TRUE is 1 and FALSE is 0. This coercion from logical to numeric happens for most mathematical operations. If you are interested in more detail, check out Advanced R. which(x &gt; 3) ## [1] 3 4 5 6 x[which(x &gt; 3)] ## [1] 5 7 8 9 max(x) ## [1] 9 which(x == max(x)) ## [1] 6 which.max(x) ## [1] 6 3.2.4 More Vectorization x = c(1, 3, 5, 7, 8, 9) y = 1:100 x + 2 ## [1] 3 5 7 9 10 11 x + rep(2, 6) ## [1] 3 5 7 9 10 11 x &gt; 3 ## [1] FALSE FALSE TRUE TRUE TRUE TRUE x &gt; rep(3, 6) ## [1] FALSE FALSE TRUE TRUE TRUE TRUE x + y ## Warning in x + y: longer object length is not a multiple of shorter object ## length ## [1] 2 5 8 11 13 15 8 11 14 17 19 21 14 17 20 23 25 27 ## [19] 20 23 26 29 31 33 26 29 32 35 37 39 32 35 38 41 43 45 ## [37] 38 41 44 47 49 51 44 47 50 53 55 57 50 53 56 59 61 63 ## [55] 56 59 62 65 67 69 62 65 68 71 73 75 68 71 74 77 79 81 ## [73] 74 77 80 83 85 87 80 83 86 89 91 93 86 89 92 95 97 99 ## [91] 92 95 98 101 103 105 98 101 104 107 length(x) ## [1] 6 length(y) ## [1] 100 length(y) / length(x) ## [1] 16.66667 (x + y) - y ## Warning in x + y: longer object length is not a multiple of shorter object ## length ## [1] 1 3 5 7 8 9 1 3 5 7 8 9 1 3 5 7 8 9 1 3 5 7 8 9 1 3 5 7 8 9 1 3 5 7 8 9 1 ## [38] 3 5 7 8 9 1 3 5 7 8 9 1 3 5 7 8 9 1 3 5 7 8 9 1 3 5 7 8 9 1 3 5 7 8 9 1 3 ## [75] 5 7 8 9 1 3 5 7 8 9 1 3 5 7 8 9 1 3 5 7 8 9 1 3 5 7 y = 1:60 x + y ## [1] 2 5 8 11 13 15 8 11 14 17 19 21 14 17 20 23 25 27 20 23 26 29 31 33 26 ## [26] 29 32 35 37 39 32 35 38 41 43 45 38 41 44 47 49 51 44 47 50 53 55 57 50 53 ## [51] 56 59 61 63 56 59 62 65 67 69 length(y) / length(x) ## [1] 10 rep(x, 10) + y ## [1] 2 5 8 11 13 15 8 11 14 17 19 21 14 17 20 23 25 27 20 23 26 29 31 33 26 ## [26] 29 32 35 37 39 32 35 38 41 43 45 38 41 44 47 49 51 44 47 50 53 55 57 50 53 ## [51] 56 59 61 63 56 59 62 65 67 69 all(x + y == rep(x, 10) + y) ## [1] TRUE identical(x + y, rep(x, 10) + y) ## [1] TRUE # ?any # ?all.equal x = c(1, 3, 5) y = c(1, 2, 4) x == y ## [1] TRUE FALSE FALSE all(x == y) ## [1] FALSE any(x == y) ## [1] TRUE While all returns TRUE only when all of its arguments are TRUE, any returns TRUE when at least one of its arguments is TRUE. x = c(10 ^ (-8)) y = c(10 ^ (-9)) all(x == y) ## [1] FALSE all.equal(x, y) ## [1] TRUE The all.equal function tests “near equality” with a default tolerance value around 1.5e-8 and returns TRUE if all of its arguments have differences smaller than the tolerance. 3.2.5 Matrices R can also be used for matrix calculations. Matrices have rows and columns containing a single data type. In a matrix, the order of rows and columns is important. (This is not true of data frames, which we will see later.) Matrices can be created using the matrix function. x = 1:9 x ## [1] 1 2 3 4 5 6 7 8 9 X = matrix(x, nrow = 3, ncol = 3) X ## [,1] [,2] [,3] ## [1,] 1 4 7 ## [2,] 2 5 8 ## [3,] 3 6 9 Note here that we are using two different variables: lower case x, which stores a vector and capital X, which stores a matrix. (Following the usual mathematical convention.) We can do this because R is case sensitive. By default the matrix function reorders a vector into columns, but we can also tell R to use rows instead. Y = matrix(x, nrow = 3, ncol = 3, byrow = TRUE) Y ## [,1] [,2] [,3] ## [1,] 1 2 3 ## [2,] 4 5 6 ## [3,] 7 8 9 We can also create a matrix of a specified dimension where every element is the same, in this case 0. Z = matrix(0, 2, 4) Z ## [,1] [,2] [,3] [,4] ## [1,] 0 0 0 0 ## [2,] 0 0 0 0 Like vectors, matrices can be subsetted using square brackets, []. However, since matrices are two-dimensional, we need to specify both a row and a column when subsetting. X ## [,1] [,2] [,3] ## [1,] 1 4 7 ## [2,] 2 5 8 ## [3,] 3 6 9 X[1, 2] ## [1] 4 Here we accessed the element in the first row and the second column. We could also subset an entire row or column. X[1, ] ## [1] 1 4 7 X[, 2] ## [1] 4 5 6 We can also use vectors to subset more than one row or column at a time. Here we subset to the first and third column of the second row. X[2, c(1, 3)] ## [1] 2 8 Matrices can also be created by combining vectors as columns, using cbind, or combining vectors as rows, using rbind. x = 1:9 rev(x) ## [1] 9 8 7 6 5 4 3 2 1 rep(1, 9) ## [1] 1 1 1 1 1 1 1 1 1 rbind(x, rev(x), rep(1, 9)) ## [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] ## x 1 2 3 4 5 6 7 8 9 ## 9 8 7 6 5 4 3 2 1 ## 1 1 1 1 1 1 1 1 1 cbind(col_1 = x, col_2 = rev(x), col_3 = rep(1, 9)) ## col_1 col_2 col_3 ## [1,] 1 9 1 ## [2,] 2 8 1 ## [3,] 3 7 1 ## [4,] 4 6 1 ## [5,] 5 5 1 ## [6,] 6 4 1 ## [7,] 7 3 1 ## [8,] 8 2 1 ## [9,] 9 1 1 When using rbind and cbind you can specify “argument” names that will be used as column names. R can then be used to perform matrix calculations. x = 1:9 y = 9:1 X = matrix(x, 3, 3) Y = matrix(y, 3, 3) X ## [,1] [,2] [,3] ## [1,] 1 4 7 ## [2,] 2 5 8 ## [3,] 3 6 9 Y ## [,1] [,2] [,3] ## [1,] 9 6 3 ## [2,] 8 5 2 ## [3,] 7 4 1 X + Y ## [,1] [,2] [,3] ## [1,] 10 10 10 ## [2,] 10 10 10 ## [3,] 10 10 10 X - Y ## [,1] [,2] [,3] ## [1,] -8 -2 4 ## [2,] -6 0 6 ## [3,] -4 2 8 X * Y ## [,1] [,2] [,3] ## [1,] 9 24 21 ## [2,] 16 25 16 ## [3,] 21 24 9 X / Y ## [,1] [,2] [,3] ## [1,] 0.1111111 0.6666667 2.333333 ## [2,] 0.2500000 1.0000000 4.000000 ## [3,] 0.4285714 1.5000000 9.000000 Note that X * Y is not matrix multiplication. It is element by element multiplication. (Same for X / Y). Instead, matrix multiplication uses %*%. Other matrix functions include t() which gives the transpose of a matrix and solve() which returns the inverse of a square matrix if it is invertible. X %*% Y ## [,1] [,2] [,3] ## [1,] 90 54 18 ## [2,] 114 69 24 ## [3,] 138 84 30 t(X) ## [,1] [,2] [,3] ## [1,] 1 2 3 ## [2,] 4 5 6 ## [3,] 7 8 9 Z = matrix(c(9, 2, -3, 2, 4, -2, -3, -2, 16), 3, byrow = TRUE) Z ## [,1] [,2] [,3] ## [1,] 9 2 -3 ## [2,] 2 4 -2 ## [3,] -3 -2 16 solve(Z) ## [,1] [,2] [,3] ## [1,] 0.12931034 -0.05603448 0.01724138 ## [2,] -0.05603448 0.29094828 0.02586207 ## [3,] 0.01724138 0.02586207 0.06896552 To verify that solve(Z) returns the inverse, we multiply it by Z. We would expect this to return the identity matrix, however we see that this is not the case due to some computational issues. However, R also has the all.equal() function which checks for equality, with some small tolerance which accounts for some computational issues. The identical() function is used to check for exact equality. solve(Z) %*% Z ## [,1] [,2] [,3] ## [1,] 1.000000e+00 -6.245005e-17 0.000000e+00 ## [2,] 7.979728e-17 1.000000e+00 5.551115e-17 ## [3,] 2.775558e-17 0.000000e+00 1.000000e+00 diag(3) ## [,1] [,2] [,3] ## [1,] 1 0 0 ## [2,] 0 1 0 ## [3,] 0 0 1 all.equal(solve(Z) %*% Z, diag(3)) ## [1] TRUE R has a number of matrix specific functions for obtaining dimension and summary information. X = matrix(1:6, 2, 3) X ## [,1] [,2] [,3] ## [1,] 1 3 5 ## [2,] 2 4 6 dim(X) ## [1] 2 3 rowSums(X) ## [1] 9 12 colSums(X) ## [1] 3 7 11 rowMeans(X) ## [1] 3 4 colMeans(X) ## [1] 1.5 3.5 5.5 The diag() function can be used in a number of ways. We can extract the diagonal of a matrix. diag(Z) ## [1] 9 4 16 Or create a matrix with specified elements on the diagonal. (And 0 on the off-diagonals.) diag(1:5) ## [,1] [,2] [,3] [,4] [,5] ## [1,] 1 0 0 0 0 ## [2,] 0 2 0 0 0 ## [3,] 0 0 3 0 0 ## [4,] 0 0 0 4 0 ## [5,] 0 0 0 0 5 Or, lastly, create a square matrix of a certain dimension with 1 for every element of the diagonal and 0 for the off-diagonals. diag(5) ## [,1] [,2] [,3] [,4] [,5] ## [1,] 1 0 0 0 0 ## [2,] 0 1 0 0 0 ## [3,] 0 0 1 0 0 ## [4,] 0 0 0 1 0 ## [5,] 0 0 0 0 1 Calculations with Vectors and Matrices Certain operations in R, for example %*% have different behavior on vectors and matrices. To illustrate this, we will first create two vectors. a_vec = c(1, 2, 3) b_vec = c(2, 2, 2) Note that these are indeed vectors. They are not matrices. c(is.vector(a_vec), is.vector(b_vec)) ## [1] TRUE TRUE c(is.matrix(a_vec), is.matrix(b_vec)) ## [1] FALSE FALSE When this is the case, the %*% operator is used to calculate the dot product, also known as the inner product of the two vectors. The dot product of vectors \\(\\boldsymbol{a} = \\lbrack a_1, a_2, \\cdots a_n \\rbrack\\) and \\(\\boldsymbol{b} = \\lbrack b_1, b_2, \\cdots b_n \\rbrack\\) is defined to be \\[ \\boldsymbol{a} \\cdot \\boldsymbol{b} = \\sum_{i = 1}^{n} a_i b_i = a_1 b_1 + a_2 b_2 + \\cdots a_n b_n. \\] a_vec %*% b_vec # inner product ## [,1] ## [1,] 12 a_vec %o% b_vec # outer product ## [,1] [,2] [,3] ## [1,] 2 2 2 ## [2,] 4 4 4 ## [3,] 6 6 6 The %o% operator is used to calculate the outer product of the two vectors. When vectors are coerced to become matrices, they are column vectors. So a vector of length \\(n\\) becomes an \\(n \\times 1\\) matrix after coercion. as.matrix(a_vec) ## [,1] ## [1,] 1 ## [2,] 2 ## [3,] 3 If we use the %*% operator on matrices, %*% again performs the expected matrix multiplication. So you might expect the following to produce an error, because the dimensions are incorrect. as.matrix(a_vec) %*% b_vec ## [,1] [,2] [,3] ## [1,] 2 2 2 ## [2,] 4 4 4 ## [3,] 6 6 6 At face value this is a \\(3 \\times 1\\) matrix, multiplied by a \\(3 \\times 1\\) matrix. However, when b_vec is automatically coerced to be a matrix, R decided to make it a “row vector,” a \\(1 \\times 3\\) matrix, so that the multiplication has conformable dimensions. If we had coerced both, then R would produce an error. as.matrix(a_vec) %*% as.matrix(b_vec) Another way to calculate a dot product is with the crossprod() function. Given two vectors, the crossprod() function calculates their dot product. The function has a rather misleading name. crossprod(a_vec, b_vec) # inner product ## [,1] ## [1,] 12 tcrossprod(a_vec, b_vec) # outer product ## [,1] [,2] [,3] ## [1,] 2 2 2 ## [2,] 4 4 4 ## [3,] 6 6 6 These functions could be very useful later. When used with matrices \\(X\\) and \\(Y\\) as arguments, it calculates \\[ X^\\top Y. \\] When dealing with linear models, the calculation \\[ X^\\top X \\] is used repeatedly. C_mat = matrix(c(1, 2, 3, 4, 5, 6), 2, 3) D_mat = matrix(c(2, 2, 2, 2, 2, 2), 2, 3) This is useful both as a shortcut for a frequent calculation and as a more efficient implementation than using t() and %*%. crossprod(C_mat, D_mat) ## [,1] [,2] [,3] ## [1,] 6 6 6 ## [2,] 14 14 14 ## [3,] 22 22 22 t(C_mat) %*% D_mat ## [,1] [,2] [,3] ## [1,] 6 6 6 ## [2,] 14 14 14 ## [3,] 22 22 22 all.equal(crossprod(C_mat, D_mat), t(C_mat) %*% D_mat) ## [1] TRUE crossprod(C_mat, C_mat) ## [,1] [,2] [,3] ## [1,] 5 11 17 ## [2,] 11 25 39 ## [3,] 17 39 61 t(C_mat) %*% C_mat ## [,1] [,2] [,3] ## [1,] 5 11 17 ## [2,] 11 25 39 ## [3,] 17 39 61 all.equal(crossprod(C_mat, C_mat), t(C_mat) %*% C_mat) ## [1] TRUE 3.2.6 Lists A list is a one-dimensional heterogeneous data structure. So it is indexed like a vector with a single integer value, but each element can contain an element of any type. # creation list(42, &quot;Hello&quot;, TRUE) ## [[1]] ## [1] 42 ## ## [[2]] ## [1] &quot;Hello&quot; ## ## [[3]] ## [1] TRUE ex_list = list( a = c(1, 2, 3, 4), b = TRUE, c = &quot;Hello!&quot;, d = function(arg = 42) {print(&quot;Hello World!&quot;)}, e = diag(5) ) Lists can be subset using two syntaxes, the $ operator, and square brackets []. The $ operator returns a named element of a list. The [] syntax returns a list, while the [[]] returns an element of a list. ex_list[1] returns a list containing the first element. ex_list[[1]] returns the first element of the list, in this case, a vector. # subsetting ex_list$e ## [,1] [,2] [,3] [,4] [,5] ## [1,] 1 0 0 0 0 ## [2,] 0 1 0 0 0 ## [3,] 0 0 1 0 0 ## [4,] 0 0 0 1 0 ## [5,] 0 0 0 0 1 ex_list[1:2] ## $a ## [1] 1 2 3 4 ## ## $b ## [1] TRUE ex_list[1] ## $a ## [1] 1 2 3 4 ex_list[[1]] ## [1] 1 2 3 4 ex_list[c(&quot;e&quot;, &quot;a&quot;)] ## $e ## [,1] [,2] [,3] [,4] [,5] ## [1,] 1 0 0 0 0 ## [2,] 0 1 0 0 0 ## [3,] 0 0 1 0 0 ## [4,] 0 0 0 1 0 ## [5,] 0 0 0 0 1 ## ## $a ## [1] 1 2 3 4 ex_list[&quot;e&quot;] ## $e ## [,1] [,2] [,3] [,4] [,5] ## [1,] 1 0 0 0 0 ## [2,] 0 1 0 0 0 ## [3,] 0 0 1 0 0 ## [4,] 0 0 0 1 0 ## [5,] 0 0 0 0 1 ex_list[[&quot;e&quot;]] ## [,1] [,2] [,3] [,4] [,5] ## [1,] 1 0 0 0 0 ## [2,] 0 1 0 0 0 ## [3,] 0 0 1 0 0 ## [4,] 0 0 0 1 0 ## [5,] 0 0 0 0 1 ex_list$d ## function(arg = 42) {print(&quot;Hello World!&quot;)} ex_list$d(arg = 1) ## [1] &quot;Hello World!&quot; 3.2.7 Data Frames We have previously seen vectors and matrices for storing data as we introduced R. We will now introduce a data frame which will be the most common way that we store and interact with data in this course. example_data = data.frame(x = c(1, 3, 5, 7, 9, 1, 3, 5, 7, 9), y = c(rep(&quot;Hello&quot;, 9), &quot;Goodbye&quot;), z = rep(c(TRUE, FALSE), 5)) Unlike a matrix, which can be thought of as a vector rearranged into rows and columns, a data frame is not required to have the same data type for each element. A data frame is a list of vectors. So, each vector must contain the same data type, but the different vectors can store different data types. example_data ## x y z ## 1 1 Hello TRUE ## 2 3 Hello FALSE ## 3 5 Hello TRUE ## 4 7 Hello FALSE ## 5 9 Hello TRUE ## 6 1 Hello FALSE ## 7 3 Hello TRUE ## 8 5 Hello FALSE ## 9 7 Hello TRUE ## 10 9 Goodbye FALSE Unlike a list which has more flexibility, the elements of a data frame must all be vectors, and have the same length. example_data$x ## [1] 1 3 5 7 9 1 3 5 7 9 all.equal(length(example_data$x), length(example_data$y), length(example_data$z)) ## [1] TRUE str(example_data) ## &#39;data.frame&#39;: 10 obs. of 3 variables: ## $ x: num 1 3 5 7 9 1 3 5 7 9 ## $ y: chr &quot;Hello&quot; &quot;Hello&quot; &quot;Hello&quot; &quot;Hello&quot; ... ## $ z: logi TRUE FALSE TRUE FALSE TRUE FALSE ... nrow(example_data) ## [1] 10 ncol(example_data) ## [1] 3 dim(example_data) ## [1] 10 3 The data.frame() function above is one way to create a data frame. We can also import data from various file types into R, as well as use data stored in packages. The example data above can also be found here as a .csv file. To read this data into R, we would use the read_csv() function from the readr package. Note that R has a built-in function read.csv() that operates very similarly. The readr function read_csv() has a number of advantages. For example, it is much faster reading larger data. It also uses the tibble package to read the data as a tibble. library(readr) example_data_from_csv = read_csv(&quot;data/example-data.csv&quot;) This particular line of code assumes that the file example_data.csv exists in a folder called data in your current working directory. example_data_from_csv ## # A tibble: 10 × 3 ## x y z ## &lt;dbl&gt; &lt;chr&gt; &lt;lgl&gt; ## 1 1 Hello TRUE ## 2 3 Hello FALSE ## 3 5 Hello TRUE ## 4 7 Hello FALSE ## 5 9 Hello TRUE ## 6 1 Hello FALSE ## 7 3 Hello TRUE ## 8 5 Hello FALSE ## 9 7 Hello TRUE ## 10 9 Goodbye FALSE A tibble is simply a data frame that prints with sanity. Notice in the output above that we are given additional information such as dimension and variable type. The as_tibble() function can be used to coerce a regular data frame to a tibble. library(tibble) example_data = as_tibble(example_data) example_data ## # A tibble: 10 × 3 ## x y z ## &lt;dbl&gt; &lt;chr&gt; &lt;lgl&gt; ## 1 1 Hello TRUE ## 2 3 Hello FALSE ## 3 5 Hello TRUE ## 4 7 Hello FALSE ## 5 9 Hello TRUE ## 6 1 Hello FALSE ## 7 3 Hello TRUE ## 8 5 Hello FALSE ## 9 7 Hello TRUE ## 10 9 Goodbye FALSE Alternatively, we could use the “Import Dataset” feature in RStudio which can be found in the environment window. (By default, the top-right pane of RStudio.) Once completed, this process will automatically generate the code to import a file. The resulting code will be shown in the console window. In recent versions of RStudio, read_csv() is used by default, thus reading in a tibble. Earlier we looked at installing packages, in particular the ggplot2 package. (A package for visualization. While not necessary for this course, it is quickly growing in popularity.) library(ggplot2) Inside the ggplot2 package is a dataset called mpg. By loading the package using the library() function, we can now access mpg. When using data from inside a package, there are three things we would generally like to do: Look at the raw data. Understand the data. (Where did it come from? What are the variables? Etc.) Visualize the data. To look at the data, we have two useful commands: head() and str(). head(mpg, n = 10) ## # A tibble: 10 × 11 ## manufacturer model displ year cyl trans drv cty hwy fl class ## &lt;chr&gt; &lt;chr&gt; &lt;dbl&gt; &lt;int&gt; &lt;int&gt; &lt;chr&gt; &lt;chr&gt; &lt;int&gt; &lt;int&gt; &lt;chr&gt; &lt;chr&gt; ## 1 audi a4 1.8 1999 4 auto… f 18 29 p comp… ## 2 audi a4 1.8 1999 4 manu… f 21 29 p comp… ## 3 audi a4 2 2008 4 manu… f 20 31 p comp… ## 4 audi a4 2 2008 4 auto… f 21 30 p comp… ## 5 audi a4 2.8 1999 6 auto… f 16 26 p comp… ## 6 audi a4 2.8 1999 6 manu… f 18 26 p comp… ## 7 audi a4 3.1 2008 6 auto… f 18 27 p comp… ## 8 audi a4 quattro 1.8 1999 4 manu… 4 18 26 p comp… ## 9 audi a4 quattro 1.8 1999 4 auto… 4 16 25 p comp… ## 10 audi a4 quattro 2 2008 4 manu… 4 20 28 p comp… The function head() will display the first n observations of the data frame. The head() function was more useful before tibbles. Notice that mpg is a tibble already, so the output from head() indicates there are only 10 observations. Note that this applies to head(mpg, n = 10) and not mpg itself. Also note that tibbles print a limited number of rows and columns by default. The last line of the printed output indicates which rows and columns were omitted. mpg ## # A tibble: 234 × 11 ## manufacturer model displ year cyl trans drv cty hwy fl class ## &lt;chr&gt; &lt;chr&gt; &lt;dbl&gt; &lt;int&gt; &lt;int&gt; &lt;chr&gt; &lt;chr&gt; &lt;int&gt; &lt;int&gt; &lt;chr&gt; &lt;chr&gt; ## 1 audi a4 1.8 1999 4 auto… f 18 29 p comp… ## 2 audi a4 1.8 1999 4 manu… f 21 29 p comp… ## 3 audi a4 2 2008 4 manu… f 20 31 p comp… ## 4 audi a4 2 2008 4 auto… f 21 30 p comp… ## 5 audi a4 2.8 1999 6 auto… f 16 26 p comp… ## 6 audi a4 2.8 1999 6 manu… f 18 26 p comp… ## 7 audi a4 3.1 2008 6 auto… f 18 27 p comp… ## 8 audi a4 quattro 1.8 1999 4 manu… 4 18 26 p comp… ## 9 audi a4 quattro 1.8 1999 4 auto… 4 16 25 p comp… ## 10 audi a4 quattro 2 2008 4 manu… 4 20 28 p comp… ## # ℹ 224 more rows The function str() will display the “structure” of the data frame. It will display the number of observations and variables, list the variables, give the type of each variable, and show some elements of each variable. This information can also be found in the “Environment” window in RStudio. str(mpg) ## tibble [234 × 11] (S3: tbl_df/tbl/data.frame) ## $ manufacturer: chr [1:234] &quot;audi&quot; &quot;audi&quot; &quot;audi&quot; &quot;audi&quot; ... ## $ model : chr [1:234] &quot;a4&quot; &quot;a4&quot; &quot;a4&quot; &quot;a4&quot; ... ## $ displ : num [1:234] 1.8 1.8 2 2 2.8 2.8 3.1 1.8 1.8 2 ... ## $ year : int [1:234] 1999 1999 2008 2008 1999 1999 2008 1999 1999 2008 ... ## $ cyl : int [1:234] 4 4 4 4 6 6 6 4 4 4 ... ## $ trans : chr [1:234] &quot;auto(l5)&quot; &quot;manual(m5)&quot; &quot;manual(m6)&quot; &quot;auto(av)&quot; ... ## $ drv : chr [1:234] &quot;f&quot; &quot;f&quot; &quot;f&quot; &quot;f&quot; ... ## $ cty : int [1:234] 18 21 20 21 16 18 18 18 16 20 ... ## $ hwy : int [1:234] 29 29 31 30 26 26 27 26 25 28 ... ## $ fl : chr [1:234] &quot;p&quot; &quot;p&quot; &quot;p&quot; &quot;p&quot; ... ## $ class : chr [1:234] &quot;compact&quot; &quot;compact&quot; &quot;compact&quot; &quot;compact&quot; ... It is important to note that while matrices have rows and columns, data frames (tibbles) instead have observations and variables. When displayed in the console or viewer, each row is an observation and each column is a variable. However generally speaking, their order does not matter, it is simply a side-effect of how the data was entered or stored. In this dataset an observation is for a particular model-year of a car, and the variables describe attributes of the car, for example its highway fuel efficiency. To understand more about the data set, we use the ? operator to pull up the documentation for the data. ?mpg R has a number of functions for quickly working with and extracting basic information from data frames. To quickly obtain a vector of the variable names, we use the names() function. names(mpg) ## [1] &quot;manufacturer&quot; &quot;model&quot; &quot;displ&quot; &quot;year&quot; &quot;cyl&quot; ## [6] &quot;trans&quot; &quot;drv&quot; &quot;cty&quot; &quot;hwy&quot; &quot;fl&quot; ## [11] &quot;class&quot; To access one of the variables as a vector, we use the $ operator. mpg$year ## [1] 1999 1999 2008 2008 1999 1999 2008 1999 1999 2008 2008 1999 1999 2008 2008 ## [16] 1999 2008 2008 2008 2008 2008 1999 2008 1999 1999 2008 2008 2008 2008 2008 ## [31] 1999 1999 1999 2008 1999 2008 2008 1999 1999 1999 1999 2008 2008 2008 1999 ## [46] 1999 2008 2008 2008 2008 1999 1999 2008 2008 2008 1999 1999 1999 2008 2008 ## [61] 2008 1999 2008 1999 2008 2008 2008 2008 2008 2008 1999 1999 2008 1999 1999 ## [76] 1999 2008 1999 1999 1999 2008 2008 1999 1999 1999 1999 1999 2008 1999 2008 ## [91] 1999 1999 2008 2008 1999 1999 2008 2008 2008 1999 1999 1999 1999 1999 2008 ## [106] 2008 2008 2008 1999 1999 2008 2008 1999 1999 2008 1999 1999 2008 2008 2008 ## [121] 2008 2008 2008 2008 1999 1999 2008 2008 2008 2008 1999 2008 2008 1999 1999 ## [136] 1999 2008 1999 2008 2008 1999 1999 1999 2008 2008 2008 2008 1999 1999 2008 ## [151] 1999 1999 2008 2008 1999 1999 1999 2008 2008 1999 1999 2008 2008 2008 2008 ## [166] 1999 1999 1999 1999 2008 2008 2008 2008 1999 1999 1999 1999 2008 2008 1999 ## [181] 1999 2008 2008 1999 1999 2008 1999 1999 2008 2008 1999 1999 2008 1999 1999 ## [196] 1999 2008 2008 1999 2008 1999 1999 2008 1999 1999 2008 2008 1999 1999 2008 ## [211] 2008 1999 1999 1999 1999 2008 2008 2008 2008 1999 1999 1999 1999 1999 1999 ## [226] 2008 2008 1999 1999 2008 2008 1999 1999 2008 mpg$hwy ## [1] 29 29 31 30 26 26 27 26 25 28 27 25 25 25 25 24 25 23 20 15 20 17 17 26 23 ## [26] 26 25 24 19 14 15 17 27 30 26 29 26 24 24 22 22 24 24 17 22 21 23 23 19 18 ## [51] 17 17 19 19 12 17 15 17 17 12 17 16 18 15 16 12 17 17 16 12 15 16 17 15 17 ## [76] 17 18 17 19 17 19 19 17 17 17 16 16 17 15 17 26 25 26 24 21 22 23 22 20 33 ## [101] 32 32 29 32 34 36 36 29 26 27 30 31 26 26 28 26 29 28 27 24 24 24 22 19 20 ## [126] 17 12 19 18 14 15 18 18 15 17 16 18 17 19 19 17 29 27 31 32 27 26 26 25 25 ## [151] 17 17 20 18 26 26 27 28 25 25 24 27 25 26 23 26 26 26 26 25 27 25 27 20 20 ## [176] 19 17 20 17 29 27 31 31 26 26 28 27 29 31 31 26 26 27 30 33 35 37 35 15 18 ## [201] 20 20 22 17 19 18 20 29 26 29 29 24 44 29 26 29 29 29 29 23 24 44 41 29 26 ## [226] 28 29 29 29 28 29 26 26 26 We can use the dim(), nrow() and ncol() functions to obtain information about the dimension of the data frame. dim(mpg) ## [1] 234 11 nrow(mpg) ## [1] 234 ncol(mpg) ## [1] 11 Here nrow() is also the number of observations, which in most cases is the sample size. Subsetting data frames can work much like subsetting matrices using square brackets, [,]. Here, we find fuel efficient vehicles earning over 35 miles per gallon and only display manufacturer, model and year. mpg[mpg$hwy &gt; 35, c(&quot;manufacturer&quot;, &quot;model&quot;, &quot;year&quot;)] ## # A tibble: 6 × 3 ## manufacturer model year ## &lt;chr&gt; &lt;chr&gt; &lt;int&gt; ## 1 honda civic 2008 ## 2 honda civic 2008 ## 3 toyota corolla 2008 ## 4 volkswagen jetta 1999 ## 5 volkswagen new beetle 1999 ## 6 volkswagen new beetle 1999 An alternative would be to use the subset() function, which has a much more readable syntax. subset(mpg, subset = hwy &gt; 35, select = c(&quot;manufacturer&quot;, &quot;model&quot;, &quot;year&quot;)) Lastly, we could use the filter and select functions from the dplyr package which introduces the %&gt;% operator from the magrittr package. This is not necessary for this course, however the dplyr package is something you should be aware of as it is becoming a popular tool in the R world. library(dplyr) mpg %&gt;% filter(hwy &gt; 35) %&gt;% select(manufacturer, model, year) All three approaches produce the same results. Which you use will be largely based on a given situation as well as user preference. When subsetting a data frame, be aware of what is being returned, as sometimes it may be a vector instead of a data frame. Also note that there are differences between subsetting a data frame and a tibble. A data frame operates more like a matrix where it is possible to reduce the subset to a vector. A tibble operates more like a list where it always subsets to another tibble. 3.3 Programming Basics 3.3.1 Control Flow In R, the if/else syntax is: if (...) { some R code } else { more R code } For example, x = 1 y = 3 if (x &gt; y) { z = x * y print(&quot;x is larger than y&quot;) } else { z = x + 5 * y print(&quot;x is less than or equal to y&quot;) } ## [1] &quot;x is less than or equal to y&quot; z ## [1] 16 R also has a special function ifelse() which is very useful. It returns one of two specified values based on a conditional statement. ifelse(4 &gt; 3, 1, 0) ## [1] 1 The real power of ifelse() comes from its ability to be applied to vectors. fib = c(1, 1, 2, 3, 5, 8, 13, 21) ifelse(fib &gt; 6, &quot;Foo&quot;, &quot;Bar&quot;) ## [1] &quot;Bar&quot; &quot;Bar&quot; &quot;Bar&quot; &quot;Bar&quot; &quot;Bar&quot; &quot;Foo&quot; &quot;Foo&quot; &quot;Foo&quot; Now a for loop example, x = 11:15 for (i in 1:5) { x[i] = x[i] * 2 } x ## [1] 22 24 26 28 30 Note that this for loop is very normal in many programming languages, but not in R. In R we would not use a loop, instead we would simply use a vectorized operation. x = 11:15 x = x * 2 x ## [1] 22 24 26 28 30 3.3.2 Functions So far we have been using functions, but haven’t actually discussed some of their details. function_name(arg1 = 10, arg2 = 20) To use a function, you simply type its name, followed by an open parenthesis, then specify values of its arguments, then finish with a closing parenthesis. An argument is a variable which is used in the body of the function. Specifying the values of the arguments is essentially providing the inputs to the function. We can also write our own functions in R. For example, we often like to “standardize” variables, that is, subtracting the sample mean, and dividing by the sample standard deviation. \\[ \\frac{x - \\bar{x}}{s} \\] In R we would write a function to do this. When writing a function, there are three things you must do. Give the function a name. Preferably something that is short, but descriptive. Specify the arguments using function(). Write the body of the function within curly braces, {}. standardize = function(x) { m = mean(x) std = sd(x) result = (x - m) / std result } Here the name of the function is standardize, and the function has a single argument x which is used in the body of function. Note that the output of the final line of the body is what is returned by the function. In this case the function returns the vector stored in the variable result. To test our function, we will take a random sample of size n = 10 from a normal distribution with a mean of 2 and a standard deviation of 5. (test_sample = rnorm(n = 10, mean = 2, sd = 5)) ## [1] 3.4663306 -1.8992181 -3.3143635 -3.4093843 5.3309762 0.7177388 ## [7] 2.0023602 0.9305203 0.1863598 2.1499918 standardize(x = test_sample) ## [1] 1.00144306 -0.88379048 -1.38101447 -1.41440089 1.65660299 0.03570074 ## [7] 0.48706398 0.11046344 -0.15100402 0.53893564 This function could be written much more succinctly, simply performing all the operations on one line and immediately returning the result, without storing any of the intermediate results. standardize = function(x) { (x - mean(x)) / sd(x) } When specifying arguments, you can provide default arguments. power_of_num = function(num, power = 2) { num ^ power } Let’s look at a number of ways that we could run this function to perform the operation 10^2 resulting in 100. power_of_num(10) ## [1] 100 power_of_num(10, 2) ## [1] 100 power_of_num(num = 10, power = 2) ## [1] 100 power_of_num(power = 2, num = 10) ## [1] 100 Note that without using the argument names, the order matters. The following code will not evaluate to the same output as the previous example. power_of_num(2, 10) ## [1] 1024 Also, the following line of code would produce an error since arguments without a default value must be specified. power_of_num(power = 5) To further illustrate a function with a default argument, we will write a function that calculates sample variance two ways. By default, it will calculate the unbiased estimate of \\(\\sigma^2\\), which we will call \\(s^2\\). \\[ s^2 = \\frac{1}{n - 1}\\sum_{i=1}^{n}(x - \\bar{x})^2 \\] It will also have the ability to return the biased estimate (based on maximum likelihood) which we will call \\(\\hat{\\sigma}^2\\). \\[ \\hat{\\sigma}^2 = \\frac{1}{n}\\sum_{i=1}^{n}(x - \\bar{x})^2 \\] get_var = function(x, biased = FALSE) { n = length(x) - 1 * !biased (1 / n) * sum((x - mean(x)) ^ 2) } get_var(test_sample) ## [1] 8.100242 get_var(test_sample, biased = FALSE) ## [1] 8.100242 var(test_sample) ## [1] 8.100242 We see the function is working as expected, and when returning the unbiased estimate it matches R’s built-in function var(). Finally, let’s examine the biased estimate of \\(\\sigma^2\\). get_var(test_sample, biased = TRUE) ## [1] 7.290217 "],["summarizing-data.html", "Chapter 4 Summarizing Data 4.1 Summary Statistics 4.2 Plotting", " Chapter 4 Summarizing Data 4.1 Summary Statistics R has built-in functions for a large number of summary statistics. For numeric variables, we can summarize data with the center and spread. We’ll again look at the mpg dataset from the ggplot2 package. library(ggplot2) Central Tendency Measure R Result Mean mean(mpg$cty) 16.8589744 Median median(mpg$cty) 17 Spread Measure R Result Variance var(mpg$cty) 18.1130736 Standard Deviation sd(mpg$cty) 4.2559457 IQR IQR(mpg$cty) 5 Minimum min(mpg$cty) 9 Maximum max(mpg$cty) 35 Range range(mpg$cty) 9, 35 Categorical For categorical variables, counts and percentages can be used for summary. table(mpg$drv) ## ## 4 f r ## 103 106 25 table(mpg$drv) / nrow(mpg) ## ## 4 f r ## 0.4401709 0.4529915 0.1068376 4.2 Plotting Now that we have some data to work with, and we have learned about the data at the most basic level, our next task is to visualize the data. Often, a proper visualization can illuminate features of the data that can inform further analysis. We will look at four methods of visualizing data that we will use throughout the course: Histograms Barplots Boxplots Scatterplots 4.2.1 Histograms When visualizing a single numerical variable, a histogram will be our go-to tool, which can be created in R using the hist() function. hist(mpg$cty) The histogram function has a number of parameters which can be changed to make our plot look much nicer. Use the ? operator to read the documentation for the hist() to see a full list of these parameters. hist(mpg$cty, xlab = &quot;Miles Per Gallon (City)&quot;, main = &quot;Histogram of MPG (City)&quot;, breaks = 12, col = &quot;dodgerblue&quot;, border = &quot;darkorange&quot;) Importantly, you should always be sure to label your axes and give the plot a title. The argument breaks is specific to hist(). Entering an integer will give a suggestion to R for how many bars to use for the histogram. By default R will attempt to intelligently guess a good number of breaks, but as we can see here, it is sometimes useful to modify this yourself. 4.2.2 Barplots Somewhat similar to a histogram, a barplot can provide a visual summary of a categorical variable, or a numeric variable with a finite number of values, like a ranking from 1 to 10. barplot(table(mpg$drv)) barplot(table(mpg$drv), xlab = &quot;Drivetrain (f = FWD, r = RWD, 4 = 4WD)&quot;, ylab = &quot;Frequency&quot;, main = &quot;Drivetrains&quot;, col = &quot;dodgerblue&quot;, border = &quot;darkorange&quot;) 4.2.3 Boxplots To visualize the relationship between a numerical and categorical variable, we will use a boxplot. In the mpg dataset, the drv variable takes a small, finite number of values. A car can only be front wheel drive, 4 wheel drive, or rear wheel drive. unique(mpg$drv) ## [1] &quot;f&quot; &quot;4&quot; &quot;r&quot; First note that we can use a single boxplot as an alternative to a histogram for visualizing a single numerical variable. To do so in R, we use the boxplot() function. boxplot(mpg$hwy) However, more often we will use boxplots to compare a numerical variable for different values of a categorical variable. boxplot(hwy ~ drv, data = mpg) Here we used the boxplot() command to create side-by-side boxplots. However, since we are now dealing with two variables, the syntax has changed. The R syntax hwy ~ drv, data = mpg reads “Plot the hwy variable against the drv variable using the dataset mpg.” We see the use of a ~ (which specifies a formula) and also a data = argument. This will be a syntax that is common to many functions we will use in this course. boxplot(hwy ~ drv, data = mpg, xlab = &quot;Drivetrain (f = FWD, r = RWD, 4 = 4WD)&quot;, ylab = &quot;Miles Per Gallon (Highway)&quot;, main = &quot;MPG (Highway) vs Drivetrain&quot;, pch = 20, cex = 2, col = &quot;darkorange&quot;, border = &quot;dodgerblue&quot;) Again, boxplot() has a number of additional arguments which have the ability to make our plot more visually appealing. 4.2.4 Scatterplots Lastly, to visualize the relationship between two numeric variables we will use a scatterplot. This can be done with the plot() function and the ~ syntax we just used with a boxplot. (The function plot() can also be used more generally; see the documentation for details.) plot(hwy ~ displ, data = mpg) plot(hwy ~ displ, data = mpg, xlab = &quot;Engine Displacement (in Liters)&quot;, ylab = &quot;Miles Per Gallon (Highway)&quot;, main = &quot;MPG (Highway) vs Engine Displacement&quot;, pch = 20, cex = 2, col = &quot;dodgerblue&quot;) "],["probability-and-statistics-in-r.html", "Chapter 5 Probability and Statistics in R 5.1 Probability in R 5.2 Hypothesis Tests in R 5.3 Simulation", " Chapter 5 Probability and Statistics in R 5.1 Probability in R 5.1.1 Distributions When working with different statistical distributions, we often want to make probabilistic statements based on the distribution. We typically want to know one of four things: The density (pdf) at a particular value. The distribution (cdf) at a particular value. The quantile value corresponding to a particular probability. A random draw of values from a particular distribution. This used to be done with statistical tables printed in the back of textbooks. Now, R has functions for obtaining density, distribution, quantile and random values. The general naming structure of the relevant R functions is: dname calculates density (pdf) at input x. pname calculates distribution (cdf) at input x. qname calculates the quantile at an input probability. rname generates a random draw from a particular distribution. Note that name represents the name of the given distribution. For example, consider a random variable \\(X\\) which is \\(N(\\mu = 2, \\sigma^2 = 25)\\). (Note, we are parameterizing using the variance \\(\\sigma^2\\). R however uses the standard deviation.) To calculate the value of the pdf at x = 3, that is, the height of the curve at x = 3, use: dnorm(x = 3, mean = 2, sd = 5) ## [1] 0.07820854 To calculate the value of the cdf at x = 3, that is, \\(P(X \\leq 3)\\), the probability that \\(X\\) is less than or equal to 3, use: pnorm(q = 3, mean = 2, sd = 5) ## [1] 0.5792597 Or, to calculate the quantile for probability 0.975, use: qnorm(p = 0.975, mean = 2, sd = 5) ## [1] 11.79982 Lastly, to generate a random sample of size n = 10, use: rnorm(n = 10, mean = 2, sd = 5) ## [1] 1.3334356 -5.0754528 14.1540511 2.1927133 -3.8482335 -0.2280363 ## [7] 7.6598749 -2.7304969 -10.8570448 -6.9123819 These functions exist for many other distributions, including but not limited to: Command Distribution *binom Binomial *t t *pois Poisson *f F *chisq Chi-Squared Where * can be d, p, q, and r. Each distribution will have its own set of parameters which need to be passed to the functions as arguments. For example, dbinom() would not have arguments for mean and sd, since those are not parameters of the distribution. Instead a binomial distribution is usually parameterized by \\(n\\) and \\(p\\), however R chooses to call them something else. To find the names that R uses we would use ?dbinom and see that R instead calls the arguments size and prob. For example: dbinom(x = 6, size = 10, prob = 0.75) ## [1] 0.145998 Also note that, when using the dname functions with discrete distributions, they are the pmf of the distribution. For example, the above command is \\(P(Y = 6)\\) if \\(Y \\sim b(n = 10, p = 0.75)\\). (The probability of flipping an unfair coin 10 times and seeing 6 heads, if the probability of heads is 0.75.) 5.2 Hypothesis Tests in R A prerequisite for STAT 420 is an understanding of the basics of hypothesis testing. Recall the basic structure of hypothesis tests: An overall model and related assumptions are made. (The most common being observations following a normal distribution.) The null (\\(H_{0}\\)) and alternative (\\(H_{1}\\) or \\(H_{A}\\)) hypotheses are specified. Usually the null specifies a particular value of a parameter. With given data, the value of the test statistic is calculated. Under the general assumptions, as well as assuming the null hypothesis is true, the distribution of the test statistic is known. Given the distribution and value of the test statistic, as well as the form of the alternative hypothesis, we can calculate a p-value of the test. Based on the p-value and pre-specified level of significance, we make a decision. One of: Fail to reject the null hypothesis. Reject the null hypothesis. We’ll do some quick review of two of the most common tests to show how they are performed using R. 5.2.1 One Sample t-Test: Review Suppose \\(x_{i} \\sim \\mathrm{N}(\\mu,\\sigma^{2})\\) and we want to test \\(H_{0}: \\mu = \\mu_{0}\\) versus \\(H_{1}: \\mu \\neq \\mu_{0}.\\) Assuming \\(\\sigma\\) is unknown, we use the one-sample Student’s \\(t\\) test statistic: \\[ t = \\frac{\\bar{x}-\\mu_{0}}{s/\\sqrt{n}} \\sim t_{n-1}, \\] where \\(\\bar{x} = \\displaystyle\\frac{\\sum_{i=1}^{n}x_{i}}{n}\\) and \\(s = \\sqrt{\\displaystyle\\frac{1}{n - 1}\\sum_{i=1}^{n}(x_i - \\bar{x})^2}\\). A \\(100(1 - \\alpha)\\)% confidence interval for \\(\\mu\\) is given by, \\[ \\bar{x} \\pm t_{\\alpha/2, n-1}\\frac{s}{\\sqrt{n}} \\] where \\(t_{\\alpha/2, n-1}\\) is the critical value such that \\(P\\left(t&gt;t_{\\alpha/2, n-1}\\right) = \\alpha/2\\) for \\(n-1\\) degrees of freedom. 5.2.2 One Sample t-Test: Example Suppose a grocery store sells “16 ounce” boxes of Captain Crisp cereal. A random sample of 9 boxes was taken and weighed. The weight in ounces is stored in the data frame capt_crisp. capt_crisp = data.frame(weight = c(15.5, 16.2, 16.1, 15.8, 15.6, 16.0, 15.8, 15.9, 16.2)) The company that makes Captain Crisp cereal claims that the average weight of a box is at least 16 ounces. We will assume the weight of cereal in a box is normally distributed and use a 0.05 level of significance to test the company’s claim. To test \\(H_{0}: \\mu \\geq 16\\) versus \\(H_{1}: \\mu &lt; 16\\), the test statistic is \\[ t = \\frac{\\bar{x} - \\mu_{0}}{s / \\sqrt{n}} \\] The sample mean \\(\\bar{x}\\) and the sample standard deviation \\(s\\) can be easily computed using R. We also create variables which store the hypothesized mean and the sample size. x_bar = mean(capt_crisp$weight) s = sd(capt_crisp$weight) mu_0 = 16 n = 9 We can then easily compute the test statistic. t = (x_bar - mu_0) / (s / sqrt(n)) t ## [1] -1.2 Under the null hypothesis, the test statistic has a \\(t\\) distribution with \\(n - 1\\) degrees of freedom, in this case 8. To complete the test, we need to obtain the p-value of the test. Since this is a one-sided test with a less-than alternative, we need the area to the left of -1.2 for a \\(t\\) distribution with 8 degrees of freedom. That is, \\[ P(t_{8} &lt; -1.2) \\] pt(t, df = n - 1) ## [1] 0.1322336 We now have the p-value of our test, which is greater than our significance level (0.05), so we fail to reject the null hypothesis. Alternatively, this entire process could have been completed using one line of R code. t.test(x = capt_crisp$weight, mu = 16, alternative = c(&quot;less&quot;), conf.level = 0.95) ## ## One Sample t-test ## ## data: capt_crisp$weight ## t = -1.2, df = 8, p-value = 0.1322 ## alternative hypothesis: true mean is less than 16 ## 95 percent confidence interval: ## -Inf 16.05496 ## sample estimates: ## mean of x ## 15.9 We supply R with the data, the hypothesized value of \\(\\mu\\), the alternative, and the confidence level. R then returns a wealth of information including: The value of the test statistic. The degrees of freedom of the distribution under the null hypothesis. The p-value of the test. The confidence interval which corresponds to the test. An estimate of \\(\\mu\\). Since the test was one-sided, R returned a one-sided confidence interval. If instead we wanted a two-sided interval for the mean weight of boxes of Captain Crisp cereal we could modify our code. capt_test_results = t.test(capt_crisp$weight, mu = 16, alternative = c(&quot;two.sided&quot;), conf.level = 0.95) This time we have stored the results. By doing so, we can directly access portions of the output from t.test(). To see what information is available we use the names() function. names(capt_test_results) ## [1] &quot;statistic&quot; &quot;parameter&quot; &quot;p.value&quot; &quot;conf.int&quot; &quot;estimate&quot; ## [6] &quot;null.value&quot; &quot;stderr&quot; &quot;alternative&quot; &quot;method&quot; &quot;data.name&quot; We are interested in the confidence interval which is stored in conf.int. capt_test_results$conf.int ## [1] 15.70783 16.09217 ## attr(,&quot;conf.level&quot;) ## [1] 0.95 Let’s check this interval “by hand.” The one piece of information we are missing is the critical value, \\(t_{\\alpha/2, n-1} = t_{8}(0.025)\\), which can be calculated in R using the qt() function. qt(0.975, df = 8) ## [1] 2.306004 So, the 95% CI for the mean weight of a cereal box is calculated by plugging into the formula, \\[ \\bar{x} \\pm t_{\\alpha/2, n-1} \\frac{s}{\\sqrt{n}} \\] c(mean(capt_crisp$weight) - qt(0.975, df = 8) * sd(capt_crisp$weight) / sqrt(9), mean(capt_crisp$weight) + qt(0.975, df = 8) * sd(capt_crisp$weight) / sqrt(9)) ## [1] 15.70783 16.09217 5.2.3 Two Sample t-Test: Review Suppose \\(x_{i} \\sim \\mathrm{N}(\\mu_{x}, \\sigma^{2})\\) and \\(y_{i} \\sim \\mathrm{N}(\\mu_{y}, \\sigma^{2}).\\) Want to test \\(H_{0}: \\mu_{x} - \\mu_{y} = \\mu_{0}\\) versus \\(H_{1}: \\mu_{x} - \\mu_{y} \\neq \\mu_{0}.\\) Assuming \\(\\sigma\\) is unknown, use the two-sample Student’s \\(t\\) test statistic: \\[ t = \\frac{(\\bar{x} - \\bar{y})-\\mu_{0}}{s_{p}\\sqrt{\\frac{1}{n}+\\frac{1}{m}}} \\sim t_{n+m-2}, \\] where \\(\\displaystyle\\bar{x}=\\frac{\\sum_{i=1}^{n}x_{i}}{n}\\), \\(\\displaystyle\\bar{y}=\\frac{\\sum_{i=1}^{m}y_{i}}{m}\\), and \\(s_p^2 = \\displaystyle\\frac{(n-1)s_x^2+(m-1)s_y^2}{n+m-2}\\). A \\(100(1-\\alpha)\\)% CI for \\(\\mu_{x}-\\mu_{y}\\) is given by \\[ (\\bar{x} - \\bar{y}) \\pm t_{\\alpha/2, n+m-2}\\left(s_{p}\\textstyle\\sqrt{\\frac{1}{n}+\\frac{1}{m}}\\right), \\] where \\(t_{\\alpha/2, n+m-2}\\) is the critical value such that \\(P\\left(t&gt;t_{\\alpha/2, n+m-2}\\right)=\\alpha/2\\). 5.2.4 Two Sample t-Test: Example Assume that the distributions of \\(X\\) and \\(Y\\) are \\(\\mathrm{N}(\\mu_{1},\\sigma^{2})\\) and \\(\\mathrm{N}(\\mu_{2},\\sigma^{2})\\), respectively. Given the \\(n = 6\\) observations of \\(X\\), x = c(70, 82, 78, 74, 94, 82) n = length(x) and the \\(m = 8\\) observations of \\(Y\\), y = c(64, 72, 60, 76, 72, 80, 84, 68) m = length(y) we will test \\(H_{0}: \\mu_{1} = \\mu_{2}\\) versus \\(H_{1}: \\mu_{1} &gt; \\mu_{2}\\). First, note that we can calculate the sample means and standard deviations. x_bar = mean(x) s_x = sd(x) y_bar = mean(y) s_y = sd(y) We can then calculate the pooled standard deviation. \\[ s_{p} = \\sqrt{\\frac{(n-1)s_{x}^{2}+(m-1)s_{y}^{2}}{n+m-2}} \\] s_p = sqrt(((n - 1) * s_x ^ 2 + (m - 1) * s_y ^ 2) / (n + m - 2)) Thus, the relevant \\(t\\) test statistic is given by \\[ t = \\frac{(\\bar{x}-\\bar{y})-\\mu_{0}}{s_{p}\\sqrt{\\frac{1}{n}+\\frac{1}{m}}}. \\] t = ((x_bar - y_bar) - 0) / (s_p * sqrt(1 / n + 1 / m)) t ## [1] 1.823369 Note that \\(t \\sim t_{n + m - 2} = t_{12}\\), so we can calculate the p-value, which is \\[ P(t_{12} &gt; 1.8233692). \\] 1 - pt(t, df = n + m - 2) ## [1] 0.04661961 But, then again, we could have simply performed this test in one line of R. t.test(x, y, alternative = c(&quot;greater&quot;), var.equal = TRUE) ## ## Two Sample t-test ## ## data: x and y ## t = 1.8234, df = 12, p-value = 0.04662 ## alternative hypothesis: true difference in means is greater than 0 ## 95 percent confidence interval: ## 0.1802451 Inf ## sample estimates: ## mean of x mean of y ## 80 72 Recall that a two-sample \\(t\\)-test can be done with or without an equal variance assumption. Here var.equal = TRUE tells R we would like to perform the test under the equal variance assumption. Above we carried out the analysis using two vectors x and y. In general, we will have a preference for using data frames. t_test_data = data.frame(values = c(x, y), group = c(rep(&quot;A&quot;, length(x)), rep(&quot;B&quot;, length(y)))) We now have the data stored in a single variables (values) and have created a second variable (group) which indicates which “sample” the value belongs to. t_test_data ## values group ## 1 70 A ## 2 82 A ## 3 78 A ## 4 74 A ## 5 94 A ## 6 82 A ## 7 64 B ## 8 72 B ## 9 60 B ## 10 76 B ## 11 72 B ## 12 80 B ## 13 84 B ## 14 68 B Now to perform the test, we still use the t.test() function but with the ~ syntax and a data argument. t.test(values ~ group, data = t_test_data, alternative = c(&quot;greater&quot;), var.equal = TRUE) ## ## Two Sample t-test ## ## data: values by group ## t = 1.8234, df = 12, p-value = 0.04662 ## alternative hypothesis: true difference in means between group A and group B is greater than 0 ## 95 percent confidence interval: ## 0.1802451 Inf ## sample estimates: ## mean in group A mean in group B ## 80 72 5.3 Simulation Simulation and model fitting are related but opposite processes. In simulation, the data generating process is known. We will know the form of the model as well as the value of each of the parameters. In particular, we will often control the distribution and parameters which define the randomness, or noise in the data. In model fitting, the data is known. We will then assume a certain form of model and find the best possible values of the parameters given the observed data. Essentially we are seeking to uncover the truth. Often we will attempt to fit many models, and we will learn metrics to assess which model fits best. Simulation vs Modeling Often we will simulate data according to a process we decide, then use a modeling method seen in class. We can then verify how well the method works, since we know the data generating process. One of the biggest strengths of R is its ability to carry out simulations using built-in functions for generating random samples from certain distributions. We’ll look at two very simple examples here, however simulation will be a topic we revisit several times throughout the course. 5.3.1 Paired Differences Consider the model: \\[ \\begin{split} X_{11}, X_{12}, \\ldots, X_{1n} \\sim N(\\mu_1,\\sigma^2)\\\\ X_{21}, X_{22}, \\ldots, X_{2n} \\sim N(\\mu_2,\\sigma^2) \\end{split} \\] Assume that \\(\\mu_1 = 6\\), \\(\\mu_2 = 5\\), \\(\\sigma^2 = 4\\) and \\(n = 25\\). Let \\[ \\begin{aligned} \\bar{X}_1 &amp;= \\displaystyle\\frac{1}{n}\\sum_{i=1}^{n}X_{1i}\\\\ \\bar{X}_2 &amp;= \\displaystyle\\frac{1}{n}\\sum_{i=1}^{n}X_{2i}\\\\ D &amp;= \\bar{X}_1 - \\bar{X}_2. \\end{aligned} \\] Suppose we would like to calculate \\(P(0 &lt; D &lt; 2)\\). First we will need to obtain the distribution of \\(D\\). Recall, \\[ \\bar{X}_1 \\sim N\\left(\\mu_1,\\frac{\\sigma^2}{n}\\right) \\] and \\[ \\bar{X}_2 \\sim N\\left(\\mu_2,\\frac{\\sigma^2}{n}\\right). \\] Then, \\[ D = \\bar{X}_1 - \\bar{X}_2 \\sim N\\left(\\mu_1-\\mu_2, \\frac{\\sigma^2}{n} + \\frac{\\sigma^2}{n}\\right) = N\\left(6-5, \\frac{4}{25} + \\frac{4}{25}\\right). \\] So, \\[ D \\sim N(\\mu = 1, \\sigma^2 = 0.32). \\] Thus, \\[ P(0 &lt; D &lt; 2) = P(D &lt; 2) - P(D &lt; 0). \\] This can then be calculated using R without a need to first standardize, or use a table. pnorm(2, mean = 1, sd = sqrt(0.32)) - pnorm(0, mean = 1, sd = sqrt(0.32)) ## [1] 0.9229001 An alternative approach, would be to simulate a large number of observations of \\(D\\) then use the empirical distribution to calculate the probability. Our strategy will be to repeatedly: Generate a sample of 25 random observations from \\(N(\\mu_1 = 6,\\sigma^2 = 4)\\). Call the mean of this sample \\(\\bar{x}_{1s}\\). Generate a sample of 25 random observations from \\(N(\\mu_1 = 5,\\sigma^2 = 4)\\). Call the mean of this sample \\(\\bar{x}_{2s}\\). Calculate the differences in the means, \\(d_s = \\bar{x}_{1s} - \\bar{x}_{2s}\\). We will repeat the process a large number of times. Then we will use the distribution of the simulated observations of \\(d_s\\) as an estimate for the true distribution of \\(D\\). set.seed(42) num_samples = 10000 differences = rep(0, num_samples) Before starting our for loop to perform the operation, we set a seed for reproducibility, create and set a variable num_samples which will define the number of repetitions, and lastly create a variables differences which will store the simulate values, \\(d_s\\). By using set.seed() we can reproduce the random results of rnorm() each time starting from that line. for (s in 1:num_samples) { x1 = rnorm(n = 25, mean = 6, sd = 2) x2 = rnorm(n = 25, mean = 5, sd = 2) differences[s] = mean(x1) - mean(x2) } To estimate \\(P(0 &lt; D &lt; 2)\\) we will find the proportion of values of \\(d_s\\) (among the 10^{4} values of \\(d_s\\) generated) that are between 0 and 2. mean(0 &lt; differences &amp; differences &lt; 2) ## [1] 0.9222 Recall that above we derived the distribution of \\(D\\) to be \\(N(\\mu = 1, \\sigma^2 = 0.32)\\) If we look at a histogram of the differences, we find that it looks very much like a normal distribution. hist(differences, breaks = 20, main = &quot;Empirical Distribution of D&quot;, xlab = &quot;Simulated Values of D&quot;, col = &quot;dodgerblue&quot;, border = &quot;darkorange&quot;) Also the sample mean and variance are very close to what we would expect. mean(differences) ## [1] 1.001423 var(differences) ## [1] 0.3230183 We could have also accomplished this task with a single line of more “idiomatic” R. set.seed(42) diffs = replicate(10000, mean(rnorm(25, 6, 2)) - mean(rnorm(25, 5, 2))) Use ?replicate to take a look at the documentation for the replicate function and see if you can understand how this line performs the same operations that our for loop above executed. mean(differences == diffs) ## [1] 1 We see that by setting the same seed for the randomization, we actually obtain identical results! 5.3.2 Distribution of a Sample Mean For another example of simulation, we will simulate observations from a Poisson distribution, and examine the empirical distribution of the sample mean of these observations. Recall, if \\[ X \\sim Pois(\\mu) \\] then \\[ E[X] = \\mu \\] and \\[ Var[X] = \\mu. \\] Also, recall that for a random variable \\(X\\) with finite mean \\(\\mu\\) and finite variance \\(\\sigma^2\\), the central limit theorem tells us that the mean, \\(\\bar{X}\\) of a random sample of size \\(n\\) is approximately normal for large values of \\(n\\). Specifically, as \\(n \\to \\infty\\), \\[ \\bar{X} \\overset{d}{\\to} N\\left(\\mu, \\frac{\\sigma^2}{n}\\right). \\] The following verifies this result for a Poisson distribution with \\(\\mu = 10\\) and a sample size of \\(n = 50\\). set.seed(1337) mu = 10 sample_size = 50 samples = 100000 x_bars = rep(0, samples) for(i in 1:samples){ x_bars[i] = mean(rpois(sample_size, lambda = mu)) } x_bar_hist = hist(x_bars, breaks = 50, main = &quot;Histogram of Sample Means&quot;, xlab = &quot;Sample Means&quot;) Now we will compare sample statistics from the empirical distribution with their known values based on the parent distribution. c(mean(x_bars), mu) ## [1] 10.00008 10.00000 c(var(x_bars), mu / sample_size) ## [1] 0.1989732 0.2000000 c(sd(x_bars), sqrt(mu) / sqrt(sample_size)) ## [1] 0.4460641 0.4472136 And here, we will calculate the proportion of sample means that are within 2 standard deviations of the population mean. mean(x_bars &gt; mu - 2 * sqrt(mu) / sqrt(sample_size) &amp; x_bars &lt; mu + 2 * sqrt(mu) / sqrt(sample_size)) ## [1] 0.95429 This last histogram uses a bit of a trick to approximately shade the bars that are within two standard deviations of the mean.) shading = ifelse(x_bar_hist$breaks &gt; mu - 2 * sqrt(mu) / sqrt(sample_size) &amp; x_bar_hist$breaks &lt; mu + 2 * sqrt(mu) / sqrt(sample_size), &quot;darkorange&quot;, &quot;dodgerblue&quot;) x_bar_hist = hist(x_bars, breaks = 50, col = shading, main = &quot;Histogram of Sample Means, Two Standard Deviations&quot;, xlab = &quot;Sample Means&quot;) "],["r-resources.html", "Chapter 6 R Resources 6.1 Beginner Tutorials and References 6.2 Intermediate References 6.3 Advanced References 6.4 Quick Comparisons to Other Languages 6.5 RStudio and RMarkdown Videos 6.6 RMarkdown Template", " Chapter 6 R Resources So far, we have seen a lot of R, and a lot of R quickly. Again, the preceding chapters were in no way meant to be a complete reference for the R language, but rather an introduction to many of the concepts we will need in this text. The following resources are not necessary for the remainder of this text, but you may find them useful if you would like a deeper understanding of R: 6.1 Beginner Tutorials and References Try R from Code School. An interactive introduction to the basics of R. Useful for getting up to speed on R’s syntax. Quick-R by Robert Kabacoff. A good reference for R basics. R Tutorial by Chi Yau. A combination reference and tutorial for R basics. R Programming for Data Science by Roger Peng A great text for R programming beginners. Discusses R from the ground up, highlighting programming details we might not discuss. 6.2 Intermediate References R for Data Science by Hadley Wickham and Garrett Grolemund. Similar to Advanced R, but focuses more on data analysis, while still introducing programming concepts. Especially useful for working in the tidyverse. The Art of R Programming by Norman Matloff. Gentle introduction to the programming side of R. (Whereas we will focus more on the data analysis side.) A free electronic version is available through the Illinois library. 6.3 Advanced References Advanced R by Hadley Wickham. From the author of several extremely popular R packages. Good follow-up to The Art of R Programming. (And more up-to-date material.) The R Inferno by Patrick Burns. Likens learning the tricks of R to descending through the levels of hell. Very advanced material, but may be important if R becomes a part of your everyday toolkit. Efficient R Programming by Colin Gillespie and Robin Lovelace Discusses both efficient R programs, as well as programming in R efficiently. 6.4 Quick Comparisons to Other Languages Those who are familiar with other languages may find the following “cheatsheets” helpful for transitioning to R. MATLAB, NumPy, Julia Python pandas comparison with R/R Libraries Stata SAS - Look for a resource still! Suggestions welcome. 6.5 RStudio and RMarkdown Videos The following video playlists were made as an introduction to R, RStudio, and RMarkdown for STAT 420 at UIUC. If you are currently using this text for a Coursera course, you can also find updated videos there. R and RStudio Data in R RMarkdown Note that RStudio and RMarkdown are constantly receiving excellent support and updates, so these videos may already contain some outdated information. RStudio provides their own tutorial for RMarkdown. They also have an excellent RStudio “cheatsheet” which visually identifies many of the features available in the IDE. You may also explore some additional “cheatsheets” on their website. 6.6 RMarkdown Template This .zip file contains the files necessary to produce this rendered document. This document is a more complete version of a template than what is seen in the above videos. "],["simple-linear-regression.html", "Chapter 7 Simple Linear Regression 7.1 Modeling 7.2 Least Squares Approach 7.3 Decomposition of Variation 7.4 The lm Function 7.5 Maximum Likelihood Estimation (MLE) Approach 7.6 Simulating SLR 7.7 History 7.8 R Markdown", " Chapter 7 Simple Linear Regression “All models are wrong, but some are useful.” — George E. P. Box After reading this chapter you will be able to: Understand the concept of a model. Describe two ways in which regression coefficients are derived. Estimate and visualize a regression model using R. Interpret regression coefficients and statistics in the context of real-world problems. Use a regression model to make predictions. 7.1 Modeling Let’s consider a simple example of how the speed of a car affects its stopping distance, that is, how far it travels before it comes to a stop. To examine this relationship, we will use the cars dataset, which is a default R dataset. Thus, we don’t need to load a package first; it is immediately available. To get a first look at the data you can use the View() function inside RStudio. View(cars) We could also take a look at the variable names, the dimension of the data frame, and some sample observations with str(). str(cars) ## &#39;data.frame&#39;: 50 obs. of 2 variables: ## $ speed: num 4 4 7 7 8 9 10 10 10 11 ... ## $ dist : num 2 10 4 22 16 10 18 26 34 17 ... As we have seen before with data frames, there are a number of additional functions to access some of this information directly. dim(cars) ## [1] 50 2 nrow(cars) ## [1] 50 ncol(cars) ## [1] 2 Other than the two variable names and the number of observations, this data is still just a bunch of numbers, so we should probably obtain some context. ?cars Reading the documentation we learn that this is data gathered during the 1920s about the speed of cars and the resulting distance it takes for the car to come to a stop. The interesting task here is to determine how far a car travels before stopping, when traveling at a certain speed. So, we will first plot the stopping distance against the speed. plot(dist ~ speed, data = cars, xlab = &quot;Speed (in Miles Per Hour)&quot;, ylab = &quot;Stopping Distance (in Feet)&quot;, main = &quot;Stopping Distance vs Speed&quot;, pch = 20, cex = 2, col = &quot;grey&quot;) Let’s now define some terminology. We have pairs of data, \\((x_i, y_i)\\), for \\(i = 1, 2, \\ldots n\\), where \\(n\\) is the sample size of the dataset. We use \\(i\\) as an index, simply for notation. We use \\(x_i\\) as the predictor (explanatory) variable. The predictor variable is used to help predict or explain the response (target, outcome) variable, \\(y_i\\). Other texts may use the term independent variable instead of predictor and dependent variable in place of response. However, those monikers imply mathematical characteristics that might not be true. While these other terms are not incorrect, independence is already a strictly defined concept in probability. For example, when trying to predict a person’s weight given their height, would it be accurate to say that height is independent of weight? Certainly not, but that is an unintended implication of saying “independent variable.” We prefer to stay away from this nomenclature. In the cars example, we are interested in using the predictor variable speed to predict and explain the response variable dist. Broadly speaking, we would like to model the relationship between \\(X\\) and \\(Y\\) using the form \\[ Y = f(X) + \\epsilon. \\] The function \\(f\\) describes the functional relationship between the two variables, and the \\(\\epsilon\\) term is used to account for error. This indicates that if we plug in a given value of \\(X\\) as input, our output is a value of \\(Y\\), within a certain range of error. You could think of this a number of ways: Response = Prediction + Error Response = Signal + Noise Response = Model + Unexplained Response = Deterministic + Random Response = Explainable + Unexplainable What sort of function should we use for \\(f(X)\\) for the cars data? We could try to model the data with a horizontal line. That is, the model for \\(y\\) does not depend on the value of \\(x\\). (Some function \\(f(X) = c\\).) In the plot below, we see this doesn’t seem to do a very good job. Many of the data points are very far from the orange line representing \\(c\\). This is an example of underfitting. The obvious fix is to make the function \\(f(X)\\) actually depend on \\(x\\). We could also try to model the data with a very “wiggly” function that tries to go through as many of the data points as possible. This also doesn’t seem to work very well. The stopping distance for a speed of 5 mph shouldn’t be off the chart! (Even in 1920.) This is an example of overfitting. (Note that in this example no function will go through every point, since there are some \\(x\\) values that have several possible \\(y\\) values in the data.) Lastly, we could try to model the data with a well-chosen line rather than one of the two extremes previously attempted. The line on the plot below seems to summarize the relationship between stopping distance and speed quite well. As speed increases, the distance required to come to a stop increases. There is still some variation about this line, but it seems to capture the overall trend. With this in mind, we would like to restrict our choice of \\(f(X)\\) to linear functions of \\(X\\). We will write our model using \\(\\beta_1\\) for the slope, and \\(\\beta_0\\) for the intercept, \\[ Y = \\beta_0 + \\beta_1 X + \\epsilon. \\] 7.1.1 Simple Linear Regression Model We now define what we will call the simple linear regression model, \\[ Y_i = \\beta_0 + \\beta_1 x_i + \\epsilon_i \\] where \\[ \\epsilon_i \\sim N(0, \\sigma^2). \\] That is, the \\(\\epsilon_i\\) are independent and identically distributed (iid) normal random variables with mean \\(0\\) and variance \\(\\sigma^2\\). This model has three parameters to be estimated: \\(\\beta_0\\), \\(\\beta_1\\), and \\(\\sigma^2\\), which are fixed, but unknown constants. We have slightly modified our notation here. We are now using \\(Y_i\\) and \\(x_i\\), since we will be fitting this model to a set of \\(n\\) data points, for \\(i = 1, 2, \\ldots n\\). Recall that we use capital \\(Y\\) to indicate a random variable, and lower case \\(y\\) to denote a potential value of the random variable. Since we will have \\(n\\) observations, we have \\(n\\) random variables \\(Y_i\\) and their possible values \\(y_i\\). In the simple linear regression model, the \\(x_i\\) are assumed to be fixed, known constants, and are thus notated with a lower case variable. The response \\(Y_i\\) remains a random variable because of the random behavior of the error variable, \\(\\epsilon_i\\). That is, each response \\(Y_i\\) is tied to an observable \\(x_i\\) and a random, unobservable, \\(\\epsilon_i\\). Essentially, we could explicitly think of the \\(Y_i\\) as having a different distribution for each \\(X_i\\). In other words, \\(Y_i\\) has a conditional distribution dependent on the value of \\(X_i\\), written \\(x_i\\). Doing so, we still make no distributional assumptions of the \\(X_i\\), since we are only interested in the distribution of the \\(Y_i\\) for a particular value \\(x_i\\). \\[ Y_i \\mid X_i \\sim N(\\beta_0 + \\beta_1 x_i, \\sigma^2) \\] The random \\(Y_i\\) are a function of \\(x_i\\), thus we can write its mean as a function of \\(x_i\\), \\[ \\text{E}[Y_i \\mid X_i = x_i] = \\beta_0 + \\beta_1 x_i. \\] However, its variance remains constant for each \\(x_i\\), \\[ \\text{Var}[Y_i \\mid X_i = x_i] = \\sigma^2. \\] This is visually displayed in the image below. We see that for any value \\(x\\), the expected value of \\(Y\\) is \\(\\beta_0 + \\beta_1 x\\). At each value of \\(x\\), \\(Y\\) has the same variance \\(\\sigma^2\\). Simple Linear Regression Model Introductory Statistics (Shafer and Zhang), UC Davis Stat Wiki Often, we directly talk about the assumptions that this model makes. They can be cleverly shortened to LINE. Linear. The relationship between \\(Y\\) and \\(x\\) is linear, of the form \\(\\beta_0 + \\beta_1 x\\). Independent. The errors \\(\\epsilon\\) are independent. Normal. The errors, \\(\\epsilon\\) are normally distributed. That is the “error” around the line follows a normal distribution. Equal Variance. At each value of \\(x\\), the variance of \\(Y\\) is the same, \\(\\sigma^2\\). We are also assuming that the values of \\(x\\) are fixed, that is, not random. We do not make a distributional assumption about the predictor variable. As a side note, we will often refer to simple linear regression as SLR. Some explanation of the name SLR: Simple refers to the fact that we are using a single predictor variable. Later we will use multiple predictor variables. Linear tells us that our model for \\(Y\\) is a linear combination of the predictors \\(X\\). (In this case just the one.) Right now, this always results in a model that is a line, but later we will see how this is not always the case. Regression simply means that we are attempting to measure the relationship between a response variable and (one or more) predictor variables. In the case of SLR, both the response and the predictor are numeric variables. So SLR models \\(Y\\) as a linear function of \\(X\\), but how do we actually define a good line? There are an infinite number of lines we could use, so we will attempt to find one with “small errors.” That is a line with as many points as close to it as possible. The question now becomes, how do we find such a line? There are many approaches we could take. We could find the line that has the smallest maximum distance from any of the points to the line. That is, \\[ \\underset{\\beta_0, \\beta_1}{\\mathrm{argmin}} \\max|y_i - (\\beta_0 + \\beta_1 x_i)|. \\] We could find the line that minimizes the sum of all the distances from the points to the line. That is, \\[ \\underset{\\beta_0, \\beta_1}{\\mathrm{argmin}} \\sum_{i = 1}^{n}|y_i - (\\beta_0 + \\beta_1 x_i)|. \\] We could find the line that minimizes the sum of all the squared distances from the points to the line. That is, \\[ \\underset{\\beta_0, \\beta_1}{\\mathrm{argmin}} \\sum_{i = 1}^{n}(y_i - (\\beta_0 + \\beta_1 x_i))^2. \\] This last option is called the method of least squares. It is essentially the de-facto method for fitting a line to data. (You may have even seen it before in a linear algebra course.) Its popularity is largely due to the fact that it is mathematically “easy.” (Which was important historically, as computers are a modern contraption.) It is also very popular because many relationships are well approximated by a linear function. 7.2 Least Squares Approach Given observations \\((x_i, y_i)\\), for \\(i = 1, 2, \\ldots n\\), we want to find values of \\(\\beta_0\\) and \\(\\beta_1\\) which minimize \\[ f(\\beta_0, \\beta_1) = \\sum_{i = 1}^{n}(y_i - (\\beta_0 + \\beta_1 x_i))^2 = \\sum_{i = 1}^{n}(y_i - \\beta_0 - \\beta_1 x_i)^2. \\] We will call these values \\(\\hat{\\beta}_0\\) and \\(\\hat{\\beta}_1\\). First, we take a partial derivative with respect to both \\(\\beta_0\\) and \\(\\beta_1\\). \\[ \\begin{aligned} \\frac{\\partial f}{\\partial \\beta_0} &amp;= -2 \\sum_{i = 1}^{n}(y_i - \\beta_0 - \\beta_1 x_i) \\\\ \\frac{\\partial f}{\\partial \\beta_1} &amp;= -2 \\sum_{i = 1}^{n}(x_i)(y_i - \\beta_0 - \\beta_1 x_i) \\end{aligned} \\] We then set each of the partial derivatives equal to zero and solve the resulting system of equations. \\[ \\begin{aligned} \\sum_{i = 1}^{n}(y_i - \\beta_0 - \\beta_1 x_i) &amp;= 0 \\\\ \\sum_{i = 1}^{n}(x_i)(y_i - \\beta_0 - \\beta_1 x_i) &amp;= 0 \\end{aligned} \\] While solving the system of equations, one common algebraic rearrangement results in the normal equations. \\[ \\begin{aligned} n \\beta_0 + \\beta_1 \\sum_{i = 1}^{n} x_i &amp;= \\sum_{i = 1}^{n} y_i\\\\ \\beta_0 \\sum_{i = 1}^{n} x_i + \\beta_1 \\sum_{i = 1}^{n} x_i^2 &amp;= \\sum_{i = 1}^{n} x_i y_i \\end{aligned} \\] Finally, we finish solving the system of equations. \\[ \\begin{aligned} \\hat{\\beta}_1 &amp;= \\frac{\\sum_{i = 1}^{n} x_i y_i - \\frac{(\\sum_{i = 1}^{n} x_i)(\\sum_{i = 1}^{n} y_i)}{n}}{\\sum_{i = 1}^{n} x_i^2 - \\frac{(\\sum_{i = 1}^{n} x_i)^2}{n}} = \\frac{S_{xy}}{S_{xx}}\\\\ \\hat{\\beta}_0 &amp;= \\bar{y} - \\hat{\\beta}_1 \\bar{x} \\end{aligned} \\] Here, we have defined some notation for the expression we’ve obtained. Note that they have alternative forms which are much easier to work with. (We won’t do it here, but you can try to prove the equalities below on your own, for “fun”). We use the capital letter \\(S\\) to denote “summation” which replaces the capital letter \\(\\Sigma\\) when we calculate these values based on observed data, \\((x_i ,y_i)\\). The subscripts such as \\(xy\\) denote over which variables the function \\((z - \\bar{z})\\) is applied. \\[ \\begin{aligned} S_{xy} &amp;= \\sum_{i = 1}^{n} x_i y_i - \\frac{(\\sum_{i = 1}^{n} x_i)(\\sum_{i = 1}^{n} y_i)}{n} = \\sum_{i = 1}^{n}(x_i - \\bar{x})(y_i - \\bar{y})\\\\ S_{xx} &amp;= \\sum_{i = 1}^{n} x_i^2 - \\frac{(\\sum_{i = 1}^{n} x_i)^2}{n} = \\sum_{i = 1}^{n}(x_i - \\bar{x})^2\\\\ S_{yy} &amp;= \\sum_{i = 1}^{n} y_i^2 - \\frac{(\\sum_{i = 1}^{n} y_i)^2}{n} = \\sum_{i = 1}^{n}(y_i - \\bar{y})^2 \\end{aligned} \\] Note that these summations \\(S\\) are not to be confused with sample standard deviation \\(s\\). By using the above alternative expressions for \\(S_{xy}\\) and \\(S_{xx}\\), we arrive at a cleaner, more useful expression for \\(\\hat{\\beta}_1\\). \\[ \\hat{\\beta}_1 = \\frac{S_{xy}}{S_{xx}} = \\frac{\\sum_{i = 1}^{n}(x_i - \\bar{x})(y_i - \\bar{y})}{\\sum_{i = 1}^{n}(x_i - \\bar{x})^2} \\] Traditionally we would now calculate \\(\\hat{\\beta}_0\\) and \\(\\hat{\\beta}_1\\) by hand for the cars dataset. However because we are living in the 21st century and are intelligent (or lazy or efficient, depending on your perspective), we will utilize R to do the number crunching for us. To keep some notation consistent with the above mathematics, we will store the response variable as y and the predictor variable as x. x = cars$speed y = cars$dist We then calculate the three sums of squares defined above. Sxy = sum((x - mean(x)) * (y - mean(y))) Sxx = sum((x - mean(x)) ^ 2) Syy = sum((y - mean(y)) ^ 2) c(Sxy, Sxx, Syy) ## [1] 5387.40 1370.00 32538.98 Then finally calculate \\(\\hat{\\beta}_0\\) and \\(\\hat{\\beta}_1\\). beta_1_hat = Sxy / Sxx beta_0_hat = mean(y) - beta_1_hat * mean(x) c(beta_0_hat, beta_1_hat) ## [1] -17.579095 3.932409 What do these values tell us about our dataset? The slope parameter \\(\\beta_1\\) tells us that for an increase in speed of one mile per hour, the mean stopping distance increases by \\(\\beta_1\\). It is important to specify that we are talking about the mean. Recall that \\(\\beta_0 + \\beta_1 x\\) is the mean of \\(Y\\), in this case stopping distance, for a particular value of \\(x\\). (In this case speed.) So \\(\\beta_1\\) tells us how the mean of \\(Y\\) is affected by a change in \\(x\\). Similarly, the estimate \\(\\hat{\\beta}_1 = 3.93\\) tells us that for an increase in speed of one mile per hour, the estimated mean stopping distance increases by \\(3.93\\) feet. Here we should be sure to specify we are discussing an estimated quantity. Recall that \\(\\hat{y}\\) is the estimated mean of \\(Y\\), so \\(\\hat{\\beta}_1\\) tells us how the estimated mean of \\(Y\\) is affected by changing \\(x\\). The intercept parameter \\(\\beta_0\\) tells us the mean stopping distance for a car traveling zero miles per hour. (Not moving.) The estimate \\(\\hat{\\beta}_0 = -17.58\\) tells us that the estimated mean stopping distance for a car traveling zero miles per hour is \\(-17.58\\) feet. So when you apply the brakes to a car that is not moving, it moves backwards? This doesn’t seem right. (Extrapolation, which we will see later, is the issue here.) 7.2.1 Making Predictions We can now write the fitted or estimated line, \\[ \\hat{y} = \\hat{\\beta}_0 + \\hat{\\beta}_1 x. \\] In this case, \\[ \\hat{y} = -17.58 + 3.93 x. \\] We can now use this line to make predictions. First, let’s see the possible \\(x\\) values in the cars dataset. Since some \\(x\\) values may appear more than once, we use the unique() to return each unique value only once. unique(cars$speed) ## [1] 4 7 8 9 10 11 12 13 14 15 16 17 18 19 20 22 23 24 25 Let’s make a prediction for the stopping distance of a car traveling at 8 miles per hour. \\[ \\hat{y} = -17.58 + 3.93 \\times 8 % = 13.88 \\] beta_0_hat + beta_1_hat * 8 ## [1] 13.88018 This tells us that the estimated mean stopping distance of a car traveling at 8 miles per hour is \\(13.88\\). Now let’s make a prediction for the stopping distance of a car traveling at 21 miles per hour. This is considered interpolation as 21 is not an observed value of \\(x\\). (But is in the data range.) We can use the special %in% operator to quickly verify this in R. 8 %in% unique(cars$speed) ## [1] TRUE 21 %in% unique(cars$speed) ## [1] FALSE min(cars$speed) &lt; 21 &amp; 21 &lt; max(cars$speed) ## [1] TRUE \\[ \\hat{y} = -17.58 + 3.93 \\times 21 % = 65 \\] beta_0_hat + beta_1_hat * 21 ## [1] 65.00149 Lastly, we can make a prediction for the stopping distance of a car traveling at 50 miles per hour. This is considered extrapolation as 50 is not an observed value of \\(x\\) and is outside data range. We should be less confident in predictions of this type. range(cars$speed) ## [1] 4 25 range(cars$speed)[1] &lt; 50 &amp; 50 &lt; range(cars$speed)[2] ## [1] FALSE \\[ \\hat{y} = -17.58 + 3.93 \\times 50 % = 179.04 \\] beta_0_hat + beta_1_hat * 50 ## [1] 179.0413 Cars travel 50 miles per hour rather easily today, but not in the 1920s! This is also an issue we saw when interpreting \\(\\hat{\\beta}_0 = -17.58\\), which is equivalent to making a prediction at \\(x = 0\\). We should not be confident in the estimated linear relationship outside of the range of data we have observed. 7.2.2 Residuals If we think of our model as “Response = Prediction + Error,” we can then write it as \\[ y = \\hat{y} + e. \\] We then define a residual to be the observed value minus the predicted value. \\[ e_i = y_i - \\hat{y}_i \\] Let’s calculate the residual for the prediction we made for a car traveling 8 miles per hour. First, we need to obtain the observed value of \\(y\\) for this \\(x\\) value. which(cars$speed == 8) ## [1] 5 cars[5, ] ## speed dist ## 5 8 16 cars[which(cars$speed == 8), ] ## speed dist ## 5 8 16 We can then calculate the residual. \\[ e = 16 - 13.88 = 2.12 \\] 16 - (beta_0_hat + beta_1_hat * 8) ## [1] 2.119825 The positive residual value indicates that the observed stopping distance is actually 2.12 feet more than what was predicted. 7.2.3 Variance Estimation We’ll now use the residuals for each of the points to create an estimate for the variance, \\(\\sigma^2\\). Recall that, \\[ \\text{E}[Y_i \\mid X_i = x_i] = \\beta_0 + \\beta_1 x_i. \\] So, \\[ \\hat{y}_i = \\hat{\\beta}_0 + \\hat{\\beta}_1 x_i \\] is a natural estimate for the mean of \\(Y_i\\) for a given value of \\(x_i\\). Also, recall that when we specified the model, we had three unknown parameters; \\(\\beta_0\\), \\(\\beta_1\\), and \\(\\sigma^2\\). The method of least squares gave us estimates for \\(\\beta_0\\) and \\(\\beta_1\\), however, we have yet to see an estimate for \\(\\sigma^2\\). We will now define \\(s_e^2\\) which will be an estimate for \\(\\sigma^2\\). \\[ \\begin{aligned} s_e^2 &amp;= \\frac{1}{n - 2} \\sum_{i = 1}^{n}(y_i - (\\hat{\\beta}_0 + \\hat{\\beta}_1 x_i))^2 \\\\ &amp;= \\frac{1}{n - 2} \\sum_{i = 1}^{n}(y_i - \\hat{y}_i)^2 \\\\ &amp;= \\frac{1}{n - 2} \\sum_{i = 1}^{n} e_i^2 \\end{aligned} \\] This probably seems like a natural estimate, aside from the use of \\(n - 2\\), which we will put off explaining until the next chapter. It should actually look rather similar to something we have seen before. \\[ s^2 = \\frac{1}{n - 1}\\sum_{i=1}^{n}(x_i - \\bar{x})^2 \\] Here, \\(s^2\\) is the estimate of \\(\\sigma^2\\) when we have a single random variable \\(X\\). In this case \\(\\bar{x}\\) is an estimate of \\(\\mu\\) which is assumed to be the same for each \\(x\\). Now, in the regression case, with \\(s_e^2\\) each \\(y\\) has a different mean because of the relationship with \\(x\\). Thus, for each \\(y_i\\), we use a different estimate of the mean, that is \\(\\hat{y}_i\\). y_hat = beta_0_hat + beta_1_hat * x e = y - y_hat n = length(e) s2_e = sum(e^2) / (n - 2) s2_e ## [1] 236.5317 Just as with the univariate measure of variance, this value of 236.53 doesn’t have a practical interpretation in terms of stopping distance. Taking the square root, however, computes the standard deviation of the residuals, also known as residual standard error. s_e = sqrt(s2_e) s_e ## [1] 15.37959 This tells us that our estimates of mean stopping distance are “typically” off by 15.38 feet. 7.3 Decomposition of Variation We can re-express \\(y_i - \\bar{y}\\), which measures the deviation of an observation from the sample mean, in the following way, \\[ y_i - \\bar{y} = (y_i - \\hat{y}_i) + (\\hat{y}_i - \\bar{y}). \\] This is the common mathematical trick of “adding zero.” In this case we both added and subtracted \\(\\hat{y}_i\\). Here, \\(y_i - \\hat{y}_i\\) measures the deviation of an observation from the fitted regression line and \\(\\hat{y}_i - \\bar{y}\\) measures the deviation of the fitted regression line from the sample mean. If we square then sum both sides of the equation above, we can obtain the following, \\[ \\sum_{i=1}^{n}(y_i - \\bar{y})^2 = \\sum_{i=1}^{n}(y_i - \\hat{y}_i)^2 + \\sum_{i=1}^{n}(\\hat{y}_i - \\bar{y})^2. \\] This should be somewhat alarming or amazing. How is this true? For now we will leave this question unanswered. (Think about this, and maybe try to prove it.) We will now define three of the quantities seen in this equation. Sum of Squares Total \\[ \\text{SST} = \\sum_{i=1}^{n}(y_i - \\bar{y})^2 \\] The quantity “Sum of Squares Total,” or \\(\\text{SST}\\), represents the total variation of the observed \\(y\\) values. This should be a familiar-looking expression. Note that, \\[ s ^ 2 = \\frac{1}{n - 1}\\sum_{i=1}^{n}(y_i - \\bar{y})^2 = \\frac{1}{n - 1} \\text{SST}. \\] Sum of Squares Regression \\[ \\text{SSReg} = \\sum_{i=1}^{n}(\\hat{y}_i - \\bar{y})^2 \\] The quantity “Sum of Squares Regression,” \\(\\text{SSReg}\\), represents the explained variation of the observed \\(y\\) values. Sum of Squares Error \\[ \\text{SSE} = \\text{RSS} = \\sum_{i=1}^{n}(y_i - \\hat{y}_i)^2 \\] The quantity “Sum of Squares Error,” \\(\\text{SSE}\\), represents the unexplained variation of the observed \\(y\\) values. You will often see \\(\\text{SSE}\\) written as \\(\\text{RSS}\\), or “Residual Sum of Squares.” SST = sum((y - mean(y)) ^ 2) SSReg = sum((y_hat - mean(y)) ^ 2) SSE = sum((y - y_hat) ^ 2) c(SST = SST, SSReg = SSReg, SSE = SSE) ## SST SSReg SSE ## 32538.98 21185.46 11353.52 Note that, \\[ s_e^2 = \\frac{\\text{SSE}}{n - 2}. \\] SSE / (n - 2) ## [1] 236.5317 We can use R to verify that this matches our previous calculation of \\(s_e^2\\). s2_e == SSE / (n - 2) ## [1] TRUE These three measures also do not have an important practical interpretation individually. But together, they’re about to reveal a new statistic to help measure the strength of an SLR model. 7.3.1 Coefficient of Determination The coefficient of determination, \\(R^2\\), is defined as \\[ \\begin{aligned} R^2 &amp;= \\frac{\\text{SSReg}}{\\text{SST}} = \\frac{\\sum_{i=1}^{n}(\\hat{y}_i - \\bar{y})^2}{\\sum_{i=1}^{n}(y_i - \\bar{y})^2} \\\\[2.5ex] &amp;= \\frac{\\text{SST} - \\text{SSE}}{\\text{SST}} = 1 - \\frac{\\text{SSE}}{\\text{SST}} \\\\[2.5ex] &amp;= 1 - \\frac{\\sum_{i=1}^{n}(y_i - \\hat{y}_i)^2}{\\sum_{i=1}^{n}(y_i - \\bar{y})^2} = 1 - \\frac{\\sum_{i = 1}^{n}e_i^2}{\\sum_{i=1}^{n}(y_i - \\bar{y})^2} \\end{aligned} \\] The coefficient of determination is interpreted as the proportion of observed variation in \\(y\\) that can be explained by the simple linear regression model. R2 = SSReg / SST R2 ## [1] 0.6510794 For the cars example, we calculate \\(R^2 = 0.65\\). We then say that \\(65\\%\\) of the observed variability in stopping distance is explained by the linear relationship with speed. The following plots visually demonstrate the three “sums of squares” for a simulated dataset which has \\(R^2 = 0.92\\) which is a somewhat high value. Notice in the final plot, that the orange arrows account for a larger proportion of the total arrow. The next plots again visually demonstrate the three “sums of squares,” this time for a simulated dataset which has \\(R^2 = 0.19\\). Notice in the final plot, that now the blue arrows account for a larger proportion of the total arrow. 7.4 The lm Function So far we have done regression by deriving the least squares estimates, then writing simple R commands to perform the necessary calculations. Since this is such a common task, this is functionality that is built directly into R via the lm() command. The lm() command is used to fit linear models which actually account for a broader class of models than simple linear regression, but we will use SLR as our first demonstration of lm(). The lm() function will be one of our most commonly used tools, so you may want to take a look at the documentation by using ?lm. You’ll notice there is a lot of information there, but we will start with just the very basics. This is documentation you will want to return to often. We’ll continue using the cars data, and essentially use the lm() function to check the work we had previously done. stop_dist_model = lm(dist ~ speed, data = cars) This line of code fits our very first linear model. The syntax should look somewhat familiar. We use the dist ~ speed syntax to tell R we would like to model the response variable dist as a linear function of the predictor variable speed. In general, you should think of the syntax as response ~ predictor. The data = cars argument then tells R that the dist and speed variables are from the dataset cars. We then store this result in a variable stop_dist_model. The variable stop_dist_model now contains a wealth of information, and we will now see how to extract and use that information. The first thing we will do is simply output whatever is stored immediately in the variable stop_dist_model. stop_dist_model ## ## Call: ## lm(formula = dist ~ speed, data = cars) ## ## Coefficients: ## (Intercept) speed ## -17.579 3.932 We see that it first tells us the formula we input into R, that is lm(formula = dist ~ speed, data = cars). We also see the coefficients of the model. We can check that these are what we had calculated previously. (Minus some rounding that R is doing when displaying the results. They are stored with full precision.) c(beta_0_hat, beta_1_hat) ## [1] -17.579095 3.932409 Next, it would be nice to add the fitted line to the scatterplot. To do so we will use the abline() function. plot(dist ~ speed, data = cars, xlab = &quot;Speed (in Miles Per Hour)&quot;, ylab = &quot;Stopping Distance (in Feet)&quot;, main = &quot;Stopping Distance vs Speed&quot;, pch = 20, cex = 2, col = &quot;grey&quot;) abline(stop_dist_model, lwd = 3, col = &quot;darkorange&quot;) The abline() function is used to add lines of the form \\(a + bx\\) to a plot. (Hence abline.) When we give it stop_dist_model as an argument, it automatically extracts the regression coefficient estimates (\\(\\hat{\\beta}_0\\) and \\(\\hat{\\beta}_1\\)) and uses them as the intercept and slope of the line. Here we also use lwd to modify the width of the line, as well as col to modify the color of the line. The “thing” that is returned by the lm() function is actually an object of class lm which is a list. The exact details of this are unimportant unless you are seriously interested in the inner-workings of R, but know that we can determine the names of the elements of the list using the names() command. names(stop_dist_model) ## [1] &quot;coefficients&quot; &quot;residuals&quot; &quot;effects&quot; &quot;rank&quot; ## [5] &quot;fitted.values&quot; &quot;assign&quot; &quot;qr&quot; &quot;df.residual&quot; ## [9] &quot;xlevels&quot; &quot;call&quot; &quot;terms&quot; &quot;model&quot; We can then use this information to, for example, access the residuals using the $ operator. stop_dist_model$residuals ## 1 2 3 4 5 6 7 ## 3.849460 11.849460 -5.947766 12.052234 2.119825 -7.812584 -3.744993 ## 8 9 10 11 12 13 14 ## 4.255007 12.255007 -8.677401 2.322599 -15.609810 -9.609810 -5.609810 ## 15 16 17 18 19 20 21 ## -1.609810 -7.542219 0.457781 0.457781 12.457781 -11.474628 -1.474628 ## 22 23 24 25 26 27 28 ## 22.525372 42.525372 -21.407036 -15.407036 12.592964 -13.339445 -5.339445 ## 29 30 31 32 33 34 35 ## -17.271854 -9.271854 0.728146 -11.204263 2.795737 22.795737 30.795737 ## 36 37 38 39 40 41 42 ## -21.136672 -11.136672 10.863328 -29.069080 -13.069080 -9.069080 -5.069080 ## 43 44 45 46 47 48 49 ## 2.930920 -2.933898 -18.866307 -6.798715 15.201285 16.201285 43.201285 ## 50 ## 4.268876 Another way to access stored information in stop_dist_model are the coef(), resid(), and fitted() functions. These return the coefficients, residuals, and fitted values, respectively. coef(stop_dist_model) ## (Intercept) speed ## -17.579095 3.932409 resid(stop_dist_model) ## 1 2 3 4 5 6 7 ## 3.849460 11.849460 -5.947766 12.052234 2.119825 -7.812584 -3.744993 ## 8 9 10 11 12 13 14 ## 4.255007 12.255007 -8.677401 2.322599 -15.609810 -9.609810 -5.609810 ## 15 16 17 18 19 20 21 ## -1.609810 -7.542219 0.457781 0.457781 12.457781 -11.474628 -1.474628 ## 22 23 24 25 26 27 28 ## 22.525372 42.525372 -21.407036 -15.407036 12.592964 -13.339445 -5.339445 ## 29 30 31 32 33 34 35 ## -17.271854 -9.271854 0.728146 -11.204263 2.795737 22.795737 30.795737 ## 36 37 38 39 40 41 42 ## -21.136672 -11.136672 10.863328 -29.069080 -13.069080 -9.069080 -5.069080 ## 43 44 45 46 47 48 49 ## 2.930920 -2.933898 -18.866307 -6.798715 15.201285 16.201285 43.201285 ## 50 ## 4.268876 fitted(stop_dist_model) ## 1 2 3 4 5 6 7 8 ## -1.849460 -1.849460 9.947766 9.947766 13.880175 17.812584 21.744993 21.744993 ## 9 10 11 12 13 14 15 16 ## 21.744993 25.677401 25.677401 29.609810 29.609810 29.609810 29.609810 33.542219 ## 17 18 19 20 21 22 23 24 ## 33.542219 33.542219 33.542219 37.474628 37.474628 37.474628 37.474628 41.407036 ## 25 26 27 28 29 30 31 32 ## 41.407036 41.407036 45.339445 45.339445 49.271854 49.271854 49.271854 53.204263 ## 33 34 35 36 37 38 39 40 ## 53.204263 53.204263 53.204263 57.136672 57.136672 57.136672 61.069080 61.069080 ## 41 42 43 44 45 46 47 48 ## 61.069080 61.069080 61.069080 68.933898 72.866307 76.798715 76.798715 76.798715 ## 49 50 ## 76.798715 80.731124 An R function that is useful in many situations is summary(). We see that when it is called on our model, it returns a good deal of information. By the end of the course, you will know what every value here is used for. For now, you should immediately notice the coefficient estimates, and you may recognize the \\(R^2\\) value we saw earlier. summary(stop_dist_model) ## ## Call: ## lm(formula = dist ~ speed, data = cars) ## ## Residuals: ## Min 1Q Median 3Q Max ## -29.069 -9.525 -2.272 9.215 43.201 ## ## Coefficients: ## Estimate Std. Error t value Pr(&gt;|t|) ## (Intercept) -17.5791 6.7584 -2.601 0.0123 * ## speed 3.9324 0.4155 9.464 1.49e-12 *** ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 ## ## Residual standard error: 15.38 on 48 degrees of freedom ## Multiple R-squared: 0.6511, Adjusted R-squared: 0.6438 ## F-statistic: 89.57 on 1 and 48 DF, p-value: 1.49e-12 The summary() command also returns a list, and we can again use names() to learn what about the elements of this list. names(summary(stop_dist_model)) ## [1] &quot;call&quot; &quot;terms&quot; &quot;residuals&quot; &quot;coefficients&quot; ## [5] &quot;aliased&quot; &quot;sigma&quot; &quot;df&quot; &quot;r.squared&quot; ## [9] &quot;adj.r.squared&quot; &quot;fstatistic&quot; &quot;cov.unscaled&quot; So, for example, if we wanted to directly access the value of \\(R^2\\), instead of copy and pasting it out of the printed statement from summary(), we could do so. summary(stop_dist_model)$r.squared ## [1] 0.6510794 Another value we may want to access is \\(s_e\\), which R calls sigma. summary(stop_dist_model)$sigma ## [1] 15.37959 Note that this is the same result seen earlier as s_e. You may also notice that this value was displayed above as a result of the summary() command, which R labeled the “Residual Standard Error.” \\[ s_e = \\text{RSE} = \\sqrt{\\frac{1}{n - 2}\\sum_{i = 1}^n e_i^2} \\] Often it is useful to talk about \\(s_e\\) (or RSE) instead of \\(s_e^2\\) because of their units. The unit of \\(s_e\\) in the cars example is feet, while the unit of \\(s_e^2\\) is feet-squared. Another useful function, which we will use almost as often as lm() is the predict() function. predict(stop_dist_model, newdata = data.frame(speed = 8)) ## 1 ## 13.88018 The above code reads “predict the stopping distance of a car traveling 8 miles per hour using the stop_dist_model.” Importantly, the second argument to predict() is a data frame that we make in place. We do this so that we can specify that 8 is a value of speed, so that predict knows how to use it with the model stored in stop_dist_model. We see that this result is what we had calculated “by hand” previously. We could also predict multiple values at once. predict(stop_dist_model, newdata = data.frame(speed = c(8, 21, 50))) ## 1 2 3 ## 13.88018 65.00149 179.04134 \\[ \\begin{aligned} \\hat{y} &amp;= -17.58 + 3.93 \\times 8 = 13.88 \\\\ \\hat{y} &amp;= -17.58 + 3.93 \\times 21 = 65 \\\\ \\hat{y} &amp;= -17.58 + 3.93 \\times 50 = 179.04 \\end{aligned} \\] Or we could calculate the fitted value for each of the original data points. We can simply supply the original data frame, cars, since it contains a variables called speed which has the values we would like to predict at. predict(stop_dist_model, newdata = cars) ## 1 2 3 4 5 6 7 8 ## -1.849460 -1.849460 9.947766 9.947766 13.880175 17.812584 21.744993 21.744993 ## 9 10 11 12 13 14 15 16 ## 21.744993 25.677401 25.677401 29.609810 29.609810 29.609810 29.609810 33.542219 ## 17 18 19 20 21 22 23 24 ## 33.542219 33.542219 33.542219 37.474628 37.474628 37.474628 37.474628 41.407036 ## 25 26 27 28 29 30 31 32 ## 41.407036 41.407036 45.339445 45.339445 49.271854 49.271854 49.271854 53.204263 ## 33 34 35 36 37 38 39 40 ## 53.204263 53.204263 53.204263 57.136672 57.136672 57.136672 61.069080 61.069080 ## 41 42 43 44 45 46 47 48 ## 61.069080 61.069080 61.069080 68.933898 72.866307 76.798715 76.798715 76.798715 ## 49 50 ## 76.798715 80.731124 # predict(stop_dist_model, newdata = data.frame(speed = cars$speed)) This is actually equivalent to simply calling predict() on stop_dist_model without a second argument. predict(stop_dist_model) ## 1 2 3 4 5 6 7 8 ## -1.849460 -1.849460 9.947766 9.947766 13.880175 17.812584 21.744993 21.744993 ## 9 10 11 12 13 14 15 16 ## 21.744993 25.677401 25.677401 29.609810 29.609810 29.609810 29.609810 33.542219 ## 17 18 19 20 21 22 23 24 ## 33.542219 33.542219 33.542219 37.474628 37.474628 37.474628 37.474628 41.407036 ## 25 26 27 28 29 30 31 32 ## 41.407036 41.407036 45.339445 45.339445 49.271854 49.271854 49.271854 53.204263 ## 33 34 35 36 37 38 39 40 ## 53.204263 53.204263 53.204263 57.136672 57.136672 57.136672 61.069080 61.069080 ## 41 42 43 44 45 46 47 48 ## 61.069080 61.069080 61.069080 68.933898 72.866307 76.798715 76.798715 76.798715 ## 49 50 ## 76.798715 80.731124 Note that then in this case, this is the same as using fitted(). fitted(stop_dist_model) ## 1 2 3 4 5 6 7 8 ## -1.849460 -1.849460 9.947766 9.947766 13.880175 17.812584 21.744993 21.744993 ## 9 10 11 12 13 14 15 16 ## 21.744993 25.677401 25.677401 29.609810 29.609810 29.609810 29.609810 33.542219 ## 17 18 19 20 21 22 23 24 ## 33.542219 33.542219 33.542219 37.474628 37.474628 37.474628 37.474628 41.407036 ## 25 26 27 28 29 30 31 32 ## 41.407036 41.407036 45.339445 45.339445 49.271854 49.271854 49.271854 53.204263 ## 33 34 35 36 37 38 39 40 ## 53.204263 53.204263 53.204263 57.136672 57.136672 57.136672 61.069080 61.069080 ## 41 42 43 44 45 46 47 48 ## 61.069080 61.069080 61.069080 68.933898 72.866307 76.798715 76.798715 76.798715 ## 49 50 ## 76.798715 80.731124 7.5 Maximum Likelihood Estimation (MLE) Approach Recall the model, \\[ Y_i = \\beta_0 + \\beta_1 x_i + \\epsilon_i \\] where \\(\\epsilon_i \\sim N(0, \\sigma^2)\\). Then we can find the mean and variance of each \\(Y_i\\). \\[ \\text{E}[Y_i \\mid X_i = x_i] = \\beta_0 + \\beta_1 x_i \\] and \\[ \\text{Var}[Y_i \\mid X_i = x_i] = \\sigma^2. \\] Additionally, the \\(Y_i\\) follow a normal distribution conditioned on the \\(x_i\\). \\[ Y_i \\mid X_i \\sim N(\\beta_0 + \\beta_1 x_i, \\sigma^2) \\] Recall that the pdf of a random variable \\(X \\sim N(\\mu, \\sigma^2)\\) is given by \\[ f_{X}(x; \\mu, \\sigma^2) = \\frac{1}{\\sqrt{2\\pi\\sigma^2}} \\exp{\\left[-\\frac{1}{2}\\left(\\frac{x - \\mu}{\\sigma}\\right)^2\\right]}. \\] Then we can write the pdf of each of the \\(Y_i\\) as \\[ f_{Y_i}(y_i; x_i, \\beta_0, \\beta_1, \\sigma^2) = \\frac{1}{\\sqrt{2\\pi\\sigma^2}} \\exp{\\left[-\\frac{1}{2}\\left(\\frac{y_i - (\\beta_0 + \\beta_1 x_i)}{\\sigma}\\right)^2\\right]}. \\] Given \\(n\\) data points \\((x_i, y_i)\\) we can write the likelihood, which is a function of the three parameters \\(\\beta_0\\), \\(\\beta_1\\), and \\(\\sigma^2\\). Since the data have been observed, we use lower case \\(y_i\\) to denote that these values are no longer random. \\[ L(\\beta_0, \\beta_1, \\sigma^2) = \\prod_{i = 1}^{n} \\frac{1}{\\sqrt{2\\pi\\sigma^2}} \\exp{\\left[-\\frac{1}{2}\\left(\\frac{y_i - \\beta_0 - \\beta_1 x_i}{\\sigma}\\right)^2\\right]} \\] Our goal is to find values of \\(\\beta_0\\), \\(\\beta_1\\), and \\(\\sigma^2\\) which maximize this function, which is a straightforward multivariate calculus problem. We’ll start by doing a bit of rearranging to make our task easier. \\[ L(\\beta_0, \\beta_1, \\sigma^2) = \\left(\\frac{1}{\\sqrt{2 \\pi \\sigma^2}}\\right)^n \\exp{\\left[-\\frac{1}{2 \\sigma^2} \\sum_{i = 1}^{n} (y_i - \\beta_0 - \\beta_1 x_i)^2\\right]} \\] Then, as is often the case when finding MLEs, for mathematical convenience we will take the natural logarithm of the likelihood function since log is a monotonically increasing function. Then we will proceed to maximize the log-likelihood, and the resulting estimates will be the same as if we had not taken the log. \\[ \\log L(\\beta_0, \\beta_1, \\sigma^2) = -\\frac{n}{2}\\log(2 \\pi) - \\frac{n}{2}\\log(\\sigma^2) - \\frac{1}{2 \\sigma^2} \\sum_{i = 1}^{n} (y_i - \\beta_0 - \\beta_1 x_i)^2 \\] Note that we use \\(\\log\\) to mean the natural logarithm. We now take a partial derivative with respect to each of the parameters. \\[ \\begin{aligned} \\frac{\\partial \\log L(\\beta_0, \\beta_1, \\sigma^2)}{\\partial \\beta_0} &amp;= \\frac{1}{\\sigma^2} \\sum_{i = 1}^{n} (y_i - \\beta_0 - \\beta_1 x_i)\\\\ \\frac{\\partial \\log L(\\beta_0, \\beta_1, \\sigma^2)}{\\partial \\beta_1} &amp;= \\frac{1}{\\sigma^2} \\sum_{i = 1}^{n}(x_i)(y_i - \\beta_0 - \\beta_1 x_i) \\\\ \\frac{\\partial \\log L(\\beta_0, \\beta_1, \\sigma^2)}{\\partial \\sigma^2} &amp;= -\\frac{n}{2 \\sigma^2} + \\frac{1}{2(\\sigma^2)^2} \\sum_{i = 1}^{n} (y_i - \\beta_0 - \\beta_1 x_i)^2 \\end{aligned} \\] We then set each of the partial derivatives equal to zero and solve the resulting system of equations. \\[ \\begin{aligned} \\sum_{i = 1}^{n} (y_i - \\beta_0 - \\beta_1 x_i) &amp;= 0\\\\ \\sum_{i = 1}^{n}(x_i)(y_i - \\beta_0 - \\beta_1 x_i) &amp;= 0\\\\ -\\frac{n}{2 \\sigma^2} + \\frac{1}{2(\\sigma^2)^2} \\sum_{i = 1}^{n} (y_i - \\beta_0 - \\beta_1 x_i)^2 &amp;= 0 \\end{aligned} \\] You may notice that the first two equations also appear in the least squares approach. Then, skipping the issue of actually checking if we have found a maximum, we then arrive at our estimates. We call these estimates the maximum likelihood estimates. \\[ \\begin{aligned} \\hat{\\beta}_1 &amp;= \\frac{\\sum_{i = 1}^{n} x_i y_i - \\frac{(\\sum_{i = 1}^{n} x_i)(\\sum_{i = 1}^{n} y_i)}{n}}{\\sum_{i = 1}^{n} x_i^2 - \\frac{(\\sum_{i = 1}^{n} x_i)^2}{n}} = \\frac{S_{xy}}{S_{xx}}\\\\ \\hat{\\beta}_0 &amp;= \\bar{y} - \\hat{\\beta}_1 \\bar{x}\\\\ \\hat{\\sigma}^2 &amp;= \\frac{1}{n} \\sum_{i = 1}^{n}(y_i - \\hat{y}_i)^2 \\end{aligned} \\] Note that \\(\\hat{\\beta}_0\\) and \\(\\hat{\\beta}_1\\) are the same as the least squares estimates. However we now have a new estimate of \\(\\sigma^2\\), that is \\(\\hat{\\sigma}^2\\). So we now have two different estimates of \\(\\sigma^2\\). \\[ \\begin{aligned} s_e^2 &amp;= \\frac{1}{n - 2} \\sum_{i = 1}^{n}(y_i - \\hat{y}_i)^2 = \\frac{1}{n - 2} \\sum_{i = 1}^{n}e_i^2 &amp; \\text{Least Squares}\\\\ \\hat{\\sigma}^2 &amp;= \\frac{1}{n} \\sum_{i = 1}^{n}(y_i - \\hat{y}_i)^2 = \\frac{1}{n} \\sum_{i = 1}^{n}e_i^2 &amp; \\text{MLE} \\end{aligned} \\] In the next chapter, we will discuss in detail the difference between these two estimates, which involves biasedness. 7.6 Simulating SLR We return again to more examples of simulation. This will be a common theme! In practice you will almost never have a true model, and you will use data to attempt to recover information about the unknown true model. With simulation, we decide the true model and simulate data from it. Then, we apply a method to the data, in this case least squares. Now, since we know the true model, we can assess how well it did. For this example, we will simulate \\(n = 21\\) observations from the model \\[ Y = 5 - 2 x + \\epsilon. \\] That is \\(\\beta_0 = 5\\), \\(\\beta_1 = -2\\), and let \\(\\epsilon \\sim N(\\mu = 0, \\sigma^2 = 9)\\). Or, even more succinctly we could write \\[ Y \\mid X \\sim N(\\mu = 5 - 2 x, \\sigma^ 2 = 9). \\] We first set the true parameters of the model to be simulated. num_obs = 21 beta_0 = 5 beta_1 = -2 sigma = 3 Next, we obtain simulated values of \\(\\epsilon_i\\) after setting a seed for reproducibility. set.seed(1) epsilon = rnorm(n = num_obs, mean = 0, sd = sigma) Now, since the \\(x_i\\) values in SLR are considered fixed and known, we simply specify 21 values. Another common practice is to generate them from a uniform distribution, and then use them for the remainder of the analysis. x_vals = seq(from = 0, to = 10, length.out = num_obs) # set.seed(1) # x_vals = runif(num_obs, 0, 10) We then generate the \\(y\\) values according to the specified functional relationship. y_vals = beta_0 + beta_1 * x_vals + epsilon The data, \\((x_i, y_i)\\), represent a possible sample from the true distribution. Now to check how well the method of least squares works, we use lm() to fit the model to our simulated data, then take a look at the estimated coefficients. sim_fit = lm(y_vals ~ x_vals) coef(sim_fit) ## (Intercept) x_vals ## 4.832639 -1.831401 And look at that, they aren’t too far from the true parameters we specified! plot(y_vals ~ x_vals) abline(sim_fit) We should say here, that we’re being sort of lazy, and not the good kind of lazy that could be considered efficient. Any time you simulate data, you should consider doing two things: writing a function, and storing the data in a data frame. The function below, sim_slr(), can be used for the same task as above, but is much more flexible. Notice that we provide x to the function, instead of generating x inside the function. In the SLR model, the \\(x_i\\) are considered known values. That is, they are not random, so we do not assume a distribution for the \\(x_i\\). Because of this, we will repeatedly use the same x values across all simulations. sim_slr = function(x, beta_0 = 10, beta_1 = 5, sigma = 1) { n = length(x) epsilon = rnorm(n, mean = 0, sd = sigma) y = beta_0 + beta_1 * x + epsilon data.frame(predictor = x, response = y) } Here, we use the function to repeat the analysis above. set.seed(1) sim_data = sim_slr(x = x_vals, beta_0 = 5, beta_1 = -2, sigma = 3) This time, the simulated observations are stored in a data frame. head(sim_data) ## predictor response ## 1 0.0 3.1206386 ## 2 0.5 4.5509300 ## 3 1.0 0.4931142 ## 4 1.5 6.7858424 ## 5 2.0 1.9885233 ## 6 2.5 -2.4614052 Now when we fit the model with lm() we can use a data argument, a very good practice. sim_fit = lm(response ~ predictor, data = sim_data) coef(sim_fit) ## (Intercept) predictor ## 4.832639 -1.831401 And this time, we’ll make the plot look a lot nicer. plot(response ~ predictor, data = sim_data, xlab = &quot;Simulated Predictor Variable&quot;, ylab = &quot;Simulated Response Variable&quot;, main = &quot;Simulated Regression Data&quot;, pch = 20, cex = 2, col = &quot;grey&quot;) abline(sim_fit, lwd = 3, lty = 1, col = &quot;darkorange&quot;) abline(beta_0, beta_1, lwd = 3, lty = 2, col = &quot;dodgerblue&quot;) legend(&quot;topright&quot;, c(&quot;Estimate&quot;, &quot;Truth&quot;), lty = c(1, 2), lwd = 2, col = c(&quot;darkorange&quot;, &quot;dodgerblue&quot;)) 7.7 History For some brief background on the history of linear regression, see “Galton, Pearson, and the Peas: A Brief History of Linear Regression for Statistics Instructors” from the Journal of Statistics Education as well as the Wikipedia page on the history of regression analysis and lastly the article for regression toward the mean which details the origins of the term “regression.” 7.8 R Markdown The R Markdown file for this chapter can be found here: slr.Rmd The file was created using R version 4.4.1. "],["inference-for-simple-linear-regression.html", "Chapter 8 Inference for Simple Linear Regression 8.1 Gauss–Markov Theorem 8.2 Sampling Distributions 8.3 Standard Errors 8.4 Confidence Intervals for Slope and Intercept 8.5 Hypothesis Tests 8.6 cars Example 8.7 Confidence Interval for Mean Response 8.8 Prediction Interval for New Observations 8.9 Confidence and Prediction Bands 8.10 Significance of Regression, F-Test 8.11 R Markdown", " Chapter 8 Inference for Simple Linear Regression “There are three types of lies: lies, damn lies, and statistics.” — Benjamin Disraeli After reading this chapter you will be able to: Understand the distributions of regression estimates. Create interval estimates for regression parameters, mean response, and predictions. Test for significance of regression. Last chapter we defined the simple linear regression model, \\[ Y_i = \\beta_0 + \\beta_1 x_i + \\epsilon_i \\] where \\(\\epsilon_i \\sim N(0, \\sigma^2)\\). We then used observations \\((x_i, y_i)\\), for \\(i = 1, 2, \\ldots n\\), to find values of \\(\\beta_0\\) and \\(\\beta_1\\) which minimized \\[ f(\\beta_0, \\beta_1) = \\sum_{i = 1}^{n}(y_i - (\\beta_0 + \\beta_1 x_i))^2. \\] We called these values \\(\\hat{\\beta}_0\\) and \\(\\hat{\\beta}_1\\), which we found to be \\[ \\begin{aligned} \\hat{\\beta}_1 &amp;= \\frac{S_{xy}}{S_{xx}} = \\frac{\\sum_{i = 1}^{n}(x_i - \\bar{x})(y_i - \\bar{y})}{\\sum_{i = 1}^{n}(x_i - \\bar{x})^2}\\\\ \\hat{\\beta}_0 &amp;= \\bar{y} - \\hat{\\beta}_1 \\bar{x}. \\end{aligned} \\] We also estimated \\(\\sigma ^2\\) using \\(s_e^2\\). In other words, we found that \\(s_e\\) is an estimate of \\(\\sigma\\), where; \\[ s_e = \\text{RSE} = \\sqrt{\\frac{1}{n - 2}\\sum_{i = 1}^n e_i^2} \\] which we also called \\(\\text{RSE}\\), for “Residual Standard Error.” When applied to the cars data, we obtained the following results: stop_dist_model = lm(dist ~ speed, data = cars) summary(stop_dist_model) ## ## Call: ## lm(formula = dist ~ speed, data = cars) ## ## Residuals: ## Min 1Q Median 3Q Max ## -29.069 -9.525 -2.272 9.215 43.201 ## ## Coefficients: ## Estimate Std. Error t value Pr(&gt;|t|) ## (Intercept) -17.5791 6.7584 -2.601 0.0123 * ## speed 3.9324 0.4155 9.464 1.49e-12 *** ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 ## ## Residual standard error: 15.38 on 48 degrees of freedom ## Multiple R-squared: 0.6511, Adjusted R-squared: 0.6438 ## F-statistic: 89.57 on 1 and 48 DF, p-value: 1.49e-12 Last chapter, we only discussed the Estimate, Residual standard error, and Multiple R-squared values. In this chapter, we will discuss all of the information under Coefficients as well as F-statistic. plot(dist ~ speed, data = cars, xlab = &quot;Speed (in Miles Per Hour)&quot;, ylab = &quot;Stopping Distance (in Feet)&quot;, main = &quot;Stopping Distance vs Speed&quot;, pch = 20, cex = 2, col = &quot;grey&quot;) abline(stop_dist_model, lwd = 5, col = &quot;darkorange&quot;) To get started, we’ll note that there is another equivalent expression for \\(S_{xy}\\) which we did not see last chapter, \\[ S_{xy}= \\sum_{i = 1}^{n}(x_i - \\bar{x})(y_i - \\bar{y}) = \\sum_{i = 1}^{n}(x_i - \\bar{x}) y_i. \\] This may be a surprising equivalence. (Maybe try to prove it.) However, it will be useful for illustrating concepts in this chapter. Note that \\(\\hat{\\beta}_1\\) is a sample statistic when calculated with observed data as written above, as is \\(\\hat{\\beta}_0\\). However, in this chapter it will often be convenient to use both \\(\\hat{\\beta}_1\\) and \\(\\hat{\\beta}_0\\) as random variables, that is, we have not yet observed the values for each \\(Y_i\\). When this is the case, we will use a slightly different notation, substituting in capital \\(Y_i\\) for lower case \\(y_i\\). \\[ \\begin{aligned} \\hat{\\beta}_1 &amp;= \\frac{\\sum_{i = 1}^{n}(x_i - \\bar{x}) Y_i}{\\sum_{i = 1}^{n}(x_i - \\bar{x})^2} \\\\ \\hat{\\beta}_0 &amp;= \\bar{Y} - \\hat{\\beta}_1 \\bar{x} \\end{aligned} \\] Last chapter we argued that these estimates of unknown model parameters \\(\\beta_0\\) and \\(\\beta_1\\) were good because we obtained them by minimizing errors. We will now discuss the Gauss–Markov theorem, which takes this idea further, showing that these estimates are actually the “best” estimates, from a certain point of view. 8.1 Gauss–Markov Theorem The Gauss–Markov theorem tells us that when estimating the parameters of the simple linear regression model \\(\\beta_0\\) and \\(\\beta_1\\), the \\(\\hat{\\beta}_0\\) and \\(\\hat{\\beta}_1\\) which we derived are the best linear unbiased estimates, or BLUE for short. (The actual conditions for the Gauss–Markov theorem are more relaxed than the SLR model.) We will now discuss linear, unbiased, and best as they relate to these estimates. Linear Recall, in the SLR setup that the \\(x_i\\) values are considered fixed and known quantities. Then a linear estimate is one which can be written as a linear combination of the \\(Y_i\\). In the case of \\(\\hat{\\beta}_1\\) we see \\[ \\hat{\\beta}_1 = \\frac{\\sum_{i = 1}^{n}(x_i - \\bar{x}) Y_i}{\\sum_{i = 1}^{n}(x_i - \\bar{x})^2} = \\sum_{i = 1}^n k_i Y_i = k_1 Y_1 + k_2 Y_2 + \\cdots k_n Y_n \\] where \\(k_i = \\displaystyle\\frac{(x_i - \\bar{x})}{\\sum_{i = 1}^{n}(x_i - \\bar{x})^2}\\). In a similar fashion, we could show that \\(\\hat{\\beta}_0\\) can be written as a linear combination of the \\(Y_i\\). Thus both \\(\\hat{\\beta}_0\\) and \\(\\hat{\\beta}_1\\) are linear estimators. Unbiased Now that we know our estimates are linear, how good are these estimates? One measure of the “goodness” of an estimate is its bias. Specifically, we prefer estimates that are unbiased, meaning their expected value is the parameter being estimated. In the case of the regression estimates, we have, \\[ \\begin{aligned} \\text{E}[\\hat{\\beta}_0] &amp;= \\beta_0 \\\\ \\text{E}[\\hat{\\beta}_1] &amp;= \\beta_1. \\end{aligned} \\] This tells us that, when the conditions of the SLR model are met, on average our estimates will be correct. However, as we saw last chapter when simulating from the SLR model, that does not mean that each individual estimate will be correct. Only that, if we repeated the process an infinite number of times, on average the estimate would be correct. Best Now, if we restrict ourselves to both linear and unbiased estimates, how do we define the best estimate? The estimate with the minimum variance. First note that it is very easy to create an estimate for \\(\\beta_1\\) that has very low variance, but is not unbiased. For example, define: \\[ \\hat{\\theta}_{BAD} = 5. \\] Then, since \\(\\hat{\\theta}_{BAD}\\) is a constant value, \\[ \\text{Var}[\\hat{\\theta}_{BAD}] = 0. \\] However since, \\[ \\text{E}[\\hat{\\theta}_{BAD}] = 5 \\] we say that \\(\\hat{\\theta}_{BAD}\\) is a biased estimator unless \\(\\beta_1 = 5\\), which we would not know ahead of time. For this reason, it is a terrible estimate (unless by chance \\(\\beta_1 = 5\\)) even though it has the smallest possible variance. This is part of the reason we restrict ourselves to unbiased estimates. What good is an estimate, if it estimates the wrong quantity? So now, the natural question is, what are the variances of \\(\\hat{\\beta}_0\\) and \\(\\hat{\\beta}_1\\)? They are, \\[ \\begin{aligned} \\text{Var}[\\hat{\\beta}_0] &amp;= \\sigma^2 \\left(\\frac{1}{n} + \\frac{\\bar{x}^2}{S_{xx}}\\right) \\\\ \\text{Var}[\\hat{\\beta}_1] &amp;= \\frac{\\sigma^2}{S_{xx}}. \\end{aligned} \\] These quantify the variability of the estimates due to random chance during sampling. Are these “the best”? Are these variances as small as we can possibly get? You’ll just have to take our word for it that they are because showing that this is true is beyond the scope of this course. 8.2 Sampling Distributions Now that we have “redefined” the estimates for \\(\\hat{\\beta}_0\\) and \\(\\hat{\\beta}_1\\) as random variables, we can discuss their sampling distribution, which is the distribution when a statistic is considered a random variable. Since both \\(\\hat{\\beta}_0\\) and \\(\\hat{\\beta}_1\\) are a linear combination of the \\(Y_i\\) and each \\(Y_i\\) is normally distributed, then both \\(\\hat{\\beta}_0\\) and \\(\\hat{\\beta}_1\\) also follow a normal distribution. Then, putting all of the above together, we arrive at the distributions of \\(\\hat{\\beta}_0\\) and \\(\\hat{\\beta}_1\\). For \\(\\hat{\\beta}_1\\) we say, \\[ \\hat{\\beta}_1 = \\frac{S_{xy}}{S_{xx}} = \\frac{\\sum_{i = 1}^{n}(x_i - \\bar{x}) Y_i}{\\sum_{i = 1}^{n}(x_i - \\bar{x})^2} \\sim N\\left( \\beta_1, \\ \\frac{\\sigma^2}{\\sum_{i = 1}^{n}(x_i - \\bar{x})^2} \\right). \\] Or more succinctly, \\[ \\hat{\\beta}_1 \\sim N\\left( \\beta_1, \\frac{\\sigma^2}{S_{xx}} \\right). \\] And for \\(\\hat{\\beta}_0\\), \\[ \\hat{\\beta}_0 = \\bar{Y} - \\hat{\\beta}_1 \\bar{x} \\sim N\\left( \\beta_0, \\ \\frac{\\sigma^2 \\sum_{i = 1}^{n}x_i^2}{n \\sum_{i = 1}^{n}(x_i - \\bar{x})^2} \\right). \\] Or more succinctly, \\[ \\hat{\\beta}_0 \\sim N\\left( \\beta_0, \\sigma^2 \\left(\\frac{1}{n} + \\frac{\\bar{x}^2}{S_{xx}}\\right) \\right) \\] At this point we have neglected to prove a number of these results. Instead of working through the tedious derivations of these sampling distributions, we will instead justify these results to ourselves using simulation. A note to current readers: These derivations and proofs may be added to an appendix at a later time. You can also find these results in nearly any standard linear regression textbook. At UIUC, these results will likely be presented in both STAT 424 and STAT 425. However, since you will not be asked to perform derivations of this type in this course, they are for now omitted. 8.2.1 Simulating Sampling Distributions To verify the above results, we will simulate samples of size \\(n = 100\\) from the model \\[ Y_i = \\beta_0 + \\beta_1 x_i + \\epsilon_i \\] where \\(\\epsilon_i \\sim N(0, \\sigma^2).\\) In this case, the parameters are known to be: \\(\\beta_0 = 3\\) \\(\\beta_1 = 6\\) \\(\\sigma^2 = 4\\) Then, based on the above, we should find that \\[ \\hat{\\beta}_1 \\sim N\\left( \\beta_1, \\frac{\\sigma^2}{S_{xx}} \\right) \\] and \\[ \\hat{\\beta}_0 \\sim N\\left( \\beta_0, \\sigma^2 \\left(\\frac{1}{n} + \\frac{\\bar{x}^2}{S_{xx}}\\right) \\right). \\] First we need to decide ahead of time what our \\(x\\) values will be for this simulation, since the \\(x\\) values in SLR are also considered known quantities. The choice of \\(x\\) values is arbitrary. Here we also set a seed for randomization, and calculate \\(S_{xx}\\) which we will need going forward. set.seed(42) sample_size = 100 # this is n x = seq(-1, 1, length = sample_size) Sxx = sum((x - mean(x)) ^ 2) We also fix our parameter values. beta_0 = 3 beta_1 = 6 sigma = 2 With this information, we know the sampling distributions should be: (var_beta_1_hat = sigma ^ 2 / Sxx) ## [1] 0.1176238 (var_beta_0_hat = sigma ^ 2 * (1 / sample_size + mean(x) ^ 2 / Sxx)) ## [1] 0.04 \\[ \\hat{\\beta}_1 \\sim N( 6, 0.1176238) \\] and \\[ \\hat{\\beta}_0 \\sim N( 3, 0.04). \\] That is, \\[ \\begin{aligned} \\text{E}[\\hat{\\beta}_1] &amp;= 6 \\\\ \\text{Var}[\\hat{\\beta}_1] &amp;= 0.1176238 \\end{aligned} \\] and \\[ \\begin{aligned} \\text{E}[\\hat{\\beta}_0] &amp;= 3 \\\\ \\text{Var}[\\hat{\\beta}_0] &amp;= 0.04. \\end{aligned} \\] We now simulate data from this model 10,000 times. Note this may not be the most R way of doing the simulation. We perform the simulation in this manner in an attempt at clarity. For example, we could have used the sim_slr() function from the previous chapter. We also simply store variables in the global environment instead of creating a data frame for each new simulated dataset. num_samples = 10000 beta_0_hats = rep(0, num_samples) beta_1_hats = rep(0, num_samples) for (i in 1:num_samples) { eps = rnorm(sample_size, mean = 0, sd = sigma) y = beta_0 + beta_1 * x + eps sim_model = lm(y ~ x) beta_0_hats[i] = coef(sim_model)[1] beta_1_hats[i] = coef(sim_model)[2] } Each time we simulated the data, we obtained values of the estimated coefficients. The variables beta_0_hats and beta_1_hats now store 10,000 simulated values of \\(\\hat{\\beta}_0\\) and \\(\\hat{\\beta}_1\\) respectively. We first verify the distribution of \\(\\hat{\\beta}_1\\). mean(beta_1_hats) # empirical mean ## [1] 6.001998 beta_1 # true mean ## [1] 6 var(beta_1_hats) # empirical variance ## [1] 0.11899 var_beta_1_hat # true variance ## [1] 0.1176238 We see that the empirical and true means and variances are very similar. We also verify that the empirical distribution is normal. To do so, we plot a histogram of the beta_1_hats, and add the curve for the true distribution of \\(\\hat{\\beta}_1\\). We use prob = TRUE to put the histogram on the same scale as the normal curve. # note need to use prob = TRUE hist(beta_1_hats, prob = TRUE, breaks = 20, xlab = expression(hat(beta)[1]), main = &quot;&quot;, border = &quot;dodgerblue&quot;) curve(dnorm(x, mean = beta_1, sd = sqrt(var_beta_1_hat)), col = &quot;darkorange&quot;, add = TRUE, lwd = 3) We then repeat the process for \\(\\hat{\\beta}_0\\). mean(beta_0_hats) # empirical mean ## [1] 3.001147 beta_0 # true mean ## [1] 3 var(beta_0_hats) # empirical variance ## [1] 0.04017924 var_beta_0_hat # true variance ## [1] 0.04 hist(beta_0_hats, prob = TRUE, breaks = 25, xlab = expression(hat(beta)[0]), main = &quot;&quot;, border = &quot;dodgerblue&quot;) curve(dnorm(x, mean = beta_0, sd = sqrt(var_beta_0_hat)), col = &quot;darkorange&quot;, add = TRUE, lwd = 3) In this simulation study, we have only simulated a finite number of samples. To truly verify the distributional results, we would need to observe an infinite number of samples. However, the following plot should make it clear that if we continued simulating, the empirical results would get closer and closer to what we should expect. par(mar = c(5, 5, 1, 1)) # adjusted plot margins, otherwise the &quot;hat&quot; does not display plot(cumsum(beta_1_hats) / (1:length(beta_1_hats)), type = &quot;l&quot;, ylim = c(5.95, 6.05), xlab = &quot;Number of Simulations&quot;, ylab = expression(&quot;Empirical Mean of &quot; ~ hat(beta)[1]), col = &quot;dodgerblue&quot;) abline(h = 6, col = &quot;darkorange&quot;, lwd = 2) par(mar = c(5, 5, 1, 1)) # adjusted plot margins, otherwise the &quot;hat&quot; does not display plot(cumsum(beta_0_hats) / (1:length(beta_0_hats)), type = &quot;l&quot;, ylim = c(2.95, 3.05), xlab = &quot;Number of Simulations&quot;, ylab = expression(&quot;Empirical Mean of &quot; ~ hat(beta)[0]), col = &quot;dodgerblue&quot;) abline(h = 3, col = &quot;darkorange&quot;, lwd = 2) 8.3 Standard Errors So now we believe the two distributional results, \\[ \\begin{aligned} \\hat{\\beta}_0 &amp;\\sim N\\left( \\beta_0, \\sigma^2 \\left(\\frac{1}{n} + \\frac{\\bar{x}^2}{S_{xx}}\\right) \\right) \\\\ \\hat{\\beta}_1 &amp;\\sim N\\left( \\beta_1, \\frac{\\sigma^2}{S_{xx}} \\right). \\end{aligned} \\] Then by standardizing these results we find that \\[ \\frac{\\hat{\\beta}_0 - \\beta_0}{\\text{SD}[\\hat{\\beta}_0]} \\sim N(0, 1) \\] and \\[ \\frac{\\hat{\\beta}_1 - \\beta_1}{\\text{SD}[\\hat{\\beta}_1]} \\sim N(0, 1) \\] where \\[ \\text{SD}[\\hat{\\beta}_0] = \\sigma\\sqrt{\\frac{1}{n} + \\frac{\\bar{x}^2}{S_{xx}}} \\] and \\[ \\text{SD}[\\hat{\\beta}_1] = \\frac{\\sigma}{\\sqrt{S_{xx}}}. \\] Since we don’t know \\(\\sigma\\) in practice, we will have to estimate it using \\(s_e\\), which we plug into our existing expression for the standard deviations of our estimates. These two new expressions are called standard errors which are the estimated standard deviations of the sampling distributions. \\[ \\text{SE}[\\hat{\\beta}_0] = s_e\\sqrt{\\frac{1}{n} + \\frac{\\bar{x}^2}{S_{xx}}} \\] \\[ \\text{SE}[\\hat{\\beta}_1] = \\frac{s_e}{\\sqrt{S_{xx}}} \\] Now if we divide by the standard error, instead of the standard deviation, we obtain the following results which will allow us to make confidence intervals and perform hypothesis testing. \\[ \\frac{\\hat{\\beta}_0 - \\beta_0}{\\text{SE}[\\hat{\\beta}_0]} \\sim t_{n-2} \\] \\[ \\frac{\\hat{\\beta}_1 - \\beta_1}{\\text{SE}[\\hat{\\beta}_1]} \\sim t_{n-2} \\] To see this, first note that, \\[ \\frac{\\text{RSS}}{\\sigma^2} = \\frac{(n-2)s_e^2}{\\sigma^2} \\sim \\chi_{n-2}^2. \\] Also recall that a random variable \\(T\\) defined as, \\[ T = \\frac{Z}{\\sqrt{\\frac{\\chi_{d}^2}{d}}} \\] follows a \\(t\\) distribution with \\(d\\) degrees of freedom, where \\(\\chi_{d}^2\\) is a \\(\\chi^2\\) random variable with \\(d\\) degrees of freedom. We write, \\[ T \\sim t_d \\] to say that the random variable \\(T\\) follows a \\(t\\) distribution with \\(d\\) degrees of freedom. Then we use the classic trick of “multiply by 1” and some rearranging to arrive at \\[ \\begin{aligned} \\frac{\\hat{\\beta}_1 - \\beta_1}{\\text{SE}[\\hat{\\beta}_1]} &amp;= \\frac{\\hat{\\beta}_1 - \\beta_1}{s_e / \\sqrt{S_{xx}}} \\\\ &amp;= \\frac{\\hat{\\beta}_1 - \\beta_1}{s_e / \\sqrt{S_{xx}}} \\cdot \\frac{\\sigma / \\sqrt{S_{xx}}}{\\sigma / \\sqrt{S_{xx}}} \\\\ &amp;= \\frac{\\hat{\\beta}_1 - \\beta_1}{\\sigma / \\sqrt{S_{xx}}} \\cdot \\frac{\\sigma / \\sqrt{S_{xx}}}{s_e / \\sqrt{S_{xx}}} \\\\ &amp;= \\frac{\\hat{\\beta}_1 - \\beta_1}{\\sigma / \\sqrt{S_{xx}}} \\bigg/ \\sqrt{\\frac{s_e^2}{\\sigma^2}} \\\\ &amp;= \\frac{\\hat{\\beta}_1 - \\beta_1}{\\text{SD}[\\hat{\\beta}_1]} \\bigg/ \\sqrt{\\frac{\\frac{(n - 2)s_e^2}{\\sigma^2}}{n - 2}} \\sim \\frac{Z}{\\sqrt{\\frac{\\chi_{n-2}^2}{n-2}}} \\sim t_{n-2} \\end{aligned} \\] where \\(Z \\sim N(0,1)\\). Recall that a \\(t\\) distribution is similar to a standard normal, but with heavier tails. As the degrees of freedom increases, the \\(t\\) distribution becomes more and more like a standard normal. Below we plot a standard normal distribution as well as two examples of a \\(t\\) distribution with different degrees of freedom. Notice how the \\(t\\) distribution with the larger degrees of freedom is more similar to the standard normal curve. # define grid of x values x = seq(-4, 4, length = 100) # plot curve for standard normal plot(x, dnorm(x), type = &quot;l&quot;, lty = 1, lwd = 2, xlab = &quot;x&quot;, ylab = &quot;Density&quot;, main = &quot;Normal vs t Distributions&quot;) # add curves for t distributions lines(x, dt(x, df = 1), lty = 3, lwd = 2, col = &quot;darkorange&quot;) lines(x, dt(x, df = 10), lty = 2, lwd = 2, col = &quot;dodgerblue&quot;) # add legend legend(&quot;topright&quot;, title = &quot;Distributions&quot;, legend = c(&quot;t, df = 1&quot;, &quot;t, df = 10&quot;, &quot;Standard Normal&quot;), lwd = 2, lty = c(3, 2, 1), col = c(&quot;darkorange&quot;, &quot;dodgerblue&quot;, &quot;black&quot;)) 8.4 Confidence Intervals for Slope and Intercept Recall that confidence intervals for means often take the form: \\[ \\text{EST} \\pm \\text{CRIT} \\cdot \\text{SE} \\] or \\[ \\text{EST} \\pm \\text{MARGIN} \\] where \\(\\text{EST}\\) is an estimate for the parameter of interest, \\(\\text{SE}\\) is the standard error of the estimate, and \\(\\text{MARGIN} = \\text{CRIT} \\cdot \\text{SE}\\). Then, for \\(\\beta_0\\) and \\(\\beta_1\\) we can create confidence intervals using \\[ \\hat{\\beta}_0 \\pm t_{\\alpha/2, n - 2} \\cdot \\text{SE}[\\hat{\\beta}_0] \\quad \\quad \\quad \\hat{\\beta}_0 \\pm t_{\\alpha/2, n - 2} \\cdot s_e\\sqrt{\\frac{1}{n}+\\frac{\\bar{x}^2}{S_{xx}}} \\] and \\[ \\hat{\\beta}_1 \\pm t_{\\alpha/2, n - 2} \\cdot \\text{SE}[\\hat{\\beta}_1] \\quad \\quad \\quad \\hat{\\beta}_1 \\pm t_{\\alpha/2, n - 2} \\cdot \\frac{s_e}{\\sqrt{S_{xx}}} \\] where \\(t_{\\alpha/2, n - 2}\\) is the critical value such that \\(P(t_{n-2} &gt; t_{\\alpha/2, n - 2}) = \\alpha/2\\). 8.5 Hypothesis Tests “We may speak of this hypothesis as the ‘null hypothesis,’ and it should be noted that the null hypothesis is never proved or established, but is possibly disproved, in the course of experimentation.” — Ronald Aylmer Fisher Recall that a test statistic (\\(\\text{TS}\\)) for testing means often takes the form: \\[ \\text{TS} = \\frac{\\text{EST} - \\text{HYP}}{\\text{SE}} \\] where \\(\\text{EST}\\) is an estimate for the parameter of interest, \\(\\text{HYP}\\) is a hypothesized value of the parameter, and \\(\\text{SE}\\) is the standard error of the estimate. So, to test \\[ H_0: \\beta_0 = \\beta_{00} \\quad \\text{vs} \\quad H_1: \\beta_0 \\neq \\beta_{00} \\] we use the test statistic \\[ t = \\frac{\\hat{\\beta}_0 - \\beta_{00}}{\\text{SE}[\\hat{\\beta}_0]} = \\frac{\\hat{\\beta}_0-\\beta_{00}}{s_e\\sqrt{\\frac{1}{n} + \\frac{\\bar{x}^2}{S_{xx}}}} \\] which, under the null hypothesis, follows a \\(t\\) distribution with \\(n - 2\\) degrees of freedom. We use \\(\\beta_{00}\\) to denote the hypothesized value of \\(\\beta_0\\). Similarly, to test \\[ H_0: \\beta_1 = \\beta_{10} \\quad \\text{vs} \\quad H_1: \\beta_1 \\neq \\beta_{10} \\] we use the test statistic \\[ t = \\frac{\\hat{\\beta}_1-\\beta_{10}}{\\text{SE}[\\hat{\\beta}_1]} = \\frac{\\hat{\\beta}_1-\\beta_{10}}{s_e / \\sqrt{S_{xx}}} \\] which again, under the null hypothesis, follows a \\(t\\) distribution with \\(n - 2\\) degrees of freedom. We now use \\(\\beta_{10}\\) to denote the hypothesized value of \\(\\beta_1\\). 8.6 cars Example We now return to the cars example from last chapter to illustrate these concepts. We first fit the model using lm() then use summary() to view the results in greater detail. stop_dist_model = lm(dist ~ speed, data = cars) summary(stop_dist_model) ## ## Call: ## lm(formula = dist ~ speed, data = cars) ## ## Residuals: ## Min 1Q Median 3Q Max ## -29.069 -9.525 -2.272 9.215 43.201 ## ## Coefficients: ## Estimate Std. Error t value Pr(&gt;|t|) ## (Intercept) -17.5791 6.7584 -2.601 0.0123 * ## speed 3.9324 0.4155 9.464 1.49e-12 *** ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 ## ## Residual standard error: 15.38 on 48 degrees of freedom ## Multiple R-squared: 0.6511, Adjusted R-squared: 0.6438 ## F-statistic: 89.57 on 1 and 48 DF, p-value: 1.49e-12 8.6.1 Tests in R We will now discuss the results displayed called Coefficients. First recall that we can extract this information directly. names(summary(stop_dist_model)) ## [1] &quot;call&quot; &quot;terms&quot; &quot;residuals&quot; &quot;coefficients&quot; ## [5] &quot;aliased&quot; &quot;sigma&quot; &quot;df&quot; &quot;r.squared&quot; ## [9] &quot;adj.r.squared&quot; &quot;fstatistic&quot; &quot;cov.unscaled&quot; summary(stop_dist_model)$coefficients ## Estimate Std. Error t value Pr(&gt;|t|) ## (Intercept) -17.579095 6.7584402 -2.601058 1.231882e-02 ## speed 3.932409 0.4155128 9.463990 1.489836e-12 The names() function tells us what information is available, and then we use the $ operator and coefficients to extract the information we are interested in. Two values here should be immediately familiar. \\[ \\hat{\\beta}_0 = -17.5790949 \\] and \\[ \\hat{\\beta}_1 = 3.9324088 \\] which are our estimates for the model parameters \\(\\beta_0\\) and \\(\\beta_1\\). Let’s now focus on the second row of output, which is relevant to \\(\\beta_1\\). summary(stop_dist_model)$coefficients[2,] ## Estimate Std. Error t value Pr(&gt;|t|) ## 3.932409e+00 4.155128e-01 9.463990e+00 1.489836e-12 Again, the first value, Estimate is \\[ \\hat{\\beta}_1 = 3.9324088. \\] The second value, Std. Error, is the standard error of \\(\\hat{\\beta}_1\\), \\[ \\text{SE}[\\hat{\\beta}_1] = \\frac{s_e}{\\sqrt{S_{xx}}} = 0.4155128. \\] The third value, t value, is the value of the test statistic for testing \\(H_0: \\beta_1 = 0\\) vs \\(H_1: \\beta_1 \\neq 0\\), \\[ t = \\frac{\\hat{\\beta}_1-0}{\\text{SE}[\\hat{\\beta}_1]} = \\frac{\\hat{\\beta}_1-0}{s_e / \\sqrt{S_{xx}}} = 9.46399. \\] Lastly, Pr(&gt;|t|), gives us the p-value of that test. \\[ \\text{p-value} = 1.4898365\\times 10^{-12} \\] Note here, we are specifically testing whether or not \\(\\beta_1 = 0\\). The first row of output reports the same values, but for \\(\\beta_0\\). summary(stop_dist_model)$coefficients[1,] ## Estimate Std. Error t value Pr(&gt;|t|) ## -17.57909489 6.75844017 -2.60105800 0.01231882 In summary, the following code stores the information of summary(stop_dist_model)$coefficients in a new variable stop_dist_model_test_info, then extracts each element into a new variable which describes the information it contains. stop_dist_model_test_info = summary(stop_dist_model)$coefficients beta_0_hat = stop_dist_model_test_info[1, 1] # Estimate beta_0_hat_se = stop_dist_model_test_info[1, 2] # Std. Error beta_0_hat_t = stop_dist_model_test_info[1, 3] # t value beta_0_hat_pval = stop_dist_model_test_info[1, 4] # Pr(&gt;|t|) beta_1_hat = stop_dist_model_test_info[2, 1] # Estimate beta_1_hat_se = stop_dist_model_test_info[2, 2] # Std. Error beta_1_hat_t = stop_dist_model_test_info[2, 3] # t value beta_1_hat_pval = stop_dist_model_test_info[2, 4] # Pr(&gt;|t|) We can then verify some equivalent expressions: the \\(t\\) test statistic for \\(\\hat{\\beta}_1\\) and the two-sided p-value associated with that test statistic. (beta_1_hat - 0) / beta_1_hat_se ## [1] 9.46399 beta_1_hat_t ## [1] 9.46399 2 * pt(abs(beta_1_hat_t), df = length(resid(stop_dist_model)) - 2, lower.tail = FALSE) ## [1] 1.489836e-12 beta_1_hat_pval ## [1] 1.489836e-12 8.6.2 Significance of Regression, t-Test We pause to discuss the significance of regression test. First, note that based on the above distributional results, we could test \\(\\beta_0\\) and \\(\\beta_1\\) against any particular value, and perform both one and two-sided tests. However, one very specific test, \\[ H_0: \\beta_1 = 0 \\quad \\text{vs} \\quad H_1: \\beta_1 \\neq 0 \\] is used most often. Let’s think about this test in terms of the simple linear regression model, \\[ Y_i = \\beta_0 + \\beta_1 x_i + \\epsilon_i. \\] If we assume the null hypothesis is true, then \\(\\beta_1 = 0\\) and we have the model, \\[ Y_i = \\beta_0 + \\epsilon_i. \\] In this model, the response does not depend on the predictor. So then we could think of this test in the following way, Under \\(H_0\\) there is not a significant linear relationship between \\(x\\) and \\(y\\). Under \\(H_1\\) there is a significant linear relationship between \\(x\\) and \\(y\\). For the cars example, Under \\(H_0\\) there is not a significant linear relationship between speed and stopping distance. Under \\(H_1\\) there is a significant linear relationship between speed and stopping distance. Again, that test is seen in the output from summary(), \\[ \\text{p-value} = 1.4898365\\times 10^{-12}. \\] With this extremely low p-value, we would reject the null hypothesis at any reasonable \\(\\alpha\\) level, say for example \\(\\alpha = 0.01\\). So we say there is a significant linear relationship between speed and stopping distance. Notice that we emphasize linear. In this plot of simulated data, we see a clear relationship between \\(x\\) and \\(y\\), however it is not a linear relationship. If we fit a line to this data, it is very flat. The resulting test for \\(H_0: \\beta_1 = 0\\) vs \\(H_1: \\beta_1 \\neq 0\\) gives a large p-value, in this case \\(0.7564548\\), so we would fail to reject and say that there is no significant linear relationship between \\(x\\) and \\(y\\). We will see later how to fit a curve to this data using a “linear” model, but for now, realize that testing \\(H_0: \\beta_1 = 0\\) vs \\(H_1: \\beta_1 \\neq 0\\) can only detect straight line relationships. 8.6.3 Confidence Intervals in R Using R we can very easily obtain the confidence intervals for \\(\\beta_0\\) and \\(\\beta_1\\). confint(stop_dist_model, level = 0.99) ## 0.5 % 99.5 % ## (Intercept) -35.706610 0.5484205 ## speed 2.817919 5.0468988 This automatically calculates 99% confidence intervals for both \\(\\beta_0\\) and \\(\\beta_1\\), the first row for \\(\\beta_0\\), the second row for \\(\\beta_1\\). For the cars example when interpreting these intervals, we say, we are 99% confident that for an increase in speed of 1 mile per hour, the average increase in stopping distance is between 2.8179187 and 5.0468988 feet, which is the interval for \\(\\beta_1\\). Note that this 99% confidence interval does not contain the hypothesized value of 0. Since it does not contain 0, it is equivalent to rejecting the test of \\(H_0: \\beta_1 = 0\\) vs \\(H_1: \\beta_1 \\neq 0\\) at \\(\\alpha = 0.01\\), which we had seen previously. You should be somewhat suspicious of the confidence interval for \\(\\beta_0\\), as it covers negative values, which correspond to negative stopping distances. Technically the interpretation would be that we are 99% confident that the average stopping distance of a car traveling 0 miles per hour is between -35.7066103 and 0.5484205 feet, but we don’t really believe that, since we are actually certain that it would be non-negative. Note, we can extract specific values from this output a number of ways. This code is not run, and instead, you should check how it relates to the output of the code above. confint(stop_dist_model, level = 0.99)[1,] confint(stop_dist_model, level = 0.99)[1, 1] confint(stop_dist_model, level = 0.99)[1, 2] confint(stop_dist_model, parm = &quot;(Intercept)&quot;, level = 0.99) confint(stop_dist_model, level = 0.99)[2,] confint(stop_dist_model, level = 0.99)[2, 1] confint(stop_dist_model, level = 0.99)[2, 2] confint(stop_dist_model, parm = &quot;speed&quot;, level = 0.99) We can also verify that calculations that R is performing for the \\(\\beta_1\\) interval. # store estimate beta_1_hat = coef(stop_dist_model)[2] # store standard error beta_1_hat_se = summary(stop_dist_model)$coefficients[2, 2] # calculate critical value for two-sided 99% CI crit = qt(0.995, df = length(resid(stop_dist_model)) - 2) # est - margin, est + margin c(beta_1_hat - crit * beta_1_hat_se, beta_1_hat + crit * beta_1_hat_se) ## speed speed ## 2.817919 5.046899 8.7 Confidence Interval for Mean Response In addition to confidence intervals for \\(\\beta_0\\) and \\(\\beta_1\\), there are two other common interval estimates used with regression. The first is called a confidence interval for the mean response. Often, we would like an interval estimate for the mean, \\(E[Y \\mid X = x]\\) for a particular value of \\(x\\). In this situation we use \\(\\hat{y}(x)\\) as our estimate of \\(E[Y \\mid X = x]\\). We modify our notation slightly to make it clear that the predicted value is a function of the \\(x\\) value. \\[ \\hat{y}(x) = \\hat{\\beta}_0 + \\hat{\\beta}_1 x \\] Recall that, \\[ \\text{E}[Y \\mid X = x] = \\beta_0 + \\beta_1 x. \\] Thus, \\(\\hat{y}(x)\\) is a good estimate since it is unbiased: \\[ \\text{E}[\\hat{y}(x)] = \\beta_0 + \\beta_1 x. \\] We could then derive, \\[ \\text{Var}[\\hat{y}(x)] = \\sigma^2 \\left(\\frac{1}{n}+\\frac{(x-\\bar{x})^2}{S_{xx}}\\right). \\] Like the other estimates we have seen, \\(\\hat{y}(x)\\) also follows a normal distribution. Since \\(\\hat{\\beta}_0\\) and \\(\\hat{\\beta}_1\\) are linear combinations of normal random variables, \\(\\hat{y}(x)\\) is as well. \\[ \\hat{y}(x) \\sim N \\left(\\beta_0 + \\beta_1 x, \\sigma^2 \\left(\\frac{1}{n}+\\frac{(x-\\bar{x})^2}{S_{xx}}\\right) \\right) \\] And lastly, since we need to estimate this variance, we arrive at the standard error of our estimate, \\[ \\text{SE}[\\hat{y}(x)] = s_e \\sqrt{\\frac{1}{n}+\\frac{(x-\\bar{x})^2}{S_{xx}}}. \\] We can then use this to find the confidence interval for the mean response, \\[ \\hat{y}(x) \\pm t_{\\alpha/2, n - 2} \\cdot s_e\\sqrt{\\frac{1}{n}+\\frac{(x-\\bar{x})^2}{S_{xx}}} \\] To find confidence intervals for the mean response using R, we use the predict() function. We give the function our fitted model as well as new data, stored as a data frame. (This is important, so that R knows the name of the predictor variable.) Here, we are finding the confidence interval for the mean stopping distance when a car is travelling 5 miles per hour and when a car is travelling 21 miles per hour. new_speeds = data.frame(speed = c(5, 21)) predict(stop_dist_model, newdata = new_speeds, interval = c(&quot;confidence&quot;), level = 0.99) ## fit lwr upr ## 1 2.082949 -10.89309 15.05898 ## 2 65.001489 56.45836 73.54462 8.8 Prediction Interval for New Observations Sometimes we would like an interval estimate for a new observation, \\(Y\\), for a particular value of \\(x\\). This is very similar to an interval for the mean response, \\(\\text{E}[Y \\mid X = x]\\), but different in one very important way. Our best guess for a new observation is still \\(\\hat{y}(x)\\). The estimated mean is still the best prediction we can make. The difference is in the amount of variability. We know that observations will vary about the true regression line according to a \\(N(0, \\sigma^2)\\) distribution. Because of this we add an extra factor of \\(\\sigma^2\\) to our estimate’s variability in order to account for the variability of observations about the regression line. \\[ \\begin{aligned} \\text{Var}[\\hat{y}(x) + \\epsilon] &amp;= \\text{Var}[\\hat{y}(x)] + \\text{Var}[\\epsilon] \\\\[2ex] &amp;= \\sigma^2 \\left(\\frac{1}{n}+\\frac{(x-\\bar{x})^2}{S_{xx}}\\right) + \\sigma^2 \\\\[2ex] &amp;= \\sigma^2 \\left(1 + \\frac{1}{n}+\\frac{(x-\\bar{x})^2}{S_{xx}}\\right) \\end{aligned} \\] \\[ \\hat{y}(x) + \\epsilon \\sim N \\left(\\beta_0 + \\beta_1 x, \\ \\sigma^2 \\left(1 + \\frac{1}{n}+\\frac{(x-\\bar{x})^2}{S_{xx}}\\right) \\right) \\] \\[ \\text{SE}[\\hat{y}(x) + \\epsilon] = s_e \\sqrt{1 + \\frac{1}{n}+\\frac{(x-\\bar{x})^2}{S_{xx}}} \\] We can then find a prediction interval using, \\[ \\hat{y}(x) \\pm t_{\\alpha/2, n - 2} \\cdot s_e\\sqrt{1 + \\frac{1}{n}+\\frac{(x-\\bar{x})^2}{S_{xx}}}. \\] To calculate this for a set of points in R notice there is only a minor change in syntax from finding a confidence interval for the mean response. predict(stop_dist_model, newdata = new_speeds, interval = c(&quot;prediction&quot;), level = 0.99) ## fit lwr upr ## 1 2.082949 -41.16099 45.32689 ## 2 65.001489 22.87494 107.12803 Also notice that these two intervals are wider than the corresponding confidence intervals for the mean response. 8.9 Confidence and Prediction Bands Often we will like to plot both confidence intervals for the mean response and prediction intervals for all possible values of \\(x\\). We calls these confidence and prediction bands. speed_grid = seq(min(cars$speed), max(cars$speed), by = 0.01) dist_ci_band = predict(stop_dist_model, newdata = data.frame(speed = speed_grid), interval = &quot;confidence&quot;, level = 0.99) dist_pi_band = predict(stop_dist_model, newdata = data.frame(speed = speed_grid), interval = &quot;prediction&quot;, level = 0.99) plot(dist ~ speed, data = cars, xlab = &quot;Speed (in Miles Per Hour)&quot;, ylab = &quot;Stopping Distance (in Feet)&quot;, main = &quot;Stopping Distance vs Speed&quot;, pch = 20, cex = 2, col = &quot;grey&quot;, ylim = c(min(dist_pi_band), max(dist_pi_band))) abline(stop_dist_model, lwd = 5, col = &quot;darkorange&quot;) lines(speed_grid, dist_ci_band[,&quot;lwr&quot;], col = &quot;dodgerblue&quot;, lwd = 3, lty = 2) lines(speed_grid, dist_ci_band[,&quot;upr&quot;], col = &quot;dodgerblue&quot;, lwd = 3, lty = 2) lines(speed_grid, dist_pi_band[,&quot;lwr&quot;], col = &quot;dodgerblue&quot;, lwd = 3, lty = 3) lines(speed_grid, dist_pi_band[,&quot;upr&quot;], col = &quot;dodgerblue&quot;, lwd = 3, lty = 3) points(mean(cars$speed), mean(cars$dist), pch = &quot;+&quot;, cex = 3) Some things to notice: We use the ylim argument to stretch the \\(y\\)-axis of the plot, since the bands extend further than the points. We add a point at the point \\((\\bar{x}, \\bar{y})\\). This is a point that the regression line will always pass through. (Think about why.) This is the point where both the confidence and prediction bands are the narrowest. Look at the standard errors of both to understand why. The prediction bands (dotted blue) are less curved than the confidence bands (dashed blue). This is a result of the extra factor of \\(\\sigma^2\\) added to the variance at any value of \\(x\\). 8.10 Significance of Regression, F-Test In the case of simple linear regression, the \\(t\\) test for the significance of the regression is equivalent to another test, the \\(F\\) test for the significance of the regression. This equivalence will only be true for simple linear regression, and in the next chapter we will only use the \\(F\\) test for the significance of the regression. Recall from last chapter the decomposition of variance we saw before calculating \\(R^2\\), \\[ \\sum_{i=1}^{n}(y_i - \\bar{y})^2 = \\sum_{i=1}^{n}(y_i - \\hat{y}_i)^2 + \\sum_{i=1}^{n}(\\hat{y}_i - \\bar{y})^2, \\] or, in short, \\[ \\text{SST} = \\text{SSE} + \\text{SSReg}. \\] To develop the \\(F\\) test, we will arrange this information in an ANOVA table, Source Sum of Squares Degrees of Freedom Mean Square \\(F\\) Regression \\(\\sum_{i=1}^{n}(\\hat{y}_i - \\bar{y})^2\\) \\(1\\) \\(\\text{SSReg} / 1\\) \\(\\text{MSReg} / \\text{MSE}\\) Error \\(\\sum_{i=1}^{n}(y_i - \\hat{y}_i)^2\\) \\(n - 2\\) \\(\\text{SSE} / (n - 2)\\) Total \\(\\sum_{i=1}^{n}(y_i - \\bar{y})^2\\) \\(n - 1\\) ANOVA, or Analysis of Variance will be a concept we return to often in this course. For now, we will focus on the results of the table, which is the \\(F\\) statistic, \\[ F = \\frac{\\sum_{i=1}^{n}(\\hat{y}_i - \\bar{y})^2 / 1}{\\sum_{i=1}^{n}(y_i - \\hat{y}_i)^2 / (n - 2)} \\sim F_{1, n - 2} \\] which follows an \\(F\\) distribution with degrees of freedom \\(1\\) and \\(n - 2\\) under the null hypothesis. An \\(F\\) distribution is a continuous distribution which takes only positive values and has two parameters, which are the two degrees of freedom. Recall, in the significance of the regression test, \\(Y\\) does not depend on \\(x\\) in the null hypothesis. \\[ H_0: \\beta_1 = 0 \\quad \\quad Y_i = \\beta_0 + \\epsilon_i \\] While in the alternative hypothesis \\(Y\\) may depend on \\(x\\). \\[ H_1: \\beta_1 \\neq 0 \\quad \\quad Y_i = \\beta_0 + \\beta_1 x_i + \\epsilon_i \\] We can use the \\(F\\) statistic to perform this test. \\[ F = \\frac{\\sum_{i=1}^{n}(\\hat{y}_i - \\bar{y})^2 / 1}{\\sum_{i=1}^{n}(y_i - \\hat{y}_i)^2 / (n - 2)} \\] In particular, we will reject the null when the \\(F\\) statistic is large, that is, when there is a low probability that the observations could have come from the null model by chance. We will let R calculate the p-value for us. To perform the \\(F\\) test in R you can look at the last row of the output from summary() called F-statistic which gives the value of the test statistic, the relevant degrees of freedom, as well as the p-value of the test. summary(stop_dist_model) ## ## Call: ## lm(formula = dist ~ speed, data = cars) ## ## Residuals: ## Min 1Q Median 3Q Max ## -29.069 -9.525 -2.272 9.215 43.201 ## ## Coefficients: ## Estimate Std. Error t value Pr(&gt;|t|) ## (Intercept) -17.5791 6.7584 -2.601 0.0123 * ## speed 3.9324 0.4155 9.464 1.49e-12 *** ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 ## ## Residual standard error: 15.38 on 48 degrees of freedom ## Multiple R-squared: 0.6511, Adjusted R-squared: 0.6438 ## F-statistic: 89.57 on 1 and 48 DF, p-value: 1.49e-12 Additionally, you can use the anova() function to display the information in an ANOVA table. anova(stop_dist_model) ## Analysis of Variance Table ## ## Response: dist ## Df Sum Sq Mean Sq F value Pr(&gt;F) ## speed 1 21186 21185.5 89.567 1.49e-12 *** ## Residuals 48 11354 236.5 ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 This also gives a p-value for the test. You should notice that the p-value from the \\(t\\) test was the same. You might also notice that the value of the test statistic for the \\(t\\) test, \\(9.46399\\), can be squared to obtain the value of the \\(F\\) statistic, \\(89.5671065\\). Note that there is another equivalent way to do this in R, which we will return to often to compare two models. anova(lm(dist ~ 1, data = cars), lm(dist ~ speed, data = cars)) ## Analysis of Variance Table ## ## Model 1: dist ~ 1 ## Model 2: dist ~ speed ## Res.Df RSS Df Sum of Sq F Pr(&gt;F) ## 1 49 32539 ## 2 48 11354 1 21186 89.567 1.49e-12 *** ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 The model statement lm(dist ~ 1, data = cars) applies the model \\(Y_i = \\beta_0 + \\epsilon_i\\) to the cars data. Note that \\(\\hat{y} = \\bar{y}\\) when \\(Y_i = \\beta_0 + \\epsilon_i\\). The model statement lm(dist ~ speed, data = cars) applies the model \\(Y_i = \\beta_0 + \\beta_1 x_i + \\epsilon_i\\). We can then think of this usage of anova() as directly comparing the two models. (Notice we get the same p-value again.) 8.11 R Markdown The R Markdown file for this chapter can be found here: slr-inf.Rmd The file was created using R version 4.4.1. "],["multiple-linear-regression.html", "Chapter 9 Multiple Linear Regression 9.1 Matrix Approach to Regression 9.2 Sampling Distribution 9.3 Significance of Regression 9.4 Nested Models 9.5 Simulation 9.6 R Markdown", " Chapter 9 Multiple Linear Regression “Life is really simple, but we insist on making it complicated.” — Confucius After reading this chapter you will be able to: Construct and interpret linear regression models with more than one predictor. Understand how regression models are derived using matrices. Create interval estimates and perform hypothesis tests for multiple regression parameters. Formulate and interpret interval estimates for the mean response under various conditions. Compare nested models using an ANOVA F-Test. The last two chapters we saw how to fit a model that assumed a linear relationship between a response variable and a single predictor variable. Specifically, we defined the simple linear regression model, \\[ Y_i = \\beta_0 + \\beta_1 x_i + \\epsilon_i \\] where \\(\\epsilon_i \\sim N(0, \\sigma^2)\\). However, it is rarely the case that a dataset will have a single predictor variable. It is also rarely the case that a response variable will only depend on a single variable. So in this chapter, we will extend our current linear model to allow a response to depend on multiple predictors. # read the data from the web autompg = read.table( &quot;http://archive.ics.uci.edu/ml/machine-learning-databases/auto-mpg/auto-mpg.data&quot;, quote = &quot;\\&quot;&quot;, comment.char = &quot;&quot;, stringsAsFactors = FALSE) # give the dataframe headers colnames(autompg) = c(&quot;mpg&quot;, &quot;cyl&quot;, &quot;disp&quot;, &quot;hp&quot;, &quot;wt&quot;, &quot;acc&quot;, &quot;year&quot;, &quot;origin&quot;, &quot;name&quot;) # remove missing data, which is stored as &quot;?&quot; autompg = subset(autompg, autompg$hp != &quot;?&quot;) # remove the plymouth reliant, as it causes some issues autompg = subset(autompg, autompg$name != &quot;plymouth reliant&quot;) # give the dataset row names, based on the engine, year and name rownames(autompg) = paste(autompg$cyl, &quot;cylinder&quot;, autompg$year, autompg$name) # remove the variable for name, as well as origin autompg = subset(autompg, select = c(&quot;mpg&quot;, &quot;cyl&quot;, &quot;disp&quot;, &quot;hp&quot;, &quot;wt&quot;, &quot;acc&quot;, &quot;year&quot;)) # change horsepower from character to numeric autompg$hp = as.numeric(autompg$hp) # check final structure of data str(autompg) ## &#39;data.frame&#39;: 390 obs. of 7 variables: ## $ mpg : num 18 15 18 16 17 15 14 14 14 15 ... ## $ cyl : int 8 8 8 8 8 8 8 8 8 8 ... ## $ disp: num 307 350 318 304 302 429 454 440 455 390 ... ## $ hp : num 130 165 150 150 140 198 220 215 225 190 ... ## $ wt : num 3504 3693 3436 3433 3449 ... ## $ acc : num 12 11.5 11 12 10.5 10 9 8.5 10 8.5 ... ## $ year: int 70 70 70 70 70 70 70 70 70 70 ... We will once again discuss a dataset with information about cars. This dataset, which can be found at the UCI Machine Learning Repository contains a response variable mpg which stores the city fuel efficiency of cars, as well as several predictor variables for the attributes of the vehicles. We load the data, and perform some basic tidying before moving on to analysis. For now we will focus on using two variables, wt and year, as predictor variables. That is, we would like to model the fuel efficiency (mpg) of a car as a function of its weight (wt) and model year (year). To do so, we will define the following linear model, \\[ Y_i = \\beta_0 + \\beta_1 x_{i1} + \\beta_2 x_{i2} + \\epsilon_i, \\qquad i = 1, 2, \\ldots, n \\] where \\(\\epsilon_i \\sim N(0, \\sigma^2)\\). In this notation we will define: \\(x_{i1}\\) as the weight (wt) of the \\(i\\)th car. \\(x_{i2}\\) as the model year (year) of the \\(i\\)th car. The picture below will visualize what we would like to accomplish. The data points \\((x_{i1}, x_{i2}, y_i)\\) now exist in 3-dimensional space, so instead of fitting a line to the data, we will fit a plane. (We’ll soon move to higher dimensions, so this will be the last example that is easy to visualize and think about this way.) How do we find such a plane? Well, we would like a plane that is as close as possible to the data points. That is, we would like it to minimize the errors it is making. How will we define these errors? Squared distance of course! So, we would like to minimize \\[ f(\\beta_0, \\beta_1, \\beta_2) = \\sum_{i = 1}^{n}(y_i - (\\beta_0 + \\beta_1 x_{i1} + \\beta_2 x_{i2}))^2 \\] with respect to \\(\\beta_0\\), \\(\\beta_1\\), and \\(\\beta_2\\). How do we do so? It is another straightforward multivariate calculus problem. All we have done is add an extra variable since we did this last time. So again, we take a derivative with respect to each of \\(\\beta_0\\), \\(\\beta_1\\), and \\(\\beta_2\\) and set them equal to zero, then solve the resulting system of equations. That is, \\[ \\begin{aligned} \\frac{\\partial f}{\\partial \\beta_0} &amp;= 0 \\\\ \\frac{\\partial f}{\\partial \\beta_1} &amp;= 0 \\\\ \\frac{\\partial f}{\\partial \\beta_2} &amp;= 0 \\end{aligned} \\] After doing so, we will once again obtain the normal equations. \\[ \\begin{aligned} n \\beta_0 + \\beta_1 \\sum_{i = 1}^{n} x_{i1} + \\beta_2 \\sum_{i = 1}^{n} x_{i2} &amp;= \\sum_{i = 1}^{n} y_i \\\\ \\beta_0 \\sum_{i = 1}^{n} x_{i1} + \\beta_1 \\sum_{i = 1}^{n} x_{i1}^2 + \\beta_2 \\sum_{i = 1}^{n} x_{i1}x_{i2} &amp;= \\sum_{i = 1}^{n} x_{i1}y_i \\\\ \\beta_0 \\sum_{i = 1}^{n} x_{i2} + \\beta_1 \\sum_{i = 1}^{n} x_{i1}x_{i2} + \\beta_2 \\sum_{i = 1}^{n} x_{i2}^2 &amp;= \\sum_{i = 1}^{n} x_{i2}y_i \\end{aligned} \\] We now have three equations and three variables, which we could solve, or we could simply let R solve for us. mpg_model = lm(mpg ~ wt + year, data = autompg) coef(mpg_model) ## (Intercept) wt year ## -14.637641945 -0.006634876 0.761401955 \\[ \\hat{y} = -14.6376419 + -0.0066349 x_1 + 0.761402 x_2 \\] Here we have once again fit our model using lm(), however we have introduced a new syntactical element. The formula mpg ~ wt + year now reads: “model the response variable mpg as a linear function of wt and year.” That is, it will estimate an intercept, as well as slope coefficients for wt and year. We then extract these as we have done before using coef(). In the multiple linear regression setting, some of the interpretations of the coefficients change slightly. Here, \\(\\hat{\\beta}_0 = -14.6376419\\) is our estimate for \\(\\beta_0\\), the mean miles per gallon for a car that weighs 0 pounds and was built in 1900. We see our estimate here is negative, which is a physical impossibility. However, this isn’t unexpected, as we shouldn’t expect our model to be accurate for cars from 1900 which weigh 0 pounds. (Because they never existed!) This isn’t much of a change from SLR. That is, \\(\\beta_0\\) is still simply the mean when all of the predictors are 0. The interpretation of the coefficients in front of our predictors is slightly different than before. For example \\(\\hat{\\beta}_1 = -0.0066349\\) is our estimate for \\(\\beta_1\\), the average change in miles per gallon for an increase in weight (\\(x_{1}\\)) of one-pound for a car of a certain model year, that is, for a fixed value of \\(x_{2}\\). Note that this coefficient is actually the same for any given value of \\(x_{2}\\). Later, we will look at models that allow for a different change in mean response for different values of \\(x_{2}\\). Also note that this estimate is negative, which we would expect since, in general, fuel efficiency decreases for larger vehicles. Recall that in the multiple linear regression setting, this interpretation is dependent on a fixed value for \\(x_{2}\\), that is, “for a car of a certain model year.” It is possible that the indirect relationship between fuel efficiency and weight does not hold when an additional factor, say year, is included, and thus we could have the sign of our coefficient flipped. Lastly, \\(\\hat{\\beta}_2 = 0.761402\\) is our estimate for \\(\\beta_2\\), the average change in miles per gallon for a one-year increase in model year (\\(x_{2}\\)) for a car of a certain weight, that is, for a fixed value of \\(x_{1}\\). It is not surprising that the estimate is positive. We expect that as time passes and the years march on, technology would improve so that a car of a specific weight would get better mileage now as compared to their predecessors. And yet, the coefficient could have been negative because we are also including weight as variable, and not strictly as a fixed value. 9.1 Matrix Approach to Regression In our above example we used two predictor variables, but it will only take a little more work to allow for an arbitrary number of predictor variables and derive their coefficient estimates. We can consider the model, \\[ Y_i = \\beta_0 + \\beta_1 x_{i1} + \\beta_2 x_{i2} + \\cdots + \\beta_{p-1} x_{i(p-1)} + \\epsilon_i, \\qquad i = 1, 2, \\ldots, n \\] where \\(\\epsilon_i \\sim N(0, \\sigma^2)\\). In this model, there are \\(p - 1\\) predictor variables, \\(x_1, x_2, \\cdots, x_{p-1}\\). There are a total of \\(p\\) \\(\\beta\\)-parameters and a single parameter \\(\\sigma^2\\) for the variance of the errors. (It should be noted that almost as often, authors will use \\(p\\) as the number of predictors, making the total number of \\(\\beta\\) parameters \\(p+1\\). This is always something you should be aware of when reading about multiple regression. There is not a standard that is used most often.) If we were to stack together the \\(n\\) linear equations that represent each \\(Y_i\\) into a column vector, we get the following. \\[ \\begin{bmatrix} Y_1 \\\\ Y_2 \\\\ \\vdots\\\\ Y_n \\\\ \\end{bmatrix} = \\begin{bmatrix} 1 &amp; x_{11} &amp; x_{12} &amp; \\cdots &amp; x_{1(p-1)} \\\\ 1 &amp; x_{21} &amp; x_{22} &amp; \\cdots &amp; x_{2(p-1)} \\\\ \\vdots &amp; \\vdots &amp; \\vdots &amp; &amp; \\vdots \\\\ 1 &amp; x_{n1} &amp; x_{n2} &amp; \\cdots &amp; x_{n(p-1)} \\\\ \\end{bmatrix} \\begin{bmatrix} \\beta_0 \\\\ \\beta_1 \\\\ \\beta_2 \\\\ \\vdots \\\\ \\beta_{p-1} \\\\ \\end{bmatrix} + \\begin{bmatrix} \\epsilon_1 \\\\ \\epsilon_2 \\\\ \\vdots\\\\ \\epsilon_n \\\\ \\end{bmatrix} \\] \\[ Y = X \\beta + \\epsilon \\] \\[ Y = \\begin{bmatrix} Y_1 \\\\ Y_2 \\\\ \\vdots\\\\ Y_n \\end{bmatrix}, \\quad X = \\begin{bmatrix} 1 &amp; x_{11} &amp; x_{12} &amp; \\cdots &amp; x_{1(p-1)} \\\\ 1 &amp; x_{21} &amp; x_{22} &amp; \\cdots &amp; x_{2(p-1)} \\\\ \\vdots &amp; \\vdots &amp; \\vdots &amp; &amp; \\vdots \\\\ 1 &amp; x_{n1} &amp; x_{n2} &amp; \\cdots &amp; x_{n(p-1)} \\\\ \\end{bmatrix}, \\quad \\beta = \\begin{bmatrix} \\beta_0 \\\\ \\beta_1 \\\\ \\beta_2 \\\\ \\vdots \\\\ \\beta_{p-1} \\\\ \\end{bmatrix}, \\quad \\epsilon = \\begin{bmatrix} \\epsilon_1 \\\\ \\epsilon_2 \\\\ \\vdots\\\\ \\epsilon_n \\end{bmatrix} \\] So now with data, \\[ y = \\begin{bmatrix} y_1 \\\\ y_2 \\\\ \\vdots\\\\ y_n \\end{bmatrix} \\] Just as before, we can estimate \\(\\beta\\) by minimizing, \\[ f(\\beta_0, \\beta_1, \\beta_2, \\cdots, \\beta_{p-1}) = \\sum_{i = 1}^{n}(y_i - (\\beta_0 + \\beta_1 x_{i1} + \\beta_2 x_{i2} + \\cdots + \\beta_{p-1} x_{i(p-1)}))^2, \\] which would require taking \\(p\\) derivatives, which result in the following normal equations. \\[ \\begin{bmatrix} n &amp; \\sum_{i = 1}^{n} x_{i1} &amp; \\sum_{i = 1}^{n} x_{i2} &amp; \\cdots &amp; \\sum_{i = 1}^{n} x_{i(p-1)} \\\\ \\sum_{i = 1}^{n} x_{i1} &amp; \\sum_{i = 1}^{n} x_{i1}^2 &amp; \\sum_{i = 1}^{n} x_{i1}x_{i2} &amp; \\cdots &amp; \\sum_{i = 1}^{n} x_{i1}x_{i(p-1)} \\\\ \\vdots &amp; \\vdots &amp; \\vdots &amp; &amp; \\vdots \\\\ \\sum_{i = 1}^{n} x_{i(p-1)} &amp; \\sum_{i = 1}^{n} x_{i(p-1)}x_{i1} &amp; \\sum_{i = 1}^{n} x_{i(p-1)}x_{i2} &amp; \\cdots &amp; \\sum_{i = 1}^{n} x_{i(p-1)}^2 \\\\ \\end{bmatrix} \\begin{bmatrix} \\beta_0 \\\\ \\beta_1 \\\\ \\vdots \\\\ \\beta_{p-1} \\\\ \\end{bmatrix} = \\begin{bmatrix} \\sum_{i = 1}^{n} y_i \\\\ \\sum_{i = 1}^{n} x_{i1}y_i \\\\ \\vdots \\\\ \\sum_{i = 1}^{n} x_{i(p-1)}y_i \\\\ \\end{bmatrix} \\] The normal equations can be written much more succinctly in matrix notation, \\[ X^\\top X \\beta = X^\\top y. \\] We can then solve this expression by multiplying both sides by the inverse of \\(X^\\top X\\), which exists, provided the columns of \\(X\\) are linearly independent. Then as always, we denote our solution with a hat. \\[ \\hat{\\beta} = \\left( X^\\top X \\right)^{-1}X^\\top y \\] To verify that this is what R has done for us in the case of two predictors, we create an \\(X\\) matrix. Note that the first column is all 1s, and the remaining columns contain the data. n = nrow(autompg) p = length(coef(mpg_model)) X = cbind(rep(1, n), autompg$wt, autompg$year) y = autompg$mpg (beta_hat = solve(t(X) %*% X) %*% t(X) %*% y) ## [,1] ## [1,] -14.637641945 ## [2,] -0.006634876 ## [3,] 0.761401955 coef(mpg_model) ## (Intercept) wt year ## -14.637641945 -0.006634876 0.761401955 \\[ \\hat{\\beta} = \\begin{bmatrix} -14.6376419 \\\\ -0.0066349 \\\\ 0.761402 \\\\ \\end{bmatrix} \\] In our new notation, the fitted values can be written \\[ \\hat{y} = X \\hat{\\beta}. \\] \\[ \\hat{y} = \\begin{bmatrix} \\hat{y}_1 \\\\ \\hat{y}_2 \\\\ \\vdots\\\\ \\hat{y}_n \\end{bmatrix} \\] Then, we can create a vector for the residual values, \\[ e = \\begin{bmatrix} e_1 \\\\ e_2 \\\\ \\vdots\\\\ e_n \\end{bmatrix} = \\begin{bmatrix} y_1 \\\\ y_2 \\\\ \\vdots\\\\ y_n \\end{bmatrix} - \\begin{bmatrix} \\hat{y}_1 \\\\ \\hat{y}_2 \\\\ \\vdots\\\\ \\hat{y}_n \\end{bmatrix}. \\] And lastly, we can update our estimate for \\(\\sigma^2\\). \\[ s_e^2 = \\frac{\\sum_{i=1}^n (y_i - \\hat{y}_i)^2}{n - p} = \\frac{e^\\top e}{n-p} \\] Recall, we like this estimate because it is unbiased, that is, \\[ \\text{E}[s_e^2] = \\sigma^2 \\] Note that the change from the SLR estimate to now is in the denominator. Specifically we now divide by \\(n - p\\) instead of \\(n - 2\\). Or actually, we should note that in the case of SLR, there are two \\(\\beta\\) parameters and thus \\(p = 2\\). Also note that if we fit the model \\(Y_i = \\beta + \\epsilon_i\\) that \\(\\hat{y} = \\bar{y}\\) and \\(p = 1\\) and \\(s_e^2\\) would become \\[ s_e^2 = \\frac{\\sum_{i=1}^n (y_i - \\bar{y})^2}{n - 1} \\] which is likely the very first sample variance you saw in a mathematical statistics class. The same reason for \\(n - 1\\) in this case, that we estimated one parameter, so we lose one degree of freedom. Now, in general, we are estimating \\(p\\) parameters, the \\(\\beta\\) parameters, so we lose \\(p\\) degrees of freedom. Also, recall that most often we will be interested in \\(s_e\\), the residual standard error as R calls it, \\[ s_e = \\sqrt{\\frac{\\sum_{i=1}^n (y_i - \\hat{y}_i)^2}{n - p}}. \\] In R, we could directly access \\(s_e\\) for a fitted model, as we have seen before. summary(mpg_model)$sigma ## [1] 3.431367 And we can now verify that our math above is indeed calculating the same quantities. y_hat = X %*% solve(t(X) %*% X) %*% t(X) %*% y e = y - y_hat sqrt(t(e) %*% e / (n - p)) ## [,1] ## [1,] 3.431367 sqrt(sum((y - y_hat) ^ 2) / (n - p)) ## [1] 3.431367 9.2 Sampling Distribution As we can see in the output below, the results of calling summary() are similar to SLR, but there are some differences, most obviously a new row for the added predictor variable. summary(mpg_model) ## ## Call: ## lm(formula = mpg ~ wt + year, data = autompg) ## ## Residuals: ## Min 1Q Median 3Q Max ## -8.852 -2.292 -0.100 2.039 14.325 ## ## Coefficients: ## Estimate Std. Error t value Pr(&gt;|t|) ## (Intercept) -1.464e+01 4.023e+00 -3.638 0.000312 *** ## wt -6.635e-03 2.149e-04 -30.881 &lt; 2e-16 *** ## year 7.614e-01 4.973e-02 15.312 &lt; 2e-16 *** ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 ## ## Residual standard error: 3.431 on 387 degrees of freedom ## Multiple R-squared: 0.8082, Adjusted R-squared: 0.8072 ## F-statistic: 815.6 on 2 and 387 DF, p-value: &lt; 2.2e-16 To understand these differences in detail, we will need to first obtain the sampling distribution of \\(\\hat{\\beta}\\). The derivation of the sampling distribution of \\(\\hat{\\beta}\\) involves the multivariate normal distribution. These brief notes from semesters past give a basic overview. These are simply for your information, as we will not present the derivation in full here. Our goal now is to obtain the distribution of the \\(\\hat{\\beta}\\) vector, \\[ \\hat{\\beta} = \\begin{bmatrix} \\hat{\\beta}_0 \\\\ \\hat{\\beta}_1 \\\\ \\hat{\\beta}_2 \\\\ \\vdots \\\\ \\hat{\\beta}_{p-1} \\end{bmatrix} \\] Recall from last time that when discussing sampling distributions, we now consider \\(\\hat{\\beta}\\) to be a random vector, thus we use \\(Y\\) instead of the data vector \\(y\\). \\[ \\hat{\\beta} = \\left( X^\\top X \\right)^{-1}X^\\top Y \\] Then it is a consequence of the multivariate normal distribution that, \\[ \\hat{\\beta} \\sim N\\left(\\beta, \\sigma^2 \\left(X^\\top X\\right)^{-1} \\right). \\] We then have \\[ \\text{E}[\\hat{\\beta}] = \\beta \\] and for any \\(\\hat{\\beta}_j\\) we have \\[ \\text{E}[\\hat{\\beta}_j] = \\beta_j. \\] We also have \\[ \\text{Var}[\\hat{\\beta}] = \\sigma^2 \\left( X^\\top X \\right)^{-1} \\] and for any \\(\\hat{\\beta}_j\\) we have \\[ \\text{Var}[\\hat{\\beta}_j] = \\sigma^2 C_{jj} \\] where \\[ C = \\left(X^\\top X\\right)^{-1} \\] and the elements of \\(C\\) are denoted \\[ C = \\begin{bmatrix} C_{00} &amp; C_{01} &amp; C_{02} &amp; \\cdots &amp; C_{0(p-1)} \\\\ C_{10} &amp; C_{11} &amp; C_{12} &amp; \\cdots &amp; C_{1(p-1)} \\\\ C_{20} &amp; C_{21} &amp; C_{22} &amp; \\cdots &amp; C_{2(p-1)} \\\\ \\vdots &amp; \\vdots &amp; \\vdots &amp; &amp; \\vdots \\\\ C_{(p-1)0} &amp; C_{(p-1)1} &amp; C_{(p-1)2} &amp; \\cdots &amp; C_{(p-1)(p-1)} \\\\ \\end{bmatrix}. \\] Essentially, the diagonal elements correspond to the \\(\\beta\\) vector. Then the standard error for the \\(\\hat{\\beta}\\) vector is given by \\[ \\text{SE}[\\hat{\\beta}] = s_e \\sqrt{\\left( X^\\top X \\right)^{-1}} \\] and for a particular \\(\\hat{\\beta}_j\\) \\[ \\text{SE}[\\hat{\\beta}_j] = s_e \\sqrt{C_{jj}}. \\] Lastly, each of the \\(\\hat{\\beta}_j\\) follows a normal distribution, \\[ \\hat{\\beta}_j \\sim N\\left(\\beta_j, \\sigma^2 C_{jj} \\right). \\] thus \\[ \\frac{\\hat{\\beta}_j - \\beta_j}{s_e \\sqrt{C_{jj}}} \\sim t_{n-p}. \\] Now that we have the necessary distributional results, we can move on to perform tests and make interval estimates. 9.2.1 Single Parameter Tests The first test we will see is a test for a single \\(\\beta_j\\). \\[ H_0: \\beta_j = 0 \\quad \\text{vs} \\quad H_1: \\beta_j \\neq 0 \\] Again, the test statistic takes the form \\[ \\text{TS} = \\frac{\\text{EST} - \\text{HYP}}{\\text{SE}}. \\] In particular, \\[ t = \\frac{\\hat{\\beta}_j - \\beta_j}{\\text{SE}[\\hat{\\beta}_j]} = \\frac{\\hat{\\beta}_j-0}{s_e\\sqrt{C_{jj}}}, \\] which, under the null hypothesis, follows a \\(t\\) distribution with \\(n - p\\) degrees of freedom. Recall our model for mpg, \\[ Y_i = \\beta_0 + \\beta_1 x_{i1} + \\beta_2 x_{i2} + \\epsilon_i, \\qquad i = 1, 2, \\ldots, n \\] where \\(\\epsilon_i \\sim N(0, \\sigma^2)\\). \\(x_{i1}\\) as the weight (wt) of the \\(i\\)th car. \\(x_{i2}\\) as the model year (year) of the \\(i\\)th car. Then the test \\[ H_0: \\beta_1 = 0 \\quad \\text{vs} \\quad H_1: \\beta_1 \\neq 0 \\] can be found in the summary() output, in particular: summary(mpg_model)$coef ## Estimate Std. Error t value Pr(&gt;|t|) ## (Intercept) -14.637641945 4.0233913563 -3.638135 3.118311e-04 ## wt -0.006634876 0.0002148504 -30.881372 1.850466e-106 ## year 0.761401955 0.0497265950 15.311765 1.036597e-41 The estimate (Estimate), standard error (Std. Error), test statistic (t value), and p-value (Pr(&gt;|t|)) for this test are displayed in the second row, labeled wt. Remember that the p-value given here is specifically for a two-sided test, where the hypothesized value is 0. Also note in this case, by hypothesizing that \\(\\beta_1 = 0\\) the null and alternative essentially specify two different models: \\(H_0\\): \\(Y = \\beta_0 + \\beta_2 x_{2} + \\epsilon\\) \\(H_1\\): \\(Y = \\beta_0 + \\beta_1 x_{1} + \\beta_2 x_{2} + \\epsilon\\) This is important. We are not simply testing whether or not there is a relationship between weight and fuel efficiency. We are testing if there is a relationship between weight and fuel efficiency, given that a term for year is in the model. (Note, we dropped some indexing here, for readability.) 9.2.2 Confidence Intervals Since \\(\\hat{\\beta}_j\\) is our estimate for \\(\\beta_j\\) and we have \\[ \\text{E}[\\hat{\\beta}_j] = \\beta_j \\] as well as the standard error, \\[ \\text{SE}[\\hat{\\beta}_j] = s_e\\sqrt{C_{jj}} \\] and the sampling distribution of \\(\\hat{\\beta}_j\\) is Normal, then we can easily construct confidence intervals for each of the \\(\\hat{\\beta}_j\\). \\[ \\hat{\\beta}_j \\pm t_{\\alpha/2, n - p} \\cdot s_e\\sqrt{C_{jj}} \\] We can find these in R using the same method as before. Now there will simply be additional rows for the additional \\(\\beta\\). confint(mpg_model, level = 0.99) ## 0.5 % 99.5 % ## (Intercept) -25.052563681 -4.222720208 ## wt -0.007191036 -0.006078716 ## year 0.632680051 0.890123859 9.2.3 Confidence Intervals for Mean Response As we saw in SLR, we can create confidence intervals for the mean response, that is, an interval estimate for \\(\\text{E}[Y \\mid X = x]\\). In SLR, the mean of \\(Y\\) was only dependent on a single value \\(x\\). Now, in multiple regression, \\(\\text{E}[Y \\mid X = x]\\) is dependent on the value of each of the predictors, so we define the vector \\(x_0\\) to be, \\[ x_{0} = \\begin{bmatrix} 1 \\\\ x_{01} \\\\ x_{02} \\\\ \\vdots \\\\ x_{0(p-1)} \\\\ \\end{bmatrix}. \\] Then our estimate of \\(\\text{E}[Y \\mid X = x_0]\\) for a set of values \\(x_0\\) is given by \\[ \\begin{aligned} \\hat{y}(x_0) &amp;= x_{0}^\\top\\hat{\\beta} \\\\ &amp;= \\hat{\\beta}_0 + \\hat{\\beta}_1 x_{01} + \\hat{\\beta}_2 x_{02} + \\cdots + \\hat{\\beta}_{p-1} x_{0(p-1)}. \\end{aligned} \\] As with SLR, this is an unbiased estimate. \\[ \\begin{aligned} \\text{E}[\\hat{y}(x_0)] &amp;= x_{0}^\\top\\beta \\\\ &amp;= \\beta_0 + \\beta_1 x_{01} + \\beta_2 x_{02} + \\cdots + \\beta_{p-1} x_{0(p-1)} \\end{aligned} \\] To make an interval estimate, we will also need its standard error. \\[ \\text{SE}[\\hat{y}(x_0)] = s_e \\sqrt{x_{0}^\\top\\left(X^\\top X\\right)^{-1}x_{0}} \\] Putting it all together, we obtain a confidence interval for the mean response. \\[ \\hat{y}(x_0) \\pm t_{\\alpha/2, n - p} \\cdot s_e \\sqrt{x_{0}^\\top\\left(X^\\top X\\right)^{-1}x_{0}} \\] The math has changed a bit, but the process in R remains almost identical. Here, we create a data frame for two additional cars. One car that weighs 3500 pounds produced in 1976, as well as a second car that weighs 5000 pounds which was produced in 1981. new_cars = data.frame(wt = c(3500, 5000), year = c(76, 81)) new_cars ## wt year ## 1 3500 76 ## 2 5000 81 We can then use the predict() function with interval = \"confidence\" to obtain intervals for the mean fuel efficiency for both new cars. Again, it is important to make the data passed to newdata a data frame, so that R knows which values are for which variables. predict(mpg_model, newdata = new_cars, interval = &quot;confidence&quot;, level = 0.99) ## fit lwr upr ## 1 20.00684 19.4712 20.54248 ## 2 13.86154 12.3341 15.38898 R then reports the estimate \\(\\hat{y}(x_0)\\) (fit) for each, as well as the lower (lwr) and upper (upr) bounds for the interval at a desired level (99%). A word of caution here: one of these estimates is good while one is suspect. new_cars$wt ## [1] 3500 5000 range(autompg$wt) ## [1] 1613 5140 Note that both of the weights of the new cars are within the range of observed values. new_cars$year ## [1] 76 81 range(autompg$year) ## [1] 70 82 As are the years of each of the new cars. plot(year ~ wt, data = autompg, pch = 20, col = &quot;dodgerblue&quot;, cex = 1.5) points(new_cars, col = &quot;darkorange&quot;, cex = 3, pch = &quot;X&quot;) However, we have to consider weight and year together now. And based on the above plot, one of the new cars is within the “blob” of observed values, while the other, the car from 1981 weighing 5000 pounds, is noticeably outside of the observed values. This is a hidden extrapolation which you should be aware of when using multiple regression. Shifting gears back to the new data pair that can be reasonably estimated, we do a quick verification of some of the mathematics in R. x0 = c(1, 3500, 76) x0 %*% beta_hat ## [,1] ## [1,] 20.00684 \\[ x_{0} = \\begin{bmatrix} 1 \\\\ 3500 \\\\ 76 \\\\ \\end{bmatrix} \\] \\[ \\hat{\\beta} = \\begin{bmatrix} -14.6376419 \\\\ -0.0066349 \\\\ 0.761402 \\\\ \\end{bmatrix} \\] \\[ \\hat{y}(x_0) = x_{0}^\\top\\hat{\\beta} = \\begin{bmatrix} 1 &amp; 3500 &amp; 76 \\\\ \\end{bmatrix} \\begin{bmatrix} -14.6376419 \\\\ -0.0066349 \\\\ 0.761402 \\\\ \\end{bmatrix}= 20.0068411 \\] Also note that, using a particular value for \\(x_0\\), we can essentially extract certain \\(\\hat{\\beta}_j\\) values. beta_hat ## [,1] ## [1,] -14.637641945 ## [2,] -0.006634876 ## [3,] 0.761401955 x0 = c(0, 0, 1) x0 %*% beta_hat ## [,1] ## [1,] 0.761402 With this in mind, confidence intervals for the individual \\(\\hat{\\beta}_j\\) are actually a special case of a confidence interval for mean response. 9.2.4 Prediction Intervals As with SLR, creating prediction intervals involves one slight change to the standard error to account for the fact that we are now considering an observation, instead of a mean. Here we use \\(\\hat{y}(x_0)\\) to estimate \\(Y_0\\), a new observation of \\(Y\\) at the predictor vector \\(x_0\\). \\[ \\begin{aligned} \\hat{y}(x_0) &amp;= x_{0}^\\top\\hat{\\beta} \\\\ &amp;= \\hat{\\beta}_0 + \\hat{\\beta}_1 x_{01} + \\hat{\\beta}_2 x_{02} + \\cdots + \\hat{\\beta}_{p-1} x_{0(p-1)} \\end{aligned} \\] \\[ \\begin{aligned} \\text{E}[\\hat{y}(x_0)] &amp;= x_{0}^\\top\\beta \\\\ &amp;= \\beta_0 + \\beta_1 x_{01} + \\beta_2 x_{02} + \\cdots + \\beta_{p-1} x_{0(p-1)} \\end{aligned} \\] As we did with SLR, we need to account for the additional variability of an observation about its mean. \\[ \\text{SE}[\\hat{y}(x_0) + \\epsilon] = s_e \\sqrt{1 + x_{0}^\\top\\left(X^\\top X\\right)^{-1}x_{0}} \\] Then we arrive at our updated prediction interval for MLR. \\[ \\hat{y}(x_0) \\pm t_{\\alpha/2, n - p} \\cdot s_e \\sqrt{1 + x_{0}^\\top\\left(X^\\top X\\right)^{-1}x_{0}} \\] new_cars ## wt year ## 1 3500 76 ## 2 5000 81 predict(mpg_model, newdata = new_cars, interval = &quot;prediction&quot;, level = 0.99) ## fit lwr upr ## 1 20.00684 11.108294 28.90539 ## 2 13.86154 4.848751 22.87432 9.3 Significance of Regression The decomposition of variation that we had seen in SLR still holds for MLR. \\[ \\sum_{i=1}^{n}(y_i - \\bar{y})^2 = \\sum_{i=1}^{n}(y_i - \\hat{y}_i)^2 + \\sum_{i=1}^{n}(\\hat{y}_i - \\bar{y})^2 \\] That is, \\[ \\text{SST} = \\text{SSE} + \\text{SSReg}. \\] This means that we can still calculate \\(R^2\\) in the same manner as before, which R continues to do automatically. summary(mpg_model)$r.squared ## [1] 0.8082355 The interpretation changes slightly as compared to SLR. In this MLR case, we say that \\(80.82\\%\\) of the observed variation in miles per gallon is explained by the linear relationship with the two predictor variables, weight and year. In multiple regression, the significance of regression test is \\[ H_0: \\beta_1 = \\beta_2 = \\cdots = \\beta_{p - 1} = 0. \\] Here, we see that the null hypothesis sets all of the \\(\\beta_j\\) equal to 0, except the intercept, \\(\\beta_0\\). We could then say that the null model, or “model under the null hypothesis” is \\[ Y_i = \\beta_0 + \\epsilon_i. \\] This is a model where the regression is insignificant. None of the predictors have a significant linear relationship with the response. Notationally, we will denote the fitted values of this model as \\(\\hat{y}_{0i}\\), which in this case happens to be: \\[ \\hat{y}_{0i} = \\bar{y}. \\] The alternative hypothesis here is that at least one of the \\(\\beta_j\\) from the null hypothesis is not 0. \\[ H_1: \\text{At least one of } \\beta_j \\neq 0, j = 1, 2, \\cdots, (p-1) \\] We could then say that the full model, or “model under the alternative hypothesis” is \\[ Y_i = \\beta_0 + \\beta_1 x_{i1} + \\beta_2 x_{i2} + \\cdots + \\beta_{(p-1)} x_{i(p-1)} + \\epsilon_i \\] This is a model where the regression is significant. At least one of the predictors has a significant linear relationship with the response. There is some linear relationship between \\(y\\) and the predictors, \\(x_1, x_2, \\ldots, x_{p - 1}\\). We will denote the fitted values of this model as \\(\\hat{y}_{1i}\\). To develop the \\(F\\) test for the significance of the regression, we will arrange the variance decomposition into an ANOVA table. Source Sum of Squares Degrees of Freedom Mean Square \\(F\\) Regression \\(\\sum_{i=1}^{n}(\\hat{y}_{1i} - \\bar{y})^2\\) \\(p - 1\\) \\(\\text{SSReg} / (p - 1)\\) \\(\\text{MSReg} / \\text{MSE}\\) Error \\(\\sum_{i=1}^{n}(y_i - \\hat{y}_{1i})^2\\) \\(n - p\\) \\(\\text{SSE} / (n - p)\\) Total \\(\\sum_{i=1}^{n}(y_i - \\bar{y})^2\\) \\(n - 1\\) In summary, the \\(F\\) statistic is \\[ F = \\frac{\\sum_{i=1}^{n}(\\hat{y}_{1i} - \\bar{y})^2 / (p - 1)}{\\sum_{i=1}^{n}(y_i - \\hat{y}_{1i})^2 / (n - p)}, \\] and the p-value is calculated as \\[ P(F_{p-1, n-p} &gt; F) \\] since we reject for large values of \\(F\\). A large value of the statistic corresponds to a large portion of the variance being explained by the regression. Here \\(F_{p-1, n-p}\\) represents a random variable which follows an \\(F\\) distribution with \\(p - 1\\) and \\(n - p\\) degrees of freedom. To perform this test in R, we first explicitly specify the two models in R and save the results in different variables. We then use anova() to compare the two models, giving anova() the null model first and the alternative (full) model second. (Specifying the full model first will result in the same p-value, but some nonsensical intermediate values.) In this case, \\(H_0\\): \\(Y_i = \\beta_0 + \\epsilon_i\\) \\(H_1\\): \\(Y_i = \\beta_0 + \\beta_1 x_{i1} + \\beta_2 x_{i2} + \\epsilon_i\\) That is, in the null model, we use neither of the predictors, whereas in the full (alternative) model, at least one of the predictors is useful. null_mpg_model = lm(mpg ~ 1, data = autompg) full_mpg_model = lm(mpg ~ wt + year, data = autompg) anova(null_mpg_model, full_mpg_model) ## Analysis of Variance Table ## ## Model 1: mpg ~ 1 ## Model 2: mpg ~ wt + year ## Res.Df RSS Df Sum of Sq F Pr(&gt;F) ## 1 389 23761.7 ## 2 387 4556.6 2 19205 815.55 &lt; 2.2e-16 *** ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 First, notice that R does not display the results in the same manner as the table above. More important than the layout of the table are its contents. We see that the value of the \\(F\\) statistic is 815.55, and the p-value is extremely low, so we reject the null hypothesis at any reasonable \\(\\alpha\\) and say that the regression is significant. At least one of wt or year has a useful linear relationship with mpg. summary(mpg_model) ## ## Call: ## lm(formula = mpg ~ wt + year, data = autompg) ## ## Residuals: ## Min 1Q Median 3Q Max ## -8.852 -2.292 -0.100 2.039 14.325 ## ## Coefficients: ## Estimate Std. Error t value Pr(&gt;|t|) ## (Intercept) -1.464e+01 4.023e+00 -3.638 0.000312 *** ## wt -6.635e-03 2.149e-04 -30.881 &lt; 2e-16 *** ## year 7.614e-01 4.973e-02 15.312 &lt; 2e-16 *** ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 ## ## Residual standard error: 3.431 on 387 degrees of freedom ## Multiple R-squared: 0.8082, Adjusted R-squared: 0.8072 ## F-statistic: 815.6 on 2 and 387 DF, p-value: &lt; 2.2e-16 Notice that the value reported in the row for F-statistic is indeed the \\(F\\) test statistic for the significance of regression test, and additionally it reports the two relevant degrees of freedom. Also, note that none of the individual \\(t\\)-tests are equivalent to the \\(F\\)-test as they were in SLR. This equivalence only holds for SLR because the individual test for \\(\\beta_1\\) is the same as testing for all non-intercept parameters, since there is only one. We can also verify the sums of squares and degrees of freedom directly in R. You should match these to the table from R and use this to match R’s output to the written table above. # SSReg sum((fitted(full_mpg_model) - fitted(null_mpg_model)) ^ 2) ## [1] 19205.03 # SSE sum(resid(full_mpg_model) ^ 2) ## [1] 4556.646 # SST sum(resid(null_mpg_model) ^ 2) ## [1] 23761.67 # Degrees of Freedom: Regression length(coef(full_mpg_model)) - length(coef(null_mpg_model)) ## [1] 2 # Degrees of Freedom: Error length(resid(full_mpg_model)) - length(coef(full_mpg_model)) ## [1] 387 # Degrees of Freedom: Total length(resid(null_mpg_model)) - length(coef(null_mpg_model)) ## [1] 389 9.4 Nested Models The significance of regression test is actually a special case of testing what we will call nested models. More generally we can compare two models, where one model is “nested” inside the other, meaning one model contains a subset of the predictors from only the larger model. Consider the following full model, \\[ Y_i = \\beta_0 + \\beta_1 x_{i1} + \\beta_2 x_{i2} + \\cdots + \\beta_{(p-1)} x_{i(p-1)} + \\epsilon_i \\] This model has \\(p - 1\\) predictors, for a total of \\(p\\) \\(\\beta\\)-parameters. We will denote the fitted values of this model as \\(\\hat{y}_{1i}\\). Let the null model be \\[ Y_i = \\beta_0 + \\beta_1 x_{i1} + \\beta_2 x_{i2} + \\cdots + \\beta_{(q-1)} x_{i(q-1)} + \\epsilon_i \\] where \\(q &lt; p\\). This model has \\(q - 1\\) predictors, for a total of \\(q\\) \\(\\beta\\)-parameters. We will denote the fitted values of this model as \\(\\hat{y}_{0i}\\). The difference between these two models can be codified by the null hypothesis of a test. \\[ H_0: \\beta_q = \\beta_{q+1} = \\cdots = \\beta_{p - 1} = 0. \\] Specifically, the \\(\\beta\\)-parameters from the full model that are not in the null model are zero. The resulting model, which is nested, is the null model. We can then perform this test using an \\(F\\)-test, which is the result of the following ANOVA table. Source Sum of Squares Degrees of Freedom Mean Square \\(F\\) Diff \\(\\sum_{i=1}^{n}(\\hat{y}_{1i} - \\hat{y}_{0i})^2\\) \\(p - q\\) \\(\\text{SSD} / (p - q)\\) \\(\\text{MSD} / \\text{MSE}\\) Full \\(\\sum_{i=1}^{n}(y_i - \\hat{y}_{1i})^2\\) \\(n - p\\) \\(\\text{SSE} / (n - p)\\) Null \\(\\sum_{i=1}^{n}(y_i - \\hat{y}_{0i})^2\\) \\(n - q\\) \\[ F = \\frac{\\sum_{i=1}^{n}(\\hat{y}_{1i} - \\hat{y}_{0i})^2 / (p - q)}{\\sum_{i=1}^{n}(y_i - \\hat{y}_{1i})^2 / (n - p)}. \\] Notice that the row for “Diff” compares the sum of the squared differences of the fitted values. The degrees of freedom is then the difference of the number of \\(\\beta\\)-parameters estimated between the two models. For example, the autompg dataset has a number of additional variables that we have yet to use. names(autompg) ## [1] &quot;mpg&quot; &quot;cyl&quot; &quot;disp&quot; &quot;hp&quot; &quot;wt&quot; &quot;acc&quot; &quot;year&quot; We’ll continue to use mpg as the response, but now we will consider two different models. Full: mpg ~ wt + year + cyl + disp + hp + acc Null: mpg ~ wt + year Note that these are nested models, as the null model contains a subset of the predictors from the full model, and no additional predictors. Both models have an intercept \\(\\beta_0\\) as well as a coefficient in front of each of the predictors. We could then write the null hypothesis for comparing these two models as, \\[ H_0: \\beta_{\\texttt{cyl}} = \\beta_{\\texttt{disp}} = \\beta_{\\texttt{hp}} = \\beta_{\\texttt{acc}} = 0 \\] The alternative is simply that at least one of the \\(\\beta_{j}\\) from the null is not 0. To perform this test in R we first define both models, then give them to the anova() commands. null_mpg_model = lm(mpg ~ wt + year, data = autompg) #full_mpg_model = lm(mpg ~ wt + year + cyl + disp + hp + acc, data = autompg) full_mpg_model = lm(mpg ~ ., data = autompg) anova(null_mpg_model, full_mpg_model) ## Analysis of Variance Table ## ## Model 1: mpg ~ wt + year ## Model 2: mpg ~ cyl + disp + hp + wt + acc + year ## Res.Df RSS Df Sum of Sq F Pr(&gt;F) ## 1 387 4556.6 ## 2 383 4530.5 4 26.18 0.5533 0.6967 Here we have used the formula mpg ~ . to define to full model. This is the same as the commented out line. Specifically, this is a common shortcut in R which reads, “model mpg as the response with each of the remaining variables in the data frame as predictors.” Here we see that the value of the \\(F\\) statistic is 0.553, and the p-value is very large, so we fail to reject the null hypothesis at any reasonable \\(\\alpha\\) and say that none of cyl, disp, hp, and acc are significant with wt and year already in the model. Again, we verify the sums of squares and degrees of freedom directly in R. You should match these to the table from R, and use this to match R’s output to the written table above. # SSDiff sum((fitted(full_mpg_model) - fitted(null_mpg_model)) ^ 2) ## [1] 26.17981 # SSE (For Full) sum(resid(full_mpg_model) ^ 2) ## [1] 4530.466 # SST (For Null) sum(resid(null_mpg_model) ^ 2) ## [1] 4556.646 # Degrees of Freedom: Diff length(coef(full_mpg_model)) - length(coef(null_mpg_model)) ## [1] 4 # Degrees of Freedom: Full length(resid(full_mpg_model)) - length(coef(full_mpg_model)) ## [1] 383 # Degrees of Freedom: Null length(resid(null_mpg_model)) - length(coef(null_mpg_model)) ## [1] 387 9.5 Simulation Since we ignored the derivation of certain results, we will again use simulation to convince ourselves of some of the above results. In particular, we will simulate samples of size n = 100 from the model \\[ Y_i = 5 + -2 x_{i1} + 6 x_{i2} + \\epsilon_i, \\qquad i = 1, 2, \\ldots, n \\] where \\(\\epsilon_i \\sim N(0, \\sigma^2 = 16)\\). Here we have two predictors, so \\(p = 3\\). set.seed(1337) n = 100 # sample size p = 3 beta_0 = 5 beta_1 = -2 beta_2 = 6 sigma = 4 As is the norm with regression, the \\(x\\) values are considered fixed and known quantities, so we will simulate those first, and they remain the same for the rest of the simulation study. Also note we create an x0 which is all 1, which we need to create our X matrix. If you look at the matrix formulation of regression, this unit vector of all 1s is a “predictor” that puts the intercept into the model. We also calculate the C matrix for later use. x0 = rep(1, n) x1 = sample(seq(1, 10, length = n)) x2 = sample(seq(1, 10, length = n)) X = cbind(x0, x1, x2) C = solve(t(X) %*% X) We then simulate the response according the model above. Lastly, we place the two predictors and response into a data frame. Note that we do not place x0 in the data frame. This is a result of R adding an intercept by default. eps = rnorm(n, mean = 0, sd = sigma) y = beta_0 + beta_1 * x1 + beta_2 * x2 + eps sim_data = data.frame(x1, x2, y) Plotting this data and fitting the regression produces the following plot. We then calculate \\[ \\hat{\\beta} = \\left( X^\\top X \\right)^{-1}X^\\top y. \\] (beta_hat = C %*% t(X) %*% y) ## [,1] ## x0 7.290735 ## x1 -2.282176 ## x2 5.843424 Notice that these values are the same as the coefficients found using lm() in R. coef(lm(y ~ x1 + x2, data = sim_data)) ## (Intercept) x1 x2 ## 7.290735 -2.282176 5.843424 Also, these values are close to what we would expect. c(beta_0, beta_1, beta_2) ## [1] 5 -2 6 We then calculated the fitted values in order to calculate \\(s_e\\), which we see is the same as the sigma which is returned by summary(). y_hat = X %*% beta_hat (s_e = sqrt(sum((y - y_hat) ^ 2) / (n - p))) ## [1] 4.294307 summary(lm(y ~ x1 + x2, data = sim_data))$sigma ## [1] 4.294307 So far so good. Everything checks out. Now we will finally simulate from this model repeatedly in order to obtain an empirical distribution of \\(\\hat{\\beta}_2\\). We expect \\(\\hat{\\beta}_2\\) to follow a normal distribution, \\[ \\hat{\\beta}_2 \\sim N\\left(\\beta_2, \\sigma^2 C_{22} \\right). \\] In this case, \\[ \\hat{\\beta}_2 \\sim N\\left(\\mu = 6, \\sigma^2 = 16 \\times 0.0014534 = 0.0232549 \\right). \\] \\[ \\hat{\\beta}_2 \\sim N\\left(\\mu = 6, \\sigma^2 = 0.0232549 \\right). \\] Note that \\(C_{22}\\) corresponds to the element in the third row and third column since \\(\\beta_2\\) is the third parameter in the model and because R is indexed starting at 1. However, we index the \\(C\\) matrix starting at 0 to match the diagonal elements to the corresponding \\(\\beta_j\\). C[3, 3] ## [1] 0.00145343 C[2 + 1, 2 + 1] ## [1] 0.00145343 sigma ^ 2 * C[2 + 1, 2 + 1] ## [1] 0.02325487 We now perform the simulation a large number of times. Each time, we update the y variable in the data frame, leaving the x variables the same. We then fit a model, and store \\(\\hat{\\beta}_2\\). num_sims = 10000 beta_hat_2 = rep(0, num_sims) for(i in 1:num_sims) { eps = rnorm(n, mean = 0 , sd = sigma) sim_data$y = beta_0 * x0 + beta_1 * x1 + beta_2 * x2 + eps fit = lm(y ~ x1 + x2, data = sim_data) beta_hat_2[i] = coef(fit)[3] } We then see that the mean of the simulated values is close to the true value of \\(\\beta_2\\). mean(beta_hat_2) ## [1] 5.999723 beta_2 ## [1] 6 We also see that the variance of the simulated values is close to the true variance of \\(\\hat{\\beta}_2\\). \\[ \\text{Var}[\\hat{\\beta}_2] = \\sigma^2 \\cdot C_{22} = 16 \\times 0.0014534 = 0.0232549 \\] var(beta_hat_2) ## [1] 0.02343408 sigma ^ 2 * C[2 + 1, 2 + 1] ## [1] 0.02325487 The standard deviations found from the simulated data and the parent population are also very close. sd(beta_hat_2) ## [1] 0.1530819 sqrt(sigma ^ 2 * C[2 + 1, 2 + 1]) ## [1] 0.1524955 Lastly, we plot a histogram of the simulated values, and overlay the true distribution. hist(beta_hat_2, prob = TRUE, breaks = 20, xlab = expression(hat(beta)[2]), main = &quot;&quot;, border = &quot;dodgerblue&quot;) curve(dnorm(x, mean = beta_2, sd = sqrt(sigma ^ 2 * C[2 + 1, 2 + 1])), col = &quot;darkorange&quot;, add = TRUE, lwd = 3) This looks good! The simulation-based histogram appears to be Normal with mean 6 and spread of about 0.15 as you measure from center to inflection point. That matches really well with the sampling distribution of \\(\\hat{\\beta}_2 \\sim N\\left(\\mu = 6, \\sigma^2 = 0.0232549 \\right)\\). One last check, we verify the \\(68 - 95 - 99.7\\) rule. sd_bh2 = sqrt(sigma ^ 2 * C[2 + 1, 2 + 1]) # We expect these to be: 0.68, 0.95, 0.997 mean(beta_2 - 1 * sd_bh2 &lt; beta_hat_2 &amp; beta_hat_2 &lt; beta_2 + 1 * sd_bh2) ## [1] 0.6807 mean(beta_2 - 2 * sd_bh2 &lt; beta_hat_2 &amp; beta_hat_2 &lt; beta_2 + 2 * sd_bh2) ## [1] 0.9529 mean(beta_2 - 3 * sd_bh2 &lt; beta_hat_2 &amp; beta_hat_2 &lt; beta_2 + 3 * sd_bh2) ## [1] 0.9967 9.6 R Markdown The R Markdown file for this chapter can be found here: mlr.Rmd The file was created using R version 4.4.1. "],["model-building.html", "Chapter 10 Model Building 10.1 Family, Form, and Fit 10.2 Explanation versus Prediction 10.3 Summary 10.4 R Markdown", " Chapter 10 Model Building “Statisticians, like artists, have the bad habit of falling in love with their models.” — George Box Let’s take a step back and consider the process of finding a model for data at a higher level. We are attempting to find a model for a response variable \\(y\\) based on a number of predictors \\(x_1, x_2, x_3, \\ldots, x_{p-1}\\). Essentially, we are trying to discover the functional relationship between \\(y\\) and the predictors. In the previous chapter we were fitting models for a car’s fuel efficiency (mpg) as a function of its attributes (wt, year, cyl, disp, hp, acc). We also consider \\(y\\) to be a function of some noise. Rarely if ever do we expect there to be an exact functional relationship between the predictors and the response. \\[ y = f(x_1, x_2, x_3, \\ldots, x_{p-1}) + \\epsilon \\] We can think of this as \\[ \\text{response} = \\text{signal} + \\text{noise}. \\] We could consider all sorts of complicated functions for \\(f\\). You will likely encounter several ways of doing this in future machine learning courses. So far in this course we have focused on (multiple) linear regression. That is \\[ \\begin{aligned} y &amp;= f(x_1, x_2, x_3, \\ldots, x_{p-1}) + \\epsilon \\\\ &amp;= \\beta_0 + \\beta_1 x_{1} + \\beta_2 x_{2} + \\cdots + \\beta_{p-1} x_{p-1} + \\epsilon \\end{aligned} \\] In the big picture of possible models that we could fit to this data, this is a rather restrictive model. What do we mean by a restrictive model? 10.1 Family, Form, and Fit When modeling data, there are a number of choices that need to be made. What family of models will be considered? What form of the model will be used? How will the model be fit? Let’s work backwards and discuss each of these. 10.1.1 Fit Consider one of the simplest models we could fit to data, simple linear regression. \\[ y = f(x_1, x_2, x_3, \\ldots, x_{p-1}) + \\epsilon = \\beta_0 + \\beta_1 x_{1} + \\epsilon \\] So here, despite having multiple predictors, we chose to use only one. How is this model fit? We will almost exclusively use the method of least squares, but recall, we had seen alternative methods of fitting this model. \\[ \\underset{\\beta_0, \\beta_1}{\\mathrm{argmin}} \\max|y_i - (\\beta_0 + \\beta_1 x_i)| \\] \\[ \\underset{\\beta_0, \\beta_1}{\\mathrm{argmin}} \\sum_{i = 1}^{n}|y_i - (\\beta_0 + \\beta_1 x_i)| \\] \\[ \\underset{\\beta_0, \\beta_1}{\\mathrm{argmin}} \\sum_{i = 1}^{n}(y_i - (\\beta_0 + \\beta_1 x_i))^2 \\] Any of these methods (we will always use the last, least squares) will obtain estimates of the unknown parameters \\(\\beta_0\\) and \\(\\beta_1\\). Since those are the only unknowns of the specified model, we have then fit the model. The fitted model is then \\[ \\hat{y} = \\hat{f}(x_1, x_2, x_3, \\ldots, x_{p-1}) = \\hat{\\beta}_0 + \\hat{\\beta}_1 x_{1} \\] Note that, now we have dropped the term for the noise. We don’t make any effort to model the noise, only the signal. 10.1.2 Form What are the different forms a model could take? Currently, for the linear models we have considered, the only method for altering the form of the model is to control the predictors used. For example, one form of the multiple linear regression model is simple linear regression. \\[ y = f(x_1, x_2, x_3, \\ldots, x_{p-1}) + \\epsilon = \\beta_0 + \\beta_1 x_{1} + \\epsilon \\] We could also consider a SLR model with a different predictor, thus altering the form of the model. \\[ y = f(x_1, x_2, x_3, \\ldots, x_{p-1}) + \\epsilon = \\beta_0 + \\beta_2 x_{2} + \\epsilon \\] Often, we’ll use multiple predictors in our model. Very often, we will at least try a model with all possible predictors. \\[ \\begin{aligned} y &amp;= f(x_1, x_2, x_3, \\ldots, x_{p-1}) + \\epsilon \\\\ &amp;= \\beta_0 + \\beta_1 x_{1} + \\beta_2 x_{2} + \\cdots + \\beta_{p-1} x_{p-1} + \\epsilon \\end{aligned} \\] We could also use some, but not all of the predictors. \\[ \\begin{aligned} y &amp;= f(x_1, x_2, x_3, \\ldots, x_{p-1}) + \\epsilon \\\\ &amp;= \\beta_0 + \\beta_1 x_{1} + \\beta_3 x_{3} + \\beta_5 x_{5} + \\epsilon \\end{aligned} \\] These forms are restrictive in two senses. First, they only allow for linear relationships between the response and the predictors. This seems like an obvious restriction of linear models, but in fact, we will soon see how to use linear models for non-linear relationships. (It will involve transforming variables.) Second, how one variable affects the response is the same for any values of the other predictors. Soon we will see how to create models where the effect of \\(x_{1}\\) can be different for different values of \\(x_{2}\\). We will discuss the concept of interaction. 10.1.3 Family A family of models is a broader grouping of many possible forms of a model. For example, above we saw several forms of models from the family of linear models. We will only ever concern ourselves with linear models, which model a response as a linear combination of predictors. There are certainly other families of models. For example, there are several families of non-parametric regression. Smoothing is a broad family of models. As are regression trees. In linear regression, we specified models with parameters, \\(\\beta_j\\) and fit the model by finding the best values of these parameters. This is a parametric approach. A non-parametric approach skips the step of specifying a model with parameters, and is often described as more of an algorithm. Non-parametric models are often used in machine learning. Here, SLR (parametric) is used on the left, while smoothing (non-parametric) is used on the right. SLR finds the best slope and intercept. Smoothing produces the fitted \\(y\\) value at a particular \\(x\\) value by considering the \\(y\\) values of the data in a neighborhood of the \\(x\\) value considered. (Local smoothing.) Why the focus on linear models? Two big reasons: Linear models are the go-to model. Linear models have been around for a long time, and are computationally easy. A linear model may not be the final model you use, but often, it should be the first model you try. The ideas behind linear models can be easily transferred to other modeling techniques. 10.1.4 Assumed Model, Fitted Model When searching for a model, we often need to make assumptions. These assumptions are codified in the family and form of the model. For example \\[ y = \\beta_0 + \\beta_1 x_{1} + \\beta_3 x_{3} + \\beta_5 x_{5} + \\epsilon \\] assumes that \\(y\\) is a linear combination of \\(x_{1}\\), \\(x_{3}\\), and \\(x_{5}\\) as well as some noise. This assumes that the effect of \\(x_{1}\\) on \\(y\\) is \\(\\beta_1\\), which is the same for all values of \\(x_{3}\\) and \\(x_{5}\\). That is, we are using the family of linear models with a particular form. Suppose we then fit this model to some data and obtain the fitted model. For example, in R we would use fit = lm(y ~ x1 + x3 + x5, data = some_data) This is R’s way of saying the family is linear and specifying the form from above. An additive model with the specified predictors as well as an intercept. We then obtain \\[ \\hat{y} = 1.5 + 0.9 x_{1} + 1.1 x_{3} + 2.3 x_{5}. \\] This is our best guess for the function \\(f\\) in \\[ y = f(x_1, x_2, x_3, \\ldots, x_{p-1}) + \\epsilon \\] for the assumed family and form. Fitting a model only gives us the best fit for the family and form that we specify. So the natural question is; how do we choose the correct family and form? We’ll focus on form since we are focusing on the family of linear models. 10.2 Explanation versus Prediction What is the purpose of fitting a model to data? Usually it is to accomplish one of two goals. We can use a model to explain the relationship between the response and the predictors. Models can also be used to predict the response based on the predictors. Often, a good model will do both, but we’ll discuss both goals separately since the processes of finding models for explaining and predicting have some differences. For our purposes, since we are only considering linear models, searching for a good model is essentially searching for a good form of a model. 10.2.1 Explanation If the goal of a model is to explain the relationship between the response and the predictors, we are looking for a model that is small and interpretable, but still fits the data well. When discussing linear models, the size of a model is essentially the number of \\(\\beta\\) parameters used. Suppose we would like to find a model that explains fuel efficiency (mpg) based on a car’s attributes (wt, year, cyl, disp, hp, acc). Perhaps we are a car manufacturer trying to engineer a fuel-efficient vehicle. If this is the case, we are interested in both which predictor variables are useful for explaining the car’s fuel efficiency, as well as how those variables affect fuel efficiency. By understanding this relationship, we can use this knowledge to our advantage when designing a car. To explain a relationship, we are interested in keeping models as small as possible, since smaller models are easy to interpret. The fewer predictors the fewer considerations we need to make in our design process. Note that linear models of any size are rather interpretable to begin with. Later in your data analysis careers, you will see more complicated models that may fit data better, but are much harder, if not impossible to interpret. These models aren’t nearly as useful for explaining a relationship. This is another reason to always attempt a linear model. If it fits as well as more complicated methods, it will be the easiest to understand. To find small and interpretable models, we will eventually use selection procedures, which search among many possible forms of a model. For now we will do this in a more ad-hoc manner using inference techniques we have already encountered. To use inference as we have seen it, we need an additional assumption in addition to the family and form of the model. \\[ y = \\beta_0 + \\beta_1 x_{1} + \\beta_3 x_{3} + \\beta_5 x_{5} + \\epsilon \\] Our additional assumption is about the error term. \\[ \\epsilon \\sim N(0, \\sigma^2) \\] This assumption that the errors are normally distributed with some common variance is the key to all of the inference we have done so far. We will discuss this in great detail later. So with our inference tools (ANOVA and \\(t\\)-test) we have two potential strategies. Start with a very small model (no predictors) and attempt to add predictors. Or, start with a big model (all predictors) and attempt to remove predictors. 10.2.1.1 Correlation and Causation A word of caution when using a model to explain a relationship. There are two terms often used to describe a relationship between two variables: causation and correlation. Correlation is often also referred to as association. Just because two variables are correlated does not necessarily mean that one causes the other. For example, consider modeling mpg as only a function of hp. plot(mpg ~ hp, data = autompg, col = &quot;dodgerblue&quot;, pch = 20, cex = 1.5) Does an increase in horsepower cause a drop in fuel efficiency? Or, perhaps the causality is reversed and an increase in fuel efficiency causes a decrease in horsepower. Or, perhaps there is a third variable that explains both! The issue here is that we have observational data. With observational data, we can only detect associations. To speak with confidence about causality, we would need to run experiments. Often, this decision is made for us, before we ever see data, so we can only modify our interpretation. This is a concept that you should encounter often in your statistics education. For some further reading, and some related fallacies, see: Wikipedia: Correlation does not imply causation. We’ll discuss this further when we discuss experimental design and traditional ANOVA techniques. (All of which has recently been re-branded as A/B testing.) 10.2.2 Prediction If the goal of a model is to predict the response, then the only consideration is how well the model fits the data. For this, we will need a metric. In regression problems, this is most often RMSE. \\[ \\text{RMSE}(\\text{model, data}) = \\sqrt{\\frac{1}{n} \\sum_{i = 1}^{n}(y_i - \\hat{y}_i)^2} \\] where \\(y_i\\) are the actual values of the response for the given data \\(\\hat{y}_i\\) are the predicted values using the fitted model and the predictors from the data Correlation and causation are not an issue here. If a predictor is correlated with the response, it is useful for prediction. For example, in elementary school aged children their shoe size certainly doesn’t cause them to read at a higher level, however we could very easily use shoe size to make a prediction about a child’s reading ability. The larger their shoe size, the better they read. There’s a lurking variable here though, their age! (Don’t send your kids to school with size 14 shoes, it won’t make them read better!) Also, since we are not performing inference, the extra assumption about the errors is not needed. The only thing we care about is how close the fitted model is to the data. Least squares is least squares. For a specified model, it will find the values of the parameters which will minimize the squared error loss. Your results might be largely uninterpretable and useless for inference, but for prediction none of that matters. Suppose instead of the manufacturer who would like to build a car, we are a consumer who wishes to purchase a new car. However this particular car is so new, it has not been rigorously tested, so we are unsure of what fuel efficiency to expect. (And, as skeptics, we don’t trust what the manufacturer is telling us.) In this case, we would like to use the model to help predict the fuel efficiency of this car based on its attributes, which are the predictors of the model. The smaller the errors the model makes, the more confident we are in its prediction. 10.2.2.1 Test-Train Split The trouble with using RMSE to identify how well a model fits data is that RMSE is always (equal or) lower for a larger model. This would suggest that we should always use the largest model possible when looking for a model that predicts well. The problem with this is the potential to overfit to the data. So, we want a model that fits well, but does not overfit. To understand overfitting, we need to think about applying a model to seen and unseen data. Suppose we fit a model using all data available and we evaluate RMSE on this fitted model and all of the seen data. We will call this data the training data, and this RMSE the train RMSE. Now, suppose we magically encounter some additional data. To truly assess how well the model predicts, we should evaluate how well our model predicts the response of this data. We will call this data the test data and this RMSE the test RMSE. Train RMSE: model fit on seen data, evaluated on seen data Test RMSE: model fit on seen data, evaluated on unseen data Below, we simulate some data and fit two models. We will call the solid blue line the “simple” model. The dashed orange line will be called the “complex” model, which was fit with methods we do not yet know. The left panel shows the data that was used to fit the two models. Clearly the “complex” model fits the data much better. The right panel shows additional data that was simulated in the same manner as the original data. Here we see that the “simple” model fits much better. The dashed orange line almost seems random. Model Train RMSE Test RMSE Simple 1.71 1.45 Complex 1.41 2.07 The more “complex,” wiggly, model fits the training data much better as it has a much lower train RMSE. However, we see that the “simple” model fits the test data much better, with a much lower test RMSE. This means that the complex model has overfit to the data, and we prefer the simple model. When choosing a model for prediction, we prefer a model that predicts unseen data. In practice, you can’t simply generate more data to evaluate your models. Instead we split existing data into data used to fit the model (train) and data used to evaluate the model (test). Never fit a model with test data. 10.3 Summary Models can be used to explain relationships and predict observations. When using model to, explain; we prefer small and interpretable models. predict; we prefer models that make the smallest errors possible, without overfitting. Linear models can accomplish both these goals. Later, we will see that often a linear model that accomplishes one of these goals, usually accomplishes the other. 10.4 R Markdown The R Markdown file for this chapter can be found here: model-building.Rmd The file was created using R version 4.4.1. "],["categorical-predictors-and-interactions.html", "Chapter 11 Categorical Predictors and Interactions 11.1 Dummy Variables 11.2 Interactions 11.3 Factor Variables 11.4 Parameterization 11.5 Building Larger Models 11.6 R Markdown", " Chapter 11 Categorical Predictors and Interactions “The greatest value of a picture is when it forces us to notice what we never expected to see.” — John Tukey After reading this chapter you will be able to: Include and interpret categorical variables in a linear regression model by way of dummy variables. Understand the implications of using a model with a categorical variable in two ways: levels serving as unique predictors versus levels serving as a comparison to a baseline. Construct and interpret linear regression models with interaction terms. Identify categorical variables in a data set and convert them into factor variables, if necessary, using R. So far in each of our analyses, we have only used numeric variables as predictors. We have also only used additive models, meaning the effect any predictor had on the response was not dependent on the other predictors. In this chapter, we will remove both of these restrictions. We will fit models with categorical predictors, and use models that allow predictors to interact. The mathematics of multiple regression will remain largely unchanging, however, we will pay close attention to interpretation, as well as some difference in R usage. 11.1 Dummy Variables For this chapter, we will briefly use the built-in dataset mtcars before returning to our autompg dataset that we created in the last chapter. The mtcars dataset is somewhat smaller, so we’ll quickly take a look at the entire dataset. mtcars ## mpg cyl disp hp drat wt qsec vs am gear carb ## Mazda RX4 21.0 6 160.0 110 3.90 2.620 16.46 0 1 4 4 ## Mazda RX4 Wag 21.0 6 160.0 110 3.90 2.875 17.02 0 1 4 4 ## Datsun 710 22.8 4 108.0 93 3.85 2.320 18.61 1 1 4 1 ## Hornet 4 Drive 21.4 6 258.0 110 3.08 3.215 19.44 1 0 3 1 ## Hornet Sportabout 18.7 8 360.0 175 3.15 3.440 17.02 0 0 3 2 ## Valiant 18.1 6 225.0 105 2.76 3.460 20.22 1 0 3 1 ## Duster 360 14.3 8 360.0 245 3.21 3.570 15.84 0 0 3 4 ## Merc 240D 24.4 4 146.7 62 3.69 3.190 20.00 1 0 4 2 ## Merc 230 22.8 4 140.8 95 3.92 3.150 22.90 1 0 4 2 ## Merc 280 19.2 6 167.6 123 3.92 3.440 18.30 1 0 4 4 ## Merc 280C 17.8 6 167.6 123 3.92 3.440 18.90 1 0 4 4 ## Merc 450SE 16.4 8 275.8 180 3.07 4.070 17.40 0 0 3 3 ## Merc 450SL 17.3 8 275.8 180 3.07 3.730 17.60 0 0 3 3 ## Merc 450SLC 15.2 8 275.8 180 3.07 3.780 18.00 0 0 3 3 ## Cadillac Fleetwood 10.4 8 472.0 205 2.93 5.250 17.98 0 0 3 4 ## Lincoln Continental 10.4 8 460.0 215 3.00 5.424 17.82 0 0 3 4 ## Chrysler Imperial 14.7 8 440.0 230 3.23 5.345 17.42 0 0 3 4 ## Fiat 128 32.4 4 78.7 66 4.08 2.200 19.47 1 1 4 1 ## Honda Civic 30.4 4 75.7 52 4.93 1.615 18.52 1 1 4 2 ## Toyota Corolla 33.9 4 71.1 65 4.22 1.835 19.90 1 1 4 1 ## Toyota Corona 21.5 4 120.1 97 3.70 2.465 20.01 1 0 3 1 ## Dodge Challenger 15.5 8 318.0 150 2.76 3.520 16.87 0 0 3 2 ## AMC Javelin 15.2 8 304.0 150 3.15 3.435 17.30 0 0 3 2 ## Camaro Z28 13.3 8 350.0 245 3.73 3.840 15.41 0 0 3 4 ## Pontiac Firebird 19.2 8 400.0 175 3.08 3.845 17.05 0 0 3 2 ## Fiat X1-9 27.3 4 79.0 66 4.08 1.935 18.90 1 1 4 1 ## Porsche 914-2 26.0 4 120.3 91 4.43 2.140 16.70 0 1 5 2 ## Lotus Europa 30.4 4 95.1 113 3.77 1.513 16.90 1 1 5 2 ## Ford Pantera L 15.8 8 351.0 264 4.22 3.170 14.50 0 1 5 4 ## Ferrari Dino 19.7 6 145.0 175 3.62 2.770 15.50 0 1 5 6 ## Maserati Bora 15.0 8 301.0 335 3.54 3.570 14.60 0 1 5 8 ## Volvo 142E 21.4 4 121.0 109 4.11 2.780 18.60 1 1 4 2 We will be interested in three of the variables: mpg, hp, and am. mpg: fuel efficiency, in miles per gallon. hp: horsepower, in foot-pounds per second. am: transmission. Automatic or manual. As we often do, we will start by plotting the data. We are interested in mpg as the response variable, and hp as a predictor. plot(mpg ~ hp, data = mtcars, cex = 2) Since we are also interested in the transmission type, we could also label the points accordingly. plot(mpg ~ hp, data = mtcars, col = am + 1, pch = am + 1, cex = 2) legend(&quot;topright&quot;, c(&quot;Automatic&quot;, &quot;Manual&quot;), col = c(1, 2), pch = c(1, 2)) We used a common R “trick” when plotting this data. The am variable takes two possible values; 0 for automatic transmission, and 1 for manual transmissions. R can use numbers to represent colors, however the color for 0 is white. So we take the am vector and add 1 to it. Then observations with automatic transmissions are now represented by 1, which is black in R, and manual transmission are represented by 2, which is red in R. (Note, we are only adding 1 inside the call to plot(), we are not actually modifying the values stored in am.) We now fit the SLR model \\[ Y = \\beta_0 + \\beta_1 x_1 + \\epsilon, \\] where \\(Y\\) is mpg and \\(x_1\\) is hp. For notational brevity, we drop the index \\(i\\) for observations. mpg_hp_slr = lm(mpg ~ hp, data = mtcars) We then re-plot the data and add the fitted line to the plot. plot(mpg ~ hp, data = mtcars, col = am + 1, pch = am + 1, cex = 2) abline(mpg_hp_slr, lwd = 3, col = &quot;grey&quot;) legend(&quot;topright&quot;, c(&quot;Automatic&quot;, &quot;Manual&quot;), col = c(1, 2), pch = c(1, 2)) We should notice a pattern here. The red, manual observations largely fall above the line, while the black, automatic observations are mostly below the line. This means our model underestimates the fuel efficiency of manual transmissions, and overestimates the fuel efficiency of automatic transmissions. To correct for this, we will add a predictor to our model, namely, am as \\(x_2\\). Our new model is \\[ Y = \\beta_0 + \\beta_1 x_1 + \\beta_2 x_2 + \\epsilon, \\] where \\(x_1\\) and \\(Y\\) remain the same, but now \\[ x_2 = \\begin{cases} 1 &amp; \\text{manual transmission} \\\\ 0 &amp; \\text{automatic transmission} \\end{cases}. \\] In this case, we call \\(x_2\\) a dummy variable. A dummy variable is somewhat unfortunately named, as it is in no way “dumb.” In fact, it is actually somewhat clever. A dummy variable is a numerical variable that is used in a regression analysis to “code” for a binary categorical variable. Let’s see how this works. First, note that am is already a dummy variable, since it uses the values 0 and 1 to represent automatic and manual transmissions. Often, a variable like am would store the character values auto and man and we would either have to convert these to 0 and 1, or, as we will see later, R will take care of creating dummy variables for us. So, to fit the above model, we do so like any other multiple regression model we have seen before. mpg_hp_add = lm(mpg ~ hp + am, data = mtcars) Briefly checking the output, we see that R has estimated the three \\(\\beta\\) parameters. mpg_hp_add ## ## Call: ## lm(formula = mpg ~ hp + am, data = mtcars) ## ## Coefficients: ## (Intercept) hp am ## 26.58491 -0.05889 5.27709 Since \\(x_2\\) can only take values 0 and 1, we can effectively write two different models, one for manual and one for automatic transmissions. For automatic transmissions, that is \\(x_2 = 0\\), we have, \\[ Y = \\beta_0 + \\beta_1 x_1 + \\epsilon. \\] Then for manual transmissions, that is \\(x_2 = 1\\), we have, \\[ Y = (\\beta_0 + \\beta_2) + \\beta_1 x_1 + \\epsilon. \\] Notice that these models share the same slope, \\(\\beta_1\\), but have different intercepts, differing by \\(\\beta_2\\). So the change in mpg is the same for both models, but on average mpg differs by \\(\\beta_2\\) between the two transmission types. We’ll now calculate the estimated slope and intercept of these two models so that we can add them to a plot. Note that: \\(\\hat{\\beta}_0\\) = coef(mpg_hp_add)[1] = 26.5849137 \\(\\hat{\\beta}_1\\) = coef(mpg_hp_add)[2] = -0.0588878 \\(\\hat{\\beta}_2\\) = coef(mpg_hp_add)[3] = 5.2770853 We can then combine these to calculate the estimated slope and intercepts. int_auto = coef(mpg_hp_add)[1] int_manu = coef(mpg_hp_add)[1] + coef(mpg_hp_add)[3] slope_auto = coef(mpg_hp_add)[2] slope_manu = coef(mpg_hp_add)[2] Re-plotting the data, we use these slopes and intercepts to add the “two” fitted models to the plot. plot(mpg ~ hp, data = mtcars, col = am + 1, pch = am + 1, cex = 2) abline(int_auto, slope_auto, col = 1, lty = 1, lwd = 2) # add line for auto abline(int_manu, slope_manu, col = 2, lty = 2, lwd = 2) # add line for manual legend(&quot;topright&quot;, c(&quot;Automatic&quot;, &quot;Manual&quot;), col = c(1, 2), pch = c(1, 2)) We notice right away that the points are no longer systematically incorrect. The red, manual observations vary about the red line in no particular pattern without underestimating the observations as before. The black, automatic points vary about the black line, also without an obvious pattern. They say a picture is worth a thousand words, but as a statistician, sometimes a picture is worth an entire analysis. The above picture makes it plainly obvious that \\(\\beta_2\\) is significant, but let’s verify mathematically. Essentially we would like to test: \\[ H_0: \\beta_2 = 0 \\quad \\text{vs} \\quad H_1: \\beta_2 \\neq 0. \\] This is nothing new. Again, the math is the same as the multiple regression analyses we have seen before. We could perform either a \\(t\\) or \\(F\\) test here. The only difference is a slight change in interpretation. We could think of this as testing a model with a single line (\\(H_0\\)) against a model that allows two lines (\\(H_1\\)). To obtain the test statistic and p-value for the \\(t\\)-test, we would use summary(mpg_hp_add)$coefficients[&quot;am&quot;,] ## Estimate Std. Error t value Pr(&gt;|t|) ## 5.277085e+00 1.079541e+00 4.888270e+00 3.460318e-05 To do the same for the \\(F\\) test, we would use anova(mpg_hp_slr, mpg_hp_add) ## Analysis of Variance Table ## ## Model 1: mpg ~ hp ## Model 2: mpg ~ hp + am ## Res.Df RSS Df Sum of Sq F Pr(&gt;F) ## 1 30 447.67 ## 2 29 245.44 1 202.24 23.895 3.46e-05 *** ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 Notice that these are indeed testing the same thing, as the p-values are exactly equal. (And the \\(F\\) test statistic is the \\(t\\) test statistic squared.) Recapping some interpretations: \\(\\hat{\\beta}_0 = 26.5849137\\) is the estimated average mpg for a car with an automatic transmission and 0 hp. \\(\\hat{\\beta}_0 + \\hat{\\beta}_2 = 31.8619991\\) is the estimated average mpg for a car with a manual transmission and 0 hp. \\(\\hat{\\beta}_2 = 5.2770853\\) is the estimated difference in average mpg for cars with manual transmissions as compared to those with automatic transmission, for any hp. \\(\\hat{\\beta}_1 = -0.0588878\\) is the estimated change in average mpg for an increase in one hp, for either transmission type. We should take special notice of those last two. In the model, \\[ Y = \\beta_0 + \\beta_1 x_1 + \\beta_2 x_2 + \\epsilon, \\] we see \\(\\beta_1\\) is the average change in \\(Y\\) for an increase in \\(x_1\\), no matter the value of \\(x_2\\). Also, \\(\\beta_2\\) is always the difference in the average of \\(Y\\) for any value of \\(x_1\\). These are two restrictions we won’t always want, so we need a way to specify a more flexible model. Here we restricted ourselves to a single numerical predictor \\(x_1\\) and one dummy variable \\(x_2\\). However, the concept of a dummy variable can be used with larger multiple regression models. We only use a single numerical predictor here for ease of visualization since we can think of the “two lines” interpretation. But in general, we can think of a dummy variable as creating “two models,” one for each category of a binary categorical variable. 11.2 Interactions To remove the “same slope” restriction, we will now discuss interaction. To illustrate this concept, we will return to the autompg dataset we created in the last chapter, with a few more modifications. # read data frame from the web autompg = read.table( &quot;http://archive.ics.uci.edu/ml/machine-learning-databases/auto-mpg/auto-mpg.data&quot;, quote = &quot;\\&quot;&quot;, comment.char = &quot;&quot;, stringsAsFactors = FALSE) # give the dataframe headers colnames(autompg) = c(&quot;mpg&quot;, &quot;cyl&quot;, &quot;disp&quot;, &quot;hp&quot;, &quot;wt&quot;, &quot;acc&quot;, &quot;year&quot;, &quot;origin&quot;, &quot;name&quot;) # remove missing data, which is stored as &quot;?&quot; autompg = subset(autompg, autompg$hp != &quot;?&quot;) # remove the plymouth reliant, as it causes some issues autompg = subset(autompg, autompg$name != &quot;plymouth reliant&quot;) # give the dataset row names, based on the engine, year and name rownames(autompg) = paste(autompg$cyl, &quot;cylinder&quot;, autompg$year, autompg$name) # remove the variable for name autompg = subset(autompg, select = c(&quot;mpg&quot;, &quot;cyl&quot;, &quot;disp&quot;, &quot;hp&quot;, &quot;wt&quot;, &quot;acc&quot;, &quot;year&quot;, &quot;origin&quot;)) # change horsepower from character to numeric autompg$hp = as.numeric(autompg$hp) # create a dummy variable for foreign vs domestic cars. domestic = 1. autompg$domestic = as.numeric(autompg$origin == 1) # remove 3 and 5 cylinder cars (which are very rare.) autompg = autompg[autompg$cyl != 5,] autompg = autompg[autompg$cyl != 3,] # the following line would verify the remaining cylinder possibilities are 4, 6, 8 #unique(autompg$cyl) # change cyl to a factor variable autompg$cyl = as.factor(autompg$cyl) str(autompg) ## &#39;data.frame&#39;: 383 obs. of 9 variables: ## $ mpg : num 18 15 18 16 17 15 14 14 14 15 ... ## $ cyl : Factor w/ 3 levels &quot;4&quot;,&quot;6&quot;,&quot;8&quot;: 3 3 3 3 3 3 3 3 3 3 ... ## $ disp : num 307 350 318 304 302 429 454 440 455 390 ... ## $ hp : num 130 165 150 150 140 198 220 215 225 190 ... ## $ wt : num 3504 3693 3436 3433 3449 ... ## $ acc : num 12 11.5 11 12 10.5 10 9 8.5 10 8.5 ... ## $ year : int 70 70 70 70 70 70 70 70 70 70 ... ## $ origin : int 1 1 1 1 1 1 1 1 1 1 ... ## $ domestic: num 1 1 1 1 1 1 1 1 1 1 ... We’ve removed cars with 3 and 5 cylinders , as well as created a new variable domestic which indicates whether or not a car was built in the United States. Removing the 3 and 5 cylinders is simply for ease of demonstration later in the chapter and would not be done in practice. The new variable domestic takes the value 1 if the car was built in the United States, and 0 otherwise, which we will refer to as “foreign.” (We are arbitrarily using the United States as the reference point here.) We have also made cyl and origin into factor variables, which we will discuss later. We’ll now be concerned with three variables: mpg, disp, and domestic. We will use mpg as the response. We can fit a model, \\[ Y = \\beta_0 + \\beta_1 x_1 + \\beta_2 x_2 + \\epsilon, \\] where \\(Y\\) is mpg, the fuel efficiency in miles per gallon, \\(x_1\\) is disp, the displacement in cubic inches, \\(x_2\\) is domestic as described above, which is a dummy variable. \\[ x_2 = \\begin{cases} 1 &amp; \\text{Domestic} \\\\ 0 &amp; \\text{Foreign} \\end{cases} \\] We will fit this model, extract the slope and intercept for the “two lines,” plot the data and add the lines. mpg_disp_add = lm(mpg ~ disp + domestic, data = autompg) int_for = coef(mpg_disp_add)[1] int_dom = coef(mpg_disp_add)[1] + coef(mpg_disp_add)[3] slope_for = coef(mpg_disp_add)[2] slope_dom = coef(mpg_disp_add)[2] plot(mpg ~ disp, data = autompg, col = domestic + 1, pch = domestic + 1) abline(int_for, slope_for, col = 1, lty = 1, lwd = 2) # add line for foreign cars abline(int_dom, slope_dom, col = 2, lty = 2, lwd = 2) # add line for domestic cars legend(&quot;topright&quot;, c(&quot;Foreign&quot;, &quot;Domestic&quot;), pch = c(1, 2), col = c(1, 2)) This is a model that allows for two parallel lines, meaning the mpg can be different on average between foreign and domestic cars of the same engine displacement, but the change in average mpg for an increase in displacement is the same for both. We can see this model isn’t doing very well here. The red line fits the red points fairly well, but the black line isn’t doing very well for the black points, it should clearly have a more negative slope. Essentially, we would like a model that allows for two different slopes. Consider the following model, \\[ Y = \\beta_0 + \\beta_1 x_1 + \\beta_2 x_2 + \\beta_3 x_1 x_2 + \\epsilon, \\] where \\(x_1\\), \\(x_2\\), and \\(Y\\) are the same as before, but we have added a new interaction term \\(x_1 x_2\\) which multiplies \\(x_1\\) and \\(x_2\\), so we also have an additional \\(\\beta\\) parameter \\(\\beta_3\\). This model essentially creates two slopes and two intercepts, \\(\\beta_2\\) being the difference in intercepts and \\(\\beta_3\\) being the difference in slopes. To see this, we will break down the model into the two “sub-models” for foreign and domestic cars. For foreign cars, that is \\(x_2 = 0\\), we have \\[ Y = \\beta_0 + \\beta_1 x_1 + \\epsilon. \\] For domestic cars, that is \\(x_2 = 1\\), we have \\[ Y = (\\beta_0 + \\beta_2) + (\\beta_1 + \\beta_3) x_1 + \\epsilon. \\] These two models have both different slopes and intercepts. \\(\\beta_0\\) is the average mpg for a foreign car with 0 disp. \\(\\beta_1\\) is the change in average mpg for an increase of one disp, for foreign cars. \\(\\beta_0 + \\beta_2\\) is the average mpg for a domestic car with 0 disp. \\(\\beta_1 + \\beta_3\\) is the change in average mpg for an increase of one disp, for domestic cars. How do we fit this model in R? There are a number of ways. One method would be to simply create a new variable, then fit a model like any other. autompg$x3 = autompg$disp * autompg$domestic # THIS CODE NOT RUN! do_not_do_this = lm(mpg ~ disp + domestic + x3, data = autompg) # THIS CODE NOT RUN! You should only do this as a last resort. We greatly prefer not to have to modify our data simply to fit a model. Instead, we can tell R we would like to use the existing data with an interaction term, which it will create automatically when we use the : operator. mpg_disp_int = lm(mpg ~ disp + domestic + disp:domestic, data = autompg) An alternative method, which will fit the exact same model as above would be to use the * operator. This method automatically creates the interaction term, as well as any “lower order terms,” which in this case are the first order terms for disp and domestic mpg_disp_int2 = lm(mpg ~ disp * domestic, data = autompg) We can quickly verify that these are doing the same thing. coef(mpg_disp_int) ## (Intercept) disp domestic disp:domestic ## 46.0548423 -0.1569239 -12.5754714 0.1025184 coef(mpg_disp_int2) ## (Intercept) disp domestic disp:domestic ## 46.0548423 -0.1569239 -12.5754714 0.1025184 We see that both the variables, and their coefficient estimates are indeed the same for both models. summary(mpg_disp_int) ## ## Call: ## lm(formula = mpg ~ disp + domestic + disp:domestic, data = autompg) ## ## Residuals: ## Min 1Q Median 3Q Max ## -10.8332 -2.8956 -0.8332 2.2828 18.7749 ## ## Coefficients: ## Estimate Std. Error t value Pr(&gt;|t|) ## (Intercept) 46.05484 1.80582 25.504 &lt; 2e-16 *** ## disp -0.15692 0.01668 -9.407 &lt; 2e-16 *** ## domestic -12.57547 1.95644 -6.428 3.90e-10 *** ## disp:domestic 0.10252 0.01692 6.060 3.29e-09 *** ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 ## ## Residual standard error: 4.308 on 379 degrees of freedom ## Multiple R-squared: 0.7011, Adjusted R-squared: 0.6987 ## F-statistic: 296.3 on 3 and 379 DF, p-value: &lt; 2.2e-16 We see that using summary() gives the usual output for a multiple regression model. We pay close attention to the row for disp:domestic which tests, \\[ H_0: \\beta_3 = 0. \\] In this case, testing for \\(\\beta_3 = 0\\) is testing for two lines with parallel slopes versus two lines with possibly different slopes. The disp:domestic line in the summary() output uses a \\(t\\)-test to perform the test. We could also use an ANOVA \\(F\\)-test. The additive model without interaction is our null model, and the interaction model is the alternative. anova(mpg_disp_add, mpg_disp_int) ## Analysis of Variance Table ## ## Model 1: mpg ~ disp + domestic ## Model 2: mpg ~ disp + domestic + disp:domestic ## Res.Df RSS Df Sum of Sq F Pr(&gt;F) ## 1 380 7714.0 ## 2 379 7032.6 1 681.36 36.72 3.294e-09 *** ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 Again we see this test has the same p-value as the \\(t\\)-test. Also the p-value is extremely low, so between the two, we choose the interaction model. int_for = coef(mpg_disp_int)[1] int_dom = coef(mpg_disp_int)[1] + coef(mpg_disp_int)[3] slope_for = coef(mpg_disp_int)[2] slope_dom = coef(mpg_disp_int)[2] + coef(mpg_disp_int)[4] Here we again calculate the slope and intercepts for the two lines for use in plotting. plot(mpg ~ disp, data = autompg, col = domestic + 1, pch = domestic + 1) abline(int_for, slope_for, col = 1, lty = 1, lwd = 2) # line for foreign cars abline(int_dom, slope_dom, col = 2, lty = 2, lwd = 2) # line for domestic cars legend(&quot;topright&quot;, c(&quot;Foreign&quot;, &quot;Domestic&quot;), pch = c(1, 2), col = c(1, 2)) We see that these lines fit the data much better, which matches the result of our tests. So far we have only seen interaction between a categorical variable (domestic) and a numerical variable (disp). While this is easy to visualize, since it allows for different slopes for two lines, it is not the only type of interaction we can use in a model. We can also consider interactions between two numerical variables. Consider the model, \\[ Y = \\beta_0 + \\beta_1 x_1 + \\beta_2 x_2 + \\beta_3 x_1 x_2 + \\epsilon, \\] where \\(Y\\) is mpg, the fuel efficiency in miles per gallon, \\(x_1\\) is disp, the displacement in cubic inches, \\(x_2\\) is hp, the horsepower, in foot-pounds per second. How does mpg change based on disp in this model? We can rearrange some terms to see how. \\[ Y = \\beta_0 + (\\beta_1 + \\beta_3 x_2) x_1 + \\beta_2 x_2 + \\epsilon \\] So, for a one unit increase in \\(x_1\\) (disp), the mean of \\(Y\\) (mpg) increases \\(\\beta_1 + \\beta_3 x_2\\), which is a different value depending on the value of \\(x_2\\) (hp)! Since we’re now working in three dimensions, this model can’t be easily justified via visualizations like the previous example. Instead, we will have to rely on a test. mpg_disp_add_hp = lm(mpg ~ disp + hp, data = autompg) mpg_disp_int_hp = lm(mpg ~ disp * hp, data = autompg) summary(mpg_disp_int_hp) ## ## Call: ## lm(formula = mpg ~ disp * hp, data = autompg) ## ## Residuals: ## Min 1Q Median 3Q Max ## -10.7849 -2.3104 -0.5699 2.1453 17.9211 ## ## Coefficients: ## Estimate Std. Error t value Pr(&gt;|t|) ## (Intercept) 5.241e+01 1.523e+00 34.42 &lt;2e-16 *** ## disp -1.002e-01 6.638e-03 -15.09 &lt;2e-16 *** ## hp -2.198e-01 1.987e-02 -11.06 &lt;2e-16 *** ## disp:hp 5.658e-04 5.165e-05 10.96 &lt;2e-16 *** ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 ## ## Residual standard error: 3.896 on 379 degrees of freedom ## Multiple R-squared: 0.7554, Adjusted R-squared: 0.7535 ## F-statistic: 390.2 on 3 and 379 DF, p-value: &lt; 2.2e-16 Using summary() we focus on the row for disp:hp which tests, \\[ H_0: \\beta_3 = 0. \\] Again, we see a very low p-value so we reject the null (additive model) in favor of the interaction model. Again, there is an equivalent \\(F\\)-test. anova(mpg_disp_add_hp, mpg_disp_int_hp) ## Analysis of Variance Table ## ## Model 1: mpg ~ disp + hp ## Model 2: mpg ~ disp * hp ## Res.Df RSS Df Sum of Sq F Pr(&gt;F) ## 1 380 7576.6 ## 2 379 5754.2 1 1822.3 120.03 &lt; 2.2e-16 *** ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 We can take a closer look at the coefficients of our fitted interaction model. coef(mpg_disp_int_hp) ## (Intercept) disp hp disp:hp ## 52.4081997848 -0.1001737655 -0.2198199720 0.0005658269 \\(\\hat{\\beta}_0 = 52.4081998\\) is the estimated average mpg for a car with 0 disp and 0 hp. \\(\\hat{\\beta}_1 = -0.1001738\\) is the estimated change in average mpg for an increase in 1 disp, for a car with 0 hp. \\(\\hat{\\beta}_2 = -0.21982\\) is the estimated change in average mpg for an increase in 1 hp, for a car with 0 disp. \\(\\hat{\\beta}_3 = 5.658269\\times 10^{-4}\\) is an estimate of the modification to the change in average mpg for an increase in disp, for a car of a certain hp (or vice versa). That last coefficient needs further explanation. Recall the rearrangement we made earlier \\[ Y = \\beta_0 + (\\beta_1 + \\beta_3 x_2) x_1 + \\beta_2 x_2 + \\epsilon. \\] So, our estimate for \\(\\beta_1 + \\beta_3 x_2\\), is \\(\\hat{\\beta}_1 + \\hat{\\beta}_3 x_2\\), which in this case is \\[ -0.1001738 + 5.658269\\times 10^{-4} x_2. \\] This says that, for an increase of one disp we see an estimated change in average mpg of \\(-0.1001738 + 5.658269\\times 10^{-4} x_2\\). So how disp and mpg are related, depends on the hp of the car. So for a car with 50 hp, the estimated change in average mpg for an increase of one disp is \\[ -0.1001738 + 5.658269\\times 10^{-4} \\cdot 50 = -0.0718824 \\] And for a car with 350 hp, the estimated change in average mpg for an increase of one disp is \\[ -0.1001738 + 5.658269\\times 10^{-4} \\cdot 350 = 0.0978657 \\] Notice the sign changed! 11.3 Factor Variables So far in this chapter, we have limited our use of categorical variables to binary categorical variables. Specifically, we have limited ourselves to dummy variables which take a value of 0 or 1 and represent a categorical variable numerically. We will now discuss factor variables, which is a special way that R deals with categorical variables. With factor variables, a human user can simply think about the categories of a variable, and R will take care of the necessary dummy variables without any 0/1 assignment being done by the user. is.factor(autompg$domestic) ## [1] FALSE Earlier when we used the domestic variable, it was not a factor variable. It was simply a numerical variable that only took two possible values, 1 for domestic, and 0 for foreign. Let’s create a new variable origin that stores the same information, but in a different way. autompg$origin[autompg$domestic == 1] = &quot;domestic&quot; autompg$origin[autompg$domestic == 0] = &quot;foreign&quot; head(autompg$origin) ## [1] &quot;domestic&quot; &quot;domestic&quot; &quot;domestic&quot; &quot;domestic&quot; &quot;domestic&quot; &quot;domestic&quot; Now the origin variable stores \"domestic\" for domestic cars and \"foreign\" for foreign cars. is.factor(autompg$origin) ## [1] FALSE However, this is simply a vector of character values. A vector of car models is a character variable in R. A vector of Vehicle Identification Numbers (VINs) is a character variable as well. But those don’t represent a short list of levels that might influence a response variable. We will want to coerce this origin variable to be something more: a factor variable. autompg$origin = as.factor(autompg$origin) Now when we check the structure of the autompg dataset, we see that origin is a factor variable. str(autompg) ## &#39;data.frame&#39;: 383 obs. of 9 variables: ## $ mpg : num 18 15 18 16 17 15 14 14 14 15 ... ## $ cyl : Factor w/ 3 levels &quot;4&quot;,&quot;6&quot;,&quot;8&quot;: 3 3 3 3 3 3 3 3 3 3 ... ## $ disp : num 307 350 318 304 302 429 454 440 455 390 ... ## $ hp : num 130 165 150 150 140 198 220 215 225 190 ... ## $ wt : num 3504 3693 3436 3433 3449 ... ## $ acc : num 12 11.5 11 12 10.5 10 9 8.5 10 8.5 ... ## $ year : int 70 70 70 70 70 70 70 70 70 70 ... ## $ origin : Factor w/ 2 levels &quot;domestic&quot;,&quot;foreign&quot;: 1 1 1 1 1 1 1 1 1 1 ... ## $ domestic: num 1 1 1 1 1 1 1 1 1 1 ... Factor variables have levels which are the possible values (categories) that the variable may take, in this case foreign or domestic. levels(autompg$origin) ## [1] &quot;domestic&quot; &quot;foreign&quot; Recall that previously we have fit the model \\[ Y = \\beta_0 + \\beta_1 x_1 + \\beta_2 x_2 + \\beta_3 x_1 x_2 + \\epsilon, \\] where \\(Y\\) is mpg, the fuel efficiency in miles per gallon, \\(x_1\\) is disp, the displacement in cubic inches, \\(x_2\\) is domestic, a dummy variable where 1 indicates a domestic car. (mod_dummy = lm(mpg ~ disp * domestic, data = autompg)) ## ## Call: ## lm(formula = mpg ~ disp * domestic, data = autompg) ## ## Coefficients: ## (Intercept) disp domestic disp:domestic ## 46.0548 -0.1569 -12.5755 0.1025 So here we see that \\[ \\hat{\\beta}_0 + \\hat{\\beta}_2 = 46.0548423 + -12.5754714 = 33.4793709 \\] is the estimated average mpg for a domestic car with 0 disp. Now let’s try to do the same, but using our new factor variable. (mod_factor = lm(mpg ~ disp * origin, data = autompg)) ## ## Call: ## lm(formula = mpg ~ disp * origin, data = autompg) ## ## Coefficients: ## (Intercept) disp originforeign disp:originforeign ## 33.47937 -0.05441 12.57547 -0.10252 It seems that it doesn’t produce the same results. Right away we notice that the intercept is different, as is the coefficient in front of disp. We also notice that the remaining two coefficients are of the same magnitude as their respective counterparts using the domestic variable, but with a different sign. Why is this happening? It turns out that by using a factor variable, R is automatically creating a dummy variable for us. However, it is not the dummy variable that we had originally used ourselves. R is fitting the model \\[ Y = \\beta_0 + \\beta_1 x_1 + \\beta_2 x_2 + \\beta_3 x_1 x_2 + \\epsilon, \\] where \\(Y\\) is mpg, the fuel efficiency in miles per gallon, \\(x_1\\) is disp, the displacement in cubic inches, \\(x_2\\) is a dummy variable created by R. It uses 1 to represent a foreign car. So now, \\[ \\hat{\\beta}_0 = 33.4793709 \\] is the estimated average mpg for a domestic car with 0 disp, which is indeed the same as before. When R created \\(x_2\\), the dummy variable, it used domestic cars as the reference level, that is the default value of the factor variable. So when the dummy variable is 0, the model represents this reference level, which is domestic. (R makes this choice because domestic comes before foreign alphabetically.) So the two models have different estimated coefficients, but due to the different model representations, they are actually the same model. 11.3.1 Factors with More Than Two Levels Let’s now consider a factor variable with more than two levels. In this dataset, cyl is an example. is.factor(autompg$cyl) ## [1] TRUE levels(autompg$cyl) ## [1] &quot;4&quot; &quot;6&quot; &quot;8&quot; Here the cyl variable has three possible levels: 4, 6, and 8. You may wonder, why not simply use cyl as a numerical variable? You certainly could. However, that would force the difference in average mpg between 4 and 6 cylinders to be the same as the difference in average mpg between 6 and 8 cylinders. That usually makes sense for a continuous variable, but not for a discrete variable with so few possible values. In the case of this variable, there is no such thing as a 7-cylinder engine or a 6.23-cylinder engine in personal vehicles. For these reasons, we will simply consider cyl to be categorical. This is a decision that will commonly need to be made with ordinal variables. Often, with a large number of categories, the decision to treat them as numerical variables is appropriate because, otherwise, a large number of dummy variables are then needed to represent these variables. Let’s define three dummy variables related to the cyl factor variable. \\[ v_1 = \\begin{cases} 1 &amp; \\text{4 cylinder} \\\\ 0 &amp; \\text{not 4 cylinder} \\end{cases} \\] \\[ v_2 = \\begin{cases} 1 &amp; \\text{6 cylinder} \\\\ 0 &amp; \\text{not 6 cylinder} \\end{cases} \\] \\[ v_3 = \\begin{cases} 1 &amp; \\text{8 cylinder} \\\\ 0 &amp; \\text{not 8 cylinder} \\end{cases} \\] Now, let’s fit an additive model in R, using mpg as the response, and disp and cyl as predictors. This should be a model that uses “three regression lines” to model mpg, one for each of the possible cyl levels. They will all have the same slope (since it is an additive model), but each will have its own intercept. (mpg_disp_add_cyl = lm(mpg ~ disp + cyl, data = autompg)) ## ## Call: ## lm(formula = mpg ~ disp + cyl, data = autompg) ## ## Coefficients: ## (Intercept) disp cyl6 cyl8 ## 34.99929 -0.05217 -3.63325 -2.03603 The question is, what is the model that R has fit here? It has chosen to use the model \\[ Y = \\beta_0 + \\beta_1 x + \\beta_2 v_2 + \\beta_3 v_3 + \\epsilon, \\] where \\(Y\\) is mpg, the fuel efficiency in miles per gallon, \\(x\\) is disp, the displacement in cubic inches, \\(v_2\\) and \\(v_3\\) are the dummy variables defined above. Why doesn’t R use \\(v_1\\)? Essentially because it doesn’t need to. To create three lines, it only needs two dummy variables since it is using a reference level, which in this case is a 4 cylinder car. The three “sub models” are then: 4 Cylinder: \\(Y = \\beta_0 + \\beta_1 x + \\epsilon\\) 6 Cylinder: \\(Y = (\\beta_0 + \\beta_2) + \\beta_1 x + \\epsilon\\) 8 Cylinder: \\(Y = (\\beta_0 + \\beta_3) + \\beta_1 x + \\epsilon\\) Notice that they all have the same slope. However, using the two dummy variables, we achieve the three intercepts. \\(\\beta_0\\) is the average mpg for a 4 cylinder car with 0 disp. \\(\\beta_0 + \\beta_2\\) is the average mpg for a 6 cylinder car with 0 disp. \\(\\beta_0 + \\beta_3\\) is the average mpg for a 8 cylinder car with 0 disp. So because 4 cylinder is the reference level, \\(\\beta_0\\) is specific to 4 cylinders, but \\(\\beta_2\\) and \\(\\beta_3\\) are used to represent quantities relative to 4 cylinders. As we have done before, we can extract these intercepts and slopes for the three lines, and plot them accordingly. int_4cyl = coef(mpg_disp_add_cyl)[1] int_6cyl = coef(mpg_disp_add_cyl)[1] + coef(mpg_disp_add_cyl)[3] int_8cyl = coef(mpg_disp_add_cyl)[1] + coef(mpg_disp_add_cyl)[4] slope_all_cyl = coef(mpg_disp_add_cyl)[2] plot_colors = c(&quot;Darkorange&quot;, &quot;Darkgrey&quot;, &quot;Dodgerblue&quot;) plot(mpg ~ disp, data = autompg, col = plot_colors[cyl], pch = as.numeric(cyl)) abline(int_4cyl, slope_all_cyl, col = plot_colors[1], lty = 1, lwd = 2) abline(int_6cyl, slope_all_cyl, col = plot_colors[2], lty = 2, lwd = 2) abline(int_8cyl, slope_all_cyl, col = plot_colors[3], lty = 3, lwd = 2) legend(&quot;topright&quot;, c(&quot;4 Cylinder&quot;, &quot;6 Cylinder&quot;, &quot;8 Cylinder&quot;), col = plot_colors, lty = c(1, 2, 3), pch = c(1, 2, 3)) On this plot, we have 4 Cylinder: orange dots, solid orange line. 6 Cylinder: grey dots, dashed grey line. 8 Cylinder: blue dots, dotted blue line. The odd result here is that we’re estimating that 8 cylinder cars have better fuel efficiency than 6 cylinder cars at any displacement! The dotted blue line is always above the dashed grey line. That doesn’t seem right. Maybe for very large displacement engines that could be true, but that seems wrong for medium to low displacement. To attempt to fix this, we will try using an interaction model, that is, instead of simply three intercepts and one slope, we will allow for three slopes. Again, we’ll let R take the wheel (no pun intended), then figure out what model it has applied. (mpg_disp_int_cyl = lm(mpg ~ disp * cyl, data = autompg)) ## ## Call: ## lm(formula = mpg ~ disp * cyl, data = autompg) ## ## Coefficients: ## (Intercept) disp cyl6 cyl8 disp:cyl6 disp:cyl8 ## 43.59052 -0.13069 -13.20026 -20.85706 0.08299 0.10817 # could also use mpg ~ disp + cyl + disp:cyl R has again chosen to use 4 cylinder cars as the reference level, but this also now has an effect on the interaction terms. R has fit the model. \\[ Y = \\beta_0 + \\beta_1 x + \\beta_2 v_2 + \\beta_3 v_3 + \\gamma_2 x v_2 + \\gamma_3 x v_3 + \\epsilon \\] We’re using \\(\\gamma\\) like a \\(\\beta\\) parameter for simplicity, so that, for example \\(\\beta_2\\) and \\(\\gamma_2\\) are both associated with \\(v_2\\). Now, the three “sub models” are: 4 Cylinder: \\(Y = \\beta_0 + \\beta_1 x + \\epsilon\\). 6 Cylinder: \\(Y = (\\beta_0 + \\beta_2) + (\\beta_1 + \\gamma_2) x + \\epsilon\\). 8 Cylinder: \\(Y = (\\beta_0 + \\beta_3) + (\\beta_1 + \\gamma_3) x + \\epsilon\\). Interpreting some parameters and coefficients then: \\((\\beta_0 + \\beta_2)\\) is the average mpg of a 6 cylinder car with 0 disp \\((\\hat{\\beta}_1 + \\hat{\\gamma}_3) = -0.1306935 + 0.1081714 = -0.0225221\\) is the estimated change in average mpg for an increase of one disp, for an 8 cylinder car. So, as we have seen before \\(\\beta_2\\) and \\(\\beta_3\\) change the intercepts for 6 and 8 cylinder cars relative to the reference level of \\(\\beta_0\\) for 4 cylinder cars. Now, similarly \\(\\gamma_2\\) and \\(\\gamma_3\\) change the slopes for 6 and 8 cylinder cars relative to the reference level of \\(\\beta_1\\) for 4 cylinder cars. Once again, we extract the coefficients and plot the results. int_4cyl = coef(mpg_disp_int_cyl)[1] int_6cyl = coef(mpg_disp_int_cyl)[1] + coef(mpg_disp_int_cyl)[3] int_8cyl = coef(mpg_disp_int_cyl)[1] + coef(mpg_disp_int_cyl)[4] slope_4cyl = coef(mpg_disp_int_cyl)[2] slope_6cyl = coef(mpg_disp_int_cyl)[2] + coef(mpg_disp_int_cyl)[5] slope_8cyl = coef(mpg_disp_int_cyl)[2] + coef(mpg_disp_int_cyl)[6] plot_colors = c(&quot;Darkorange&quot;, &quot;Darkgrey&quot;, &quot;Dodgerblue&quot;) plot(mpg ~ disp, data = autompg, col = plot_colors[cyl], pch = as.numeric(cyl)) abline(int_4cyl, slope_4cyl, col = plot_colors[1], lty = 1, lwd = 2) abline(int_6cyl, slope_6cyl, col = plot_colors[2], lty = 2, lwd = 2) abline(int_8cyl, slope_8cyl, col = plot_colors[3], lty = 3, lwd = 2) legend(&quot;topright&quot;, c(&quot;4 Cylinder&quot;, &quot;6 Cylinder&quot;, &quot;8 Cylinder&quot;), col = plot_colors, lty = c(1, 2, 3), pch = c(1, 2, 3)) This looks much better! We can see that for medium displacement cars, 6 cylinder cars now perform better than 8 cylinder cars, which seems much more reasonable than before. To completely justify the interaction model (i.e., a unique slope for each cyl level) compared to the additive model (single slope), we can perform an \\(F\\)-test. Notice first, that there is no \\(t\\)-test that will be able to do this since the difference between the two models is not a single parameter. We will test, \\[ H_0: \\gamma_2 = \\gamma_3 = 0 \\] which represents the parallel regression lines we saw before, \\[ Y = \\beta_0 + \\beta_1 x + \\beta_2 v_2 + \\beta_3 v_3 + \\epsilon. \\] Again, this is a difference of two parameters, thus no \\(t\\)-test will be useful. anova(mpg_disp_add_cyl, mpg_disp_int_cyl) ## Analysis of Variance Table ## ## Model 1: mpg ~ disp + cyl ## Model 2: mpg ~ disp * cyl ## Res.Df RSS Df Sum of Sq F Pr(&gt;F) ## 1 379 7299.5 ## 2 377 6551.7 2 747.79 21.515 1.419e-09 *** ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 As expected, we see a very low p-value, and thus reject the null. We prefer the interaction model over the additive model. Recapping a bit: Null Model: \\(Y = \\beta_0 + \\beta_1 x + \\beta_2 v_2 + \\beta_3 v_3 + \\epsilon\\) Number of parameters: \\(q = 4\\) Full Model: \\(Y = \\beta_0 + \\beta_1 x + \\beta_2 v_2 + \\beta_3 v_3 + \\gamma_2 x v_2 + \\gamma_3 x v_3 + \\epsilon\\) Number of parameters: \\(p = 6\\) length(coef(mpg_disp_int_cyl)) - length(coef(mpg_disp_add_cyl)) ## [1] 2 We see there is a difference of two parameters, which is also displayed in the resulting ANOVA table from R. Notice that the following two values also appear on the ANOVA table. nrow(autompg) - length(coef(mpg_disp_int_cyl)) ## [1] 377 nrow(autompg) - length(coef(mpg_disp_add_cyl)) ## [1] 379 11.4 Parameterization So far we have been simply letting R decide how to create the dummy variables, and thus R has been deciding the parameterization of the models. To illustrate the ability to use alternative parameterizations, we will recreate the data, but directly creating the dummy variables ourselves. new_param_data = data.frame( y = autompg$mpg, x = autompg$disp, v1 = 1 * as.numeric(autompg$cyl == 4), v2 = 1 * as.numeric(autompg$cyl == 6), v3 = 1 * as.numeric(autompg$cyl == 8)) head(new_param_data, 20) ## y x v1 v2 v3 ## 1 18 307 0 0 1 ## 2 15 350 0 0 1 ## 3 18 318 0 0 1 ## 4 16 304 0 0 1 ## 5 17 302 0 0 1 ## 6 15 429 0 0 1 ## 7 14 454 0 0 1 ## 8 14 440 0 0 1 ## 9 14 455 0 0 1 ## 10 15 390 0 0 1 ## 11 15 383 0 0 1 ## 12 14 340 0 0 1 ## 13 15 400 0 0 1 ## 14 14 455 0 0 1 ## 15 24 113 1 0 0 ## 16 22 198 0 1 0 ## 17 18 199 0 1 0 ## 18 21 200 0 1 0 ## 19 27 97 1 0 0 ## 20 26 97 1 0 0 Now, y is mpg x is disp, the displacement in cubic inches, v1, v2, and v3 are dummy variables as defined above. First let’s try to fit an additive model using x as well as the three dummy variables. lm(y ~ x + v1 + v2 + v3, data = new_param_data) ## ## Call: ## lm(formula = y ~ x + v1 + v2 + v3, data = new_param_data) ## ## Coefficients: ## (Intercept) x v1 v2 v3 ## 32.96326 -0.05217 2.03603 -1.59722 NA What is happening here? Notice that R is essentially ignoring v3, but why? Well, because R uses an intercept, it cannot also use v3. This is because \\[ \\boldsymbol{1} = v_1 + v_2 + v_3 \\] which means that \\(\\boldsymbol{1}\\), \\(v_1\\), \\(v_2\\), and \\(v_3\\) are linearly dependent. This would make the \\(X^\\top X\\) matrix singular, but we need to be able to invert it to solve the normal equations and obtain \\(\\hat{\\beta}.\\) With the intercept, v1, and v2, R can make the necessary “three intercepts.” So, in this case v3 is the reference level. If we remove the intercept, then we can directly obtain all “three intercepts” without a reference level. lm(y ~ 0 + x + v1 + v2 + v3, data = new_param_data) ## ## Call: ## lm(formula = y ~ 0 + x + v1 + v2 + v3, data = new_param_data) ## ## Coefficients: ## x v1 v2 v3 ## -0.05217 34.99929 31.36604 32.96326 Here, we are fitting the model \\[ Y = \\mu_1 v_1 + \\mu_2 v_2 + \\mu_3 v_3 + \\beta x +\\epsilon. \\] Thus we have: 4 Cylinder: \\(Y = \\mu_1 + \\beta x + \\epsilon\\) 6 Cylinder: \\(Y = \\mu_2 + \\beta x + \\epsilon\\) 8 Cylinder: \\(Y = \\mu_3 + \\beta x + \\epsilon\\) We could also do something similar with the interaction model, and give each line an intercept and slope, without the need for a reference level. lm(y ~ 0 + v1 + v2 + v3 + x:v1 + x:v2 + x:v3, data = new_param_data) ## ## Call: ## lm(formula = y ~ 0 + v1 + v2 + v3 + x:v1 + x:v2 + x:v3, data = new_param_data) ## ## Coefficients: ## v1 v2 v3 v1:x v2:x v3:x ## 43.59052 30.39026 22.73346 -0.13069 -0.04770 -0.02252 \\[ Y = \\mu_1 v_1 + \\mu_2 v_2 + \\mu_3 v_3 + \\beta_1 x v_1 + \\beta_2 x v_2 + \\beta_3 x v_3 +\\epsilon \\] 4 Cylinder: \\(Y = \\mu_1 + \\beta_1 x + \\epsilon\\) 6 Cylinder: \\(Y = \\mu_2 + \\beta_2 x + \\epsilon\\) 8 Cylinder: \\(Y = \\mu_3 + \\beta_3 x + \\epsilon\\) Using the original data, we have (at least) three equivalent ways to specify the interaction model with R. lm(mpg ~ disp * cyl, data = autompg) ## ## Call: ## lm(formula = mpg ~ disp * cyl, data = autompg) ## ## Coefficients: ## (Intercept) disp cyl6 cyl8 disp:cyl6 disp:cyl8 ## 43.59052 -0.13069 -13.20026 -20.85706 0.08299 0.10817 lm(mpg ~ 0 + cyl + disp : cyl, data = autompg) ## ## Call: ## lm(formula = mpg ~ 0 + cyl + disp:cyl, data = autompg) ## ## Coefficients: ## cyl4 cyl6 cyl8 cyl4:disp cyl6:disp cyl8:disp ## 43.59052 30.39026 22.73346 -0.13069 -0.04770 -0.02252 lm(mpg ~ 0 + disp + cyl + disp : cyl, data = autompg) ## ## Call: ## lm(formula = mpg ~ 0 + disp + cyl + disp:cyl, data = autompg) ## ## Coefficients: ## disp cyl4 cyl6 cyl8 disp:cyl6 disp:cyl8 ## -0.13069 43.59052 30.39026 22.73346 0.08299 0.10817 They all fit the same model, importantly each using six parameters, but the coefficients mean slightly different things in each. However, once they are interpreted as slopes and intercepts for the “three lines” they will have the same result. Use ?all.equal to learn about the all.equal() function, and think about how the following code verifies that the residuals of the two models are the same. all.equal(fitted(lm(mpg ~ disp * cyl, data = autompg)), fitted(lm(mpg ~ 0 + cyl + disp : cyl, data = autompg))) ## [1] TRUE 11.5 Building Larger Models Now that we have seen how to incorporate categorical predictors as well as interaction terms, we can start to build much larger, much more flexible models which can potentially fit data better. Let’s define a “big” model, \\[ Y = \\beta_0 + \\beta_1 x_1 + \\beta_2 x_2 + \\beta_3 x_3 + \\beta_4 x_1 x_2 + \\beta_5 x_1 x_3 + \\beta_6 x_2 x_3 + \\beta_7 x_1 x_2 x_3 + \\epsilon. \\] Here, \\(Y\\) is mpg. \\(x_1\\) is disp. \\(x_2\\) is hp. \\(x_3\\) is domestic, which is a dummy variable we defined, where 1 is a domestic vehicle. First thing to note here, we have included a new term \\(x_1 x_2 x_3\\) which is a three-way interaction. Interaction terms can be larger and larger, up to the number of predictors in the model. Since we are using the three-way interaction term, we also use all possible two-way interactions, as well as each of the first order (main effect) terms. This is the concept of a hierarchy. Any time a “higher-order” term is in a model, the related “lower-order” terms should also be included. Mathematically their inclusion or exclusion is sometimes irrelevant, but from an interpretation standpoint, it is best to follow the hierarchy rules. Let’s do some rearrangement to obtain a “coefficient” in front of \\(x_1\\). \\[ Y = \\beta_0 + \\beta_2 x_2 + \\beta_3 x_3 + \\beta_6 x_2 x_3 + (\\beta_1 + \\beta_4 x_2 + \\beta_5 x_3 + \\beta_7 x_2 x_3)x_1 + \\epsilon. \\] Specifically, the “coefficient” in front of \\(x_1\\) is \\[ (\\beta_1 + \\beta_4 x_2 + \\beta_5 x_3 + \\beta_7 x_2 x_3). \\] Let’s discuss this “coefficient” to help us understand the idea of the flexibility of a model. Recall that, \\(\\beta_1\\) is the coefficient for a first order term, \\(\\beta_4\\) and \\(\\beta_5\\) are coefficients for two-way interactions, \\(\\beta_7\\) is the coefficient for the three-way interaction. If the two and three way interactions were not in the model, the whole “coefficient” would simply be \\[ \\beta_1. \\] Thus, no matter the values of \\(x_2\\) and \\(x_3\\), \\(\\beta_1\\) would determine the relationship between \\(x_1\\) (disp) and \\(Y\\) (mpg). With the addition of the two-way interactions, now the “coefficient” would be \\[ (\\beta_1 + \\beta_4 x_2 + \\beta_5 x_3). \\] Now, changing \\(x_1\\) (disp) has a different effect on \\(Y\\) (mpg), depending on the values of \\(x_2\\) and \\(x_3\\). Lastly, adding the three-way interaction gives the whole “coefficient” \\[ (\\beta_1 + \\beta_4 x_2 + \\beta_5 x_3 + \\beta_7 x_2 x_3) \\] which is even more flexible. Now changing \\(x_1\\) (disp) has a different effect on \\(Y\\) (mpg), depending on the values of \\(x_2\\) and \\(x_3\\), but in a more flexible way which we can see with some more rearrangement. Now the “coefficient” in front of \\(x_3\\) in this “coefficient” is dependent on \\(x_2\\). \\[ (\\beta_1 + \\beta_4 x_2 + (\\beta_5 + \\beta_7 x_2) x_3) \\] It is so flexible, it is becoming hard to interpret! Let’s fit this three-way interaction model in R. big_model = lm(mpg ~ disp * hp * domestic, data = autompg) summary(big_model) ## ## Call: ## lm(formula = mpg ~ disp * hp * domestic, data = autompg) ## ## Residuals: ## Min 1Q Median 3Q Max ## -11.9410 -2.2147 -0.4008 1.9430 18.4094 ## ## Coefficients: ## Estimate Std. Error t value Pr(&gt;|t|) ## (Intercept) 6.065e+01 6.600e+00 9.189 &lt; 2e-16 *** ## disp -1.416e-01 6.344e-02 -2.232 0.0262 * ## hp -3.545e-01 8.123e-02 -4.364 1.65e-05 *** ## domestic -1.257e+01 7.064e+00 -1.780 0.0759 . ## disp:hp 1.369e-03 6.727e-04 2.035 0.0426 * ## disp:domestic 4.933e-02 6.400e-02 0.771 0.4414 ## hp:domestic 1.852e-01 8.709e-02 2.126 0.0342 * ## disp:hp:domestic -9.163e-04 6.768e-04 -1.354 0.1766 ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 ## ## Residual standard error: 3.88 on 375 degrees of freedom ## Multiple R-squared: 0.76, Adjusted R-squared: 0.7556 ## F-statistic: 169.7 on 7 and 375 DF, p-value: &lt; 2.2e-16 Do we actually need this large of a model? Let’s first test for the necessity of the three-way interaction term. That is, \\[ H_0: \\beta_7 = 0. \\] So, Full Model: \\(Y = \\beta_0 + \\beta_1 x_1 + \\beta_2 x_2 + \\beta_3 x_3 + \\beta_4 x_1 x_2 + \\beta_5 x_1 x_3 + \\beta_6 x_2 x_3 + \\beta_7 x_1 x_2 x_3 + \\epsilon\\) Null Model: \\(Y = \\beta_0 + \\beta_1 x_1 + \\beta_2 x_2 + \\beta_3 x_3 + \\beta_4 x_1 x_2 + \\beta_5 x_1 x_3 + \\beta_6 x_2 x_3 + \\epsilon\\) We fit the null model in R as two_way_int_mod, then use anova() to perform an \\(F\\)-test as usual. two_way_int_mod = lm(mpg ~ disp * hp + disp * domestic + hp * domestic, data = autompg) #two_way_int_mod = lm(mpg ~ (disp + hp + domestic) ^ 2, data = autompg) anova(two_way_int_mod, big_model) ## Analysis of Variance Table ## ## Model 1: mpg ~ disp * hp + disp * domestic + hp * domestic ## Model 2: mpg ~ disp * hp * domestic ## Res.Df RSS Df Sum of Sq F Pr(&gt;F) ## 1 376 5673.2 ## 2 375 5645.6 1 27.599 1.8332 0.1766 We see the p-value is somewhat large, so we would fail to reject. We prefer the smaller, less flexible, null model, without the three-way interaction. A quick note here: the full model does still “fit better.” Notice that it has a smaller RMSE than the null model, which means the full model makes smaller (squared) errors on average. mean(resid(big_model) ^ 2) ## [1] 14.74053 mean(resid(two_way_int_mod) ^ 2) ## [1] 14.81259 However, it is not much smaller. We could even say that the difference is insignificant. This is an idea we will return to later in greater detail. Now that we have chosen the model without the three-way interaction, can we go further? Do we need the two-way interactions? Let’s test \\[ H_0: \\beta_4 = \\beta_5 = \\beta_6 = 0. \\] Remember we already chose \\(\\beta_7 = 0\\), so, Full Model: \\(Y = \\beta_0 + \\beta_1 x_1 + \\beta_2 x_2 + \\beta_3 x_3 + \\beta_4 x_1 x_2 + \\beta_5 x_1 x_3 + \\beta_6 x_2 x_3 + \\epsilon\\) Null Model: \\(Y = \\beta_0 + \\beta_1 x_1 + \\beta_2 x_2 + \\beta_3 x_3 + \\epsilon\\) We fit the null model in R as additive_mod, then use anova() to perform an \\(F\\)-test as usual. additive_mod = lm(mpg ~ disp + hp + domestic, data = autompg) anova(additive_mod, two_way_int_mod) ## Analysis of Variance Table ## ## Model 1: mpg ~ disp + hp + domestic ## Model 2: mpg ~ disp * hp + disp * domestic + hp * domestic ## Res.Df RSS Df Sum of Sq F Pr(&gt;F) ## 1 379 7369.7 ## 2 376 5673.2 3 1696.5 37.478 &lt; 2.2e-16 *** ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 Here the p-value is small, so we reject the null, and we prefer the full (alternative) model. Of the models we have considered, our final preference is for \\[ Y = \\beta_0 + \\beta_1 x_1 + \\beta_2 x_2 + \\beta_3 x_3 + \\beta_4 x_1 x_2 + \\beta_5 x_1 x_3 + \\beta_6 x_2 x_3 + \\epsilon. \\] 11.6 R Markdown The R Markdown file for this chapter can be found here: cat-int.Rmd The file was created using R version 4.4.1. "],["analysis-of-variance.html", "Chapter 12 Analysis of Variance 12.1 Experiments 12.2 Two-Sample t-Test 12.3 One-Way ANOVA 12.4 Post Hoc Testing 12.5 Two-Way ANOVA 12.6 R Markdown", " Chapter 12 Analysis of Variance Chapter Status: This chapter should be considered optional for a first reading of this text. Its inclusion is mostly for the benefit of some courses that use the text. Additionally, this chapter is currently somewhat underdeveloped compared to the rest of the text. If you are interested in contributing, you can find several lines marked “TODO” in the source. Pull requests encouraged! .table { width: 70%; margin-left:10%; margin-right:10%; } “To find out what happens when you change something, it is necessary to change it.” — Box, Hunter, and Hunter, Statistics for Experimenters (1978) Thus far, we have built models for numeric responses, when the predictors are all numeric. We’ll take a minor detour to go back and consider models which only have categorical predictors. A categorical predictor is a variable which takes only a finite number of values, which are not ordered. For example a variable which takes possible values red, blue, green is categorical. In the context of using a categorical variable as a predictor, it would place observations into different groups (categories). We’ve also mostly been dealing with observational data. The methods in this section are most useful in experimental settings, but still work with observational data. (However, for determining causation, we require experiments.) 12.1 Experiments The biggest difference between an observational study and an experiment is how the predictor data is obtained. Is the experimenter in control? In an observational study, both response and predictor data are obtained via observation. In an experiment, the predictor data are values determined by the experimenter. The experiment is run and the response is observed. In an experiment, the predictors, which are controlled by the experimenter, are called factors. The possible values of these factors are called levels. Subjects are randomly assigned to a level of each of the factors. The design of experiments could be a course by itself. The Wikipedia article on design of experiments gives a good overview. Originally, most of the methodology was developed for agricultural applications by R. A. Fisher, but is still in use today, now in a wide variety of application areas. Notably, these methods have seen a resurgence as a part of “A/B Testing.” 12.2 Two-Sample t-Test The simplest example of an experimental design is the setup for a two-sample \\(t\\)-test. There is a single factor variable with two levels which split the subjects into two groups. Often, one level is considered the control, while the other is the treatment. The subjects are randomly assigned to one of the two groups. After being assigned to a group, each subject has some quantity measured, which is the response variable. Mathematically, we consider the model \\[ y_{ij} \\sim N(\\mu_i, \\sigma^2) \\] where \\(i = 1, 2\\) for the two groups and \\(j = 1, 2, \\ldots n_i\\). Here \\(n_i\\) is the number of subjects in group \\(i\\). So \\(y_{13}\\) would be the measurement for the third member of the first group. So measurements of subjects in group \\(1\\) follow a normal distribution with mean \\(\\mu_1\\). \\[ y_{1j} \\sim N(\\mu_1, \\sigma^2) \\] Then measurements of subjects in group \\(2\\) follow a normal distribution with mean \\(\\mu_2\\). \\[ y_{2j} \\sim N(\\mu_2, \\sigma^2) \\] This model makes a number of assumptions. Specifically, The observations follow a normal distribution. The mean of each group is different. Equal variance for each group. Independence. Which is believable if groups were randomly assigned. Later, we will investigate the normal and equal variance assumptions. For now, we will continue to assume they are reasonable. The natural question to ask: Is there a difference between the two groups? The specific question we’ll answer: Are the means of the two groups different? Mathematically, that is \\[ H_0: \\mu_1 = \\mu_2 \\quad \\text{vs} \\quad H_1: \\mu_1 \\neq \\mu_2 \\] For the stated model and assuming the null hypothesis is true, the \\(t\\) test statistic would follow a \\(t\\) distribution with degrees of freedom \\(n_1 + n_2 - 2\\). As an example, suppose we are interested in the effect of melatonin on sleep duration. A researcher obtains a random sample of 20 adult males. Of these subjects, 10 are randomly chosen for the control group, which will receive a placebo. The remaining 10 will be given 5mg of melatonin before bed. The sleep duration in hours of each subject is then measured. The researcher chooses a significance level of \\(\\alpha = 0.10\\). Was sleep duration affected by the melatonin? melatonin ## sleep group ## 1 8.145150 control ## 2 7.522362 treatment ## 3 6.935754 control ## 4 8.959435 treatment ## 5 6.985122 control ## 6 8.072651 treatment ## 7 8.313826 control ## 8 8.086409 treatment ## 9 8.922108 control ## 10 8.124743 treatment ## 11 8.065844 control ## 12 10.943974 treatment ## 13 4.833367 control ## 14 7.865453 treatment ## 15 6.340014 control ## 16 8.963140 treatment ## 17 6.158896 control ## 18 5.012253 treatment ## 19 3.571440 control ## 20 9.784136 treatment Here, we would like to test, \\[ H_0: \\mu_C = \\mu_T \\quad \\text{vs} \\quad H_1: \\mu_C \\neq \\mu_T \\] To do so in R, we use the t.test() function, with the var.equal argument set to TRUE. t.test(sleep ~ group, data = melatonin, var.equal = TRUE) ## ## Two Sample t-test ## ## data: sleep by group ## t = -2.0854, df = 18, p-value = 0.05154 ## alternative hypothesis: true difference in means between group control and group treatment is not equal to 0 ## 95 percent confidence interval: ## -3.02378261 0.01117547 ## sample estimates: ## mean in group control mean in group treatment ## 6.827152 8.333456 At a significance level of \\(\\alpha = 0.10\\), we reject the null hypothesis. It seems that the melatonin had a statistically significant effect. Be aware that statistical significance is not always the same as scientific or practical significance. To determine practical significance, we need to investigate the effect size in the context of the situation. Here the effect size is the difference of the sample means. t.test(sleep ~ group, data = melatonin, var.equal = TRUE)$estimate ## mean in group control mean in group treatment ## 6.827152 8.333456 Here we see that the subjects in the melatonin group sleep an average of about 1.5 hours longer than the control group. An hour and a half of sleep is certainly important! With a big enough sample size, we could make an effect size of say, four minutes statistically significant. Is it worth taking a pill every night to get an extra four minutes of sleep? (Probably not.) boxplot(sleep ~ group, data = melatonin, col = 5:6) 12.3 One-Way ANOVA What if there are more than two groups? Consider the model \\[ y_{ij} = \\mu + \\alpha_i + e_{ij} \\] where \\[ \\sum_{i=1}^{g} \\alpha_i = 0 \\] and \\[ e_{ij} \\sim N(0,\\sigma^{2}). \\] Here, \\(i = 1, 2, \\ldots g\\) where \\(g\\) is the number of groups. \\(j = 1, 2, \\ldots n_i\\) where \\(n_i\\) is the number of observations in group \\(i\\). Then the total sample size is \\[ N = \\sum_{i = 1}^{g} n_i \\] Observations from group \\(i\\) follow a normal distribution \\[ y_{ij} \\sim N(\\mu_i,\\sigma^{2}) \\] where the mean of each group is given by \\[ \\mu_i = \\mu + \\alpha_i. \\] Here \\(\\alpha_i\\) measures the effect of group \\(i\\). It is the difference between the overall mean and the mean of group \\(i\\). Essentially, the assumptions here are the same as the two-sample case, however now, we simply have more groups. Much like the two-sample case, we would again like to test if the means of the groups are equal. \\[ H_0: \\mu_1 = \\mu_2 = \\ldots \\mu_g \\quad \\text{vs} \\quad H_1: \\text{ Not all } \\mu_i \\text{ are equal.} \\] Notice that the alternative simply indicates that some of the means are not equal, not specifically which are not equal. More on that later. Alternatively, we could write \\[ H_0: \\alpha_1 = \\alpha_2 = \\ldots = \\alpha_g = 0 \\quad \\text{vs} \\quad H_1: \\text{ Not all } \\alpha_i \\text{ are } 0. \\] This test is called Analysis of Variance (ANOVA). ANOVA compares the variation due to specific sources (between groups) with the variation among individuals who should be similar (within groups). In particular, ANOVA tests whether several populations have the same mean by comparing how far apart the sample means are with how much variation there is within the samples. We use variability of means to test for equality of means, thus the use of variance in the name for a test about means. We’ll leave out most of the details about how the estimation is done, but we’ll see later, that it is done via least squares. We’ll use R to obtain these estimates, but they are actually rather simple. We only need to think about the sample means of the groups. \\(\\bar{y}_i\\) is the sample mean of group \\(i\\). \\(\\bar{y}\\) is the overall sample mean. \\(s_{i}^{2}\\) is the sample variance of group \\(i\\). We’ll then decompose the variance, as we’ve seen before in regression. The total variation measures how much the observations vary about the overall sample mean, ignoring the groups. \\[ SST = \\sum_{i = 1}^{g} \\sum_{j = 1}^{n_i} (y_{ij} - \\bar{y})^2 \\] The variation between groups looks at how far the individual sample means are from the overall sample mean. \\[ SSB = \\sum_{i = 1}^{g} \\sum_{j = 1}^{n_i} (\\bar{y}_i - \\bar{y})^2 = \\sum_{i = 1}^{g} n_i (\\bar{y}_i - \\bar{y})^2 \\] Lastly, the within group variation measures how far observations are from the sample mean of its group. \\[ SSW = \\sum_{i = 1}^{g} \\sum_{j = 1}^{n_i} (y_{ij} - \\bar{y}_i)^2 = \\sum_{i = 1}^{g} (n_i - 1) s_{i}^{2} \\] This could also be thought of as the error sum of squares, where \\(y_{ij}\\) is an observation and \\(\\bar{y}_i\\) is its fitted (predicted) value from the model. To develop the test statistic for ANOVA, we place this information into an ANVOA table. Source Sum of Squares Degrees of Freedom Mean Square \\(F\\) Between SSB \\(g - 1\\) SSB / DFB MSB / MSW Within SSW \\(N - g\\) SSW / DFW Total SST \\(N - 1\\) We reject the null (equal means) when the \\(F\\) statistic is large. This occurs when the variation between groups is large compared to the variation within groups. Under the null hypothesis, the distribution of the test statistic is \\(F\\) with degrees of freedom \\(g - 1\\) and \\(N - g\\). Let’s see what this looks like in a few situations. In each of the following examples, we’ll consider sampling 20 observations (\\(n_i = 20\\)) from three populations (groups; \\(g = 3\\)). First, consider \\(\\mu_A = -5, \\mu_B = 0, \\mu_C = 5\\) with \\(\\sigma = 1\\). The left panel shows the three normal distributions we are sampling from. The ticks along the \\(x\\)-axis show the randomly sampled observations. The right panel re-displays only the sampled values in a boxplot. Note that the mid-line of the boxes is usually the sample median. These boxplots have been modified to use the sample mean. Here the sample means vary a lot around the overall sample mean, which is the solid grey line on the right panel. Within the groups there is variability, but it is still obvious that the sample means are very different. As a result, we obtain a large test statistic, thus small p-value. \\(F = 374.4469511\\) \\(\\text{p-value} = 1.6349862\\times 10^{-33}\\) Now consider \\(\\mu_A = 0, \\mu_B = 0, \\mu_C = 0\\) with \\(\\sigma = 1\\). That is, equal means for the groups. Here the sample means vary only a tiny bit around the overall sample mean. Within the groups there is variability, this time much larger than the variability of the sample means. As a result, we obtain a small test statistic, thus large p-value. \\(F = 2.667892\\) \\(\\text{p-value} = 0.0780579\\) The next two examples show different means, with different levels of noise. Notice how these affect the test statistic and p-value. \\(\\mu_A = -1, \\mu_B = 0, \\mu_C = 1, \\sigma = 1\\) \\(F = 16.4879492\\) \\(\\text{p-value} = 2.2378806\\times 10^{-6}\\) Above, there isn’t obvious separation between the groups like the first example, but it is still obvious the means are different. Below, there is more noise. Visually it is somewhat hard to tell, but the test still suggests a difference of means. (At an \\(\\alpha\\) of 0.05.) \\(\\mu_A = -1, \\mu_B = 0, \\mu_C = 1, \\sigma = 2\\) \\(F = 4.6256472\\) \\(\\text{p-value} = 0.0137529\\) Let’s consider an example with real data. We’ll use the coagulation dataset from the faraway package. Here four different diets (A, B, C, D) were administered to a random sample of 24 animals. The subjects were randomly assigned to one of the four diets. For each, their blood coagulation time was measured in seconds. Here we would like to test \\[ H_0: \\mu_A = \\mu_B = \\mu_C = \\mu_D \\] where, for example, \\(\\mu_A\\) is the mean blood coagulation time for an animal that ate diet A. library(faraway) names(coagulation) ## [1] &quot;coag&quot; &quot;diet&quot; plot(coag ~ diet, data = coagulation, col = 2:5) We first load the data and create the relevant boxplot. The plot alone suggests a difference of means. The aov() function is used to obtain the relevant sums of squares. Using the summary() function on the output from aov() creates the desired ANOVA table (without the unneeded row for total). coag_aov = aov(coag ~ diet, data = coagulation) coag_aov ## Call: ## aov(formula = coag ~ diet, data = coagulation) ## ## Terms: ## diet Residuals ## Sum of Squares 228 112 ## Deg. of Freedom 3 20 ## ## Residual standard error: 2.366432 ## Estimated effects may be unbalanced summary(coag_aov) ## Df Sum Sq Mean Sq F value Pr(&gt;F) ## diet 3 228 76.0 13.57 4.66e-05 *** ## Residuals 20 112 5.6 ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 Were we to run this experiment, we would have pre-specified a significance level. However, notice that the p-value of this test is incredibly low, so using any reasonable significance level we would reject the null hypothesis. Thus we believe the diets had an effect on blood coagulation time. diets = data.frame(diet = unique(coagulation$diet)) data.frame(diets, coag = predict(coag_aov, diets)) ## diet coag ## 1 A 61 ## 2 B 66 ## 3 C 68 ## 4 D 61 Here, we’ve created a dataframe with a row for each diet. By predicting on this dataframe, we obtain the sample means of each diet (group). 12.3.1 Factor Variables When performing ANOVA in R, be sure the grouping variable is a factor variable. If it is not, your result might not be ANOVA, but instead a linear regression with the predictor variable considered numeric. set.seed(42) response = rnorm(15) group = c(rep(1, 5), rep(2, 5), rep(3, 5)) bad = data.frame(response, group) summary(aov(response ~ group, data = bad)) # wrong DF! ## Df Sum Sq Mean Sq F value Pr(&gt;F) ## group 1 0.017 0.0173 0.015 0.903 ## Residuals 13 14.698 1.1306 good = data.frame(response, group = as.factor(group)) summary(aov(response ~ group, data = good)) ## Df Sum Sq Mean Sq F value Pr(&gt;F) ## group 2 0.232 0.1158 0.096 0.909 ## Residuals 12 14.484 1.2070 is.factor(bad$group) # 1, 2, and 3 are numbers. ## [1] FALSE is.factor(good$group) # 1, 2, and 3 are labels. ## [1] TRUE 12.3.2 Some Simulation Here we verify the distribution of the test statistic under the null hypothesis. We simulate from a null model (equal variance) to obtain an empirical distribution of the \\(F\\) statistic. We add the curve for the expected distribution. library(broom) sim_anova = function(n = 10, mu_a = 0, mu_b = 0, mu_c = 0, mu_d = 0, sigma = 1, stat = TRUE) { # create data from one-way ANOVA model with four groups of equal size # response simulated from normal with group mean, shared variance # group variable indicates group A, B, C or D sim_data = data.frame( response = c(rnorm(n = n, mean = mu_a, sd = sigma), rnorm(n = n, mean = mu_b, sd = sigma), rnorm(n = n, mean = mu_c, sd = sigma), rnorm(n = n, mean = mu_d, sd = sigma)), group = c(rep(&quot;A&quot;, times = n), rep(&quot;B&quot;, times = n), rep(&quot;C&quot;, times = n), rep(&quot;D&quot;, times = n)) ) # obtain F-statistic and p-value for testing difference of means # use lm instead of aov for better result formatting with glance aov_results = lm(response ~ group, data = sim_data) f_stat = glance(aov_results)$statistic p_val = glance(aov_results)$p.value # return f_stat if stat = TRUE, otherwise, p-value ifelse(stat, f_stat, p_val) } f_stats = replicate(n = 5000, sim_anova(stat = TRUE)) hist(f_stats, breaks = 100, prob = TRUE, border = &quot;dodgerblue&quot;, main = &quot;Empirical Distribution of F&quot;) curve(df(x, df1 = 4 - 1, df2 = 40 - 4), col = &quot;darkorange&quot;, add = TRUE, lwd = 2) 12.3.3 Power Now that we’re performing experiments, getting more data means finding more test subjects, running more lab tests, etc. In other words, it will cost more time and money. We’d like to design our experiment so that we have a good chance of detecting an interesting effect size, without spending too much money. There’s no point in running an experiment if there’s only a very low chance that it has a significant result that you care about. (Remember, not all statistically significant results have practical value.) We’d like the ANOVA test to have high power for an alternative hypothesis with a minimum desired effect size. \\[ \\text{Power } = P(\\text{Reject } H_0 \\mid H_0 \\text{ False}) \\] That is, for a true difference of means that we deem interesting, we want the test to reject with high probability. A number of things can affect the power of a test: Effect size. It is easier to detect larger effects. Noise level \\(\\sigma\\). The less noise, the easier it is to detect signal (effect). We don’t have much ability to control this, except maybe to measure more accurately. Significance level \\(\\alpha\\). Lower significance level makes rejecting more difficult. (But also allows for fewer false positives.) Sample size. Large samples mean easier to detect effects. Balanced design. An equal number of observations per group leads to higher power. The following simulations look at the effect of significance level, effect size, and noise level on the power of an ANOVA \\(F\\)-test. Homework will look into sample size and balance. p_vals = replicate(n = 1000, sim_anova(mu_a = -1, mu_b = 0, mu_c = 0, mu_d = 1, sigma = 1.5, stat = FALSE)) mean(p_vals &lt; 0.05) ## [1] 0.661 mean(p_vals &lt; 0.01) ## [1] 0.384 p_vals = replicate(n = 1000, sim_anova(mu_a = -1, mu_b = 0, mu_c = 0, mu_d = 1, sigma = 2.0, stat = FALSE)) mean(p_vals &lt; 0.05) ## [1] 0.425 mean(p_vals &lt; 0.01) ## [1] 0.19 p_vals = replicate(n = 1000, sim_anova(mu_a = -2, mu_b = 0, mu_c = 0, mu_d = 2, sigma = 2.0, stat = FALSE)) mean(p_vals &lt; 0.05) ## [1] 0.956 mean(p_vals &lt; 0.01) ## [1] 0.837 12.4 Post Hoc Testing Suppose we reject the null hypothesis from the ANOVA test for equal means. That tells us that the means are different. But which means? All of them? Some of them? The obvious strategy is to test all possible comparisons of two means. We can do this easily in R. with(coagulation, pairwise.t.test(coag, diet, p.adj = &quot;none&quot;)) ## ## Pairwise comparisons using t tests with pooled SD ## ## data: coag and diet ## ## A B C ## B 0.00380 - - ## C 0.00018 0.15878 - ## D 1.00000 0.00086 2.3e-05 ## ## P value adjustment method: none # pairwise.t.test(coagulation$coag, coagulation$diet, p.adj = &quot;none&quot;) Notice the pairwise.t.test() function does not have a data argument. To avoid using attach() or the $ operator, we introduce the with() function. The commented line would perform the same operation. Also note that we are using the argument p.adj = \"none\". What is this? An adjustment (in this case not an adjustment) to the p-value of each test. Why would we need to do this? The adjustment is an attempt to correct for the multiple testing problem. (See also: Relevant XKCD. ) Imagine that you knew ahead of time that you were going to perform 100 \\(t\\)-tests. Suppose you wish to do this with a false positive rate of \\(\\alpha = 0.05\\). If we use this significance level for each test, for 100 tests, we then expect 5 false positives. That means, with 100 tests, we’re almost guaranteed to have at least one error. What we’d really like is for the family-wise error rate (FWER) to be 0.05. If we consider the 100 tests to be a single “experiment,” the FWER is the rate of one or more false positives for the full experiment (100 tests). Consider it an error rate for an entire procedure, instead of a single test. With this in mind, one of the simplest adjustments we can make, is to increase the p-values for each test, depending on the number of tests. In particular the Bonferroni correction simply multiplies by the number of tests. \\[ \\text{p-value-bonf} = \\min(1, n_{tests} \\cdot \\text{p-value}) \\] with(coagulation, pairwise.t.test(coag, diet, p.adj = &quot;bonferroni&quot;)) ## ## Pairwise comparisons using t tests with pooled SD ## ## data: coag and diet ## ## A B C ## B 0.02282 - - ## C 0.00108 0.95266 - ## D 1.00000 0.00518 0.00014 ## ## P value adjustment method: bonferroni We see that these p-values are much higher than the unadjusted p-values, thus, we are less likely to reject each test. As a result, the FWER is 0.05, instead of an error rate of 0.05 for each test. We can simulate the 100 test scenarios to illustrate this point. get_p_val = function() { # create data for two groups, equal mean y = rnorm(20, mean = 0, sd = 1) g = c(rep(&quot;A&quot;, 10), rep(&quot;B&quot;, 10)) # p-value of t-test when null is true glance(t.test(y ~ g, var.equal = TRUE))$p.value } set.seed(1337) # FWER with 100 tests # desired rate = 0.05 # no adjustment mean(replicate(1000, any(replicate(100, get_p_val()) &lt; 0.05))) ## [1] 0.994 # FWER with 100 tests # desired rate = 0.05 # bonferroni adjustment mean(replicate(1000, any(p.adjust(replicate(100, get_p_val()), &quot;bonferroni&quot;) &lt; 0.05))) ## [1] 0.058 For the specific case of testing all two-way mean differences after an ANOVA test, there are a number of potential methods for making an adjustment of this type. The pros and cons of the potential methods are beyond the scope of this course. We choose a method for its ease of use, and to a lesser extent, its developer. Tukey’s Honest Significance difference can be applied directly to an object which was created using aov(). It will adjust the p-values of the pairwise comparisons of the means to control the FWER, in this case, for 0.05. Notice it also gives confidence intervals for the difference of the means. TukeyHSD(coag_aov, conf.level = 0.95) ## Tukey multiple comparisons of means ## 95% family-wise confidence level ## ## Fit: aov(formula = coag ~ diet, data = coagulation) ## ## $diet ## diff lwr upr p adj ## B-A 5 0.7245544 9.275446 0.0183283 ## C-A 7 2.7245544 11.275446 0.0009577 ## D-A 0 -4.0560438 4.056044 1.0000000 ## C-B 2 -1.8240748 5.824075 0.4766005 ## D-B -5 -8.5770944 -1.422906 0.0044114 ## D-C -7 -10.5770944 -3.422906 0.0001268 Based on these results, we see no difference between A and D as well as B and C. All other pairwise comparisons are significant. If you return to the original boxplot, these results should not be surprising. Also, nicely, we can easily produce a plot of these confidence intervals. plot(TukeyHSD(coag_aov, conf.level = 0.95)) The creator of this method, John Tukey, is an important figure in the history of data science. He essentially predicted the rise of data science over 50 years ago. For some retrospective thoughts on those 50 years, see this paper from David Donoho. 12.5 Two-Way ANOVA What if there is more than one factor variable? Why do we need to limit ourselves to experiments with only one factor? We don’t! Consider the model \\[ y_{ijk} = \\mu + \\alpha_i + \\beta_j + (\\alpha \\beta)_{ij} + \\epsilon_{ijk} \\] where \\(\\epsilon_{ijk}\\) are \\(N(0, \\sigma^2)\\) random variables. We add constraints \\[ \\sum_{i=1}^{I} \\alpha_i = 0 \\quad \\quad \\sum_{j=1}^{J} \\beta_j = 0. \\] and \\[ (\\alpha \\beta)_{1j} + (\\alpha \\beta)_{2j} + (\\alpha \\beta)_{3j} = 0 \\\\ (\\alpha \\beta)_{i1} + (\\alpha \\beta)_{i2} + (\\alpha \\beta)_{i3} + (\\alpha \\beta)_{i4} = 0 \\] for any \\(i\\) or \\(j\\). Here, \\(i = 1, 2, \\ldots I\\) where \\(I\\) is the number of levels of factor \\(A\\). \\(j = 1, 2, \\ldots J\\) where \\(J\\) is the number of levels of factor \\(B\\). \\(k = 1, 2, \\ldots K\\) where \\(K\\) is the number of replicates per group. Here, we can think of a group as a combination of a level from each of the factors. So for example, one group will receive level \\(2\\) of factor \\(A\\) and level \\(3\\) of factor \\(B\\). The number of replicates is the number of subjects in each group. Here \\(y_{135}\\) would be the measurement for the fifth member (replicate) of the group for level \\(1\\) of factor \\(A\\) and level \\(3\\) of factor \\(B\\). We call this setup an \\(I \\times J\\) factorial design with \\(K\\) replicates. (Our current notation only allows for equal replicates in each group. It isn’t difficult to allow for different replicates for different groups, but we’ll proceed using equal replicates per group, which if possible, is desirable.) \\(\\alpha_i\\) measures the effect of level \\(i\\) of factor \\(A\\). We call these the main effects of factor \\(A\\). \\(\\beta_j\\) measures the effect of level \\(j\\) of factor \\(B\\). We call these the main effects of factor \\(B\\). \\((\\alpha \\beta)_{ij}\\) is a single parameter. We use \\(\\alpha \\beta\\) to note that this parameter measures the interaction between the two main effects. Under this setup, there are a number of models that we can compare. Consider a \\(2 \\times 2\\) factorial design. The following tables show the means for each of the possible groups under each model. Interaction Model: \\(y_{ijk} = \\mu + \\alpha_i + \\beta_j + (\\alpha \\beta)_{ij} + \\epsilon_{ijk}\\) Factor B, Level 1 Factor B, Level 2 Factor A, Level 1 \\(\\mu + \\alpha_1 + \\beta_1 + (\\alpha\\beta)_{11}\\) \\(\\mu + \\alpha_1 + \\beta_2 + (\\alpha\\beta)_{12}\\) Factor A, Level 2 \\(\\mu + \\alpha_2 + \\beta_1 + (\\alpha\\beta)_{21}\\) \\(\\mu + \\alpha_2 + \\beta_2 + (\\alpha\\beta)_{22}\\) Additive Model: \\(y_{ijk} = \\mu + \\alpha_i + \\beta_j + \\epsilon_{ijk}\\) Factor B, Level 1 Factor B, Level 2 Factor A, Level 1 \\(\\mu + \\alpha_1 + \\beta_1\\) \\(\\mu + \\alpha_1 + \\beta_2\\) Factor A, Level 2 \\(\\mu + \\alpha_2 + \\beta_1\\) \\(\\mu + \\alpha_2 + \\beta_2\\) Factor B Only Model (One-Way): \\(y_{ijk} = \\mu + \\beta_j + \\epsilon_{ijk}\\) Factor B, Level 1 Factor B, Level 2 Factor A, Level 1 \\(\\mu + \\beta_1\\) \\(\\mu + \\beta_2\\) Factor A, Level 2 \\(\\mu + \\beta_1\\) \\(\\mu + \\beta_2\\) Factor A Only Model (One-Way): \\(y_{ijk} = \\mu + \\alpha_i + \\epsilon_{ijk}\\) Factor B, Level 1 Factor B, Level 2 Factor A, Level 1 \\(\\mu + \\alpha_1\\) \\(\\mu + \\alpha_1\\) Factor A, Level 2 \\(\\mu + \\alpha_2\\) \\(\\mu + \\alpha_2\\) Null Model: \\(y_{ijk} = \\mu + \\epsilon_{ijk}\\) Factor B, Level 1 Factor B, Level 2 Factor A, Level 1 \\(\\mu\\) \\(\\mu\\) Factor A, Level 2 \\(\\mu\\) \\(\\mu\\) The question then, is which of these models should we use if we have two factors? The most important question to consider is whether or not we should model the interaction. Is the effect of Factor A the same for all levels of Factor B? In the additive model, yes. In the interaction model, no. Both models would use a different mean for each group, but in a very specific way in both cases. Let’s discuss these comparisons by looking at some examples. We’ll first look at the rats data from the faraway package. There are two factors here: poison and treat. We use the levels() function to extract the levels of a factor variable. levels(rats$poison) ## [1] &quot;I&quot; &quot;II&quot; &quot;III&quot; levels(rats$treat) ## [1] &quot;A&quot; &quot;B&quot; &quot;C&quot; &quot;D&quot; Here, 48 rats were randomly assigned both one of three poisons and one of four possible treatments. The experimenters then measured their survival time in tens of hours. A total of 12 groups, each with 4 replicates. Before running any tests, we should first look at the data. We will create interaction plots, which will help us visualize the effect of one factor, as we move through the levels of another factor. par(mfrow = c(1, 2)) with(rats, interaction.plot(poison, treat, time, lwd = 2, col = 1:4)) with(rats, interaction.plot(treat, poison, time, lwd = 2, col = 1:3)) If there is not interaction, thus an additive model, we would expect to see parallel lines. That would mean, when we change the level of one factor, there can be an effect on the response. However, the difference between the levels of the other factor should still be the same. The obvious indication of interaction would be lines that cross while heading in different directions. Here we don’t see that, but the lines aren’t strictly parallel, and there is some overlap on the right panel. However, is this interaction effect significant? Let’s fit each of the possible models, then investigate their estimates for each of the group means. rats_int = aov(time ~ poison * treat, data = rats) # interaction model rats_add = aov(time ~ poison + treat, data = rats) # additive model rats_pois = aov(time ~ poison , data = rats) # single factor model rats_treat = aov(time ~ treat, data = rats) # single factor model rats_null = aov(time ~ 1, data = rats) # null model To get the estimates, we’ll create a table which we will predict on. rats_table = expand.grid(poison = unique(rats$poison), treat = unique(rats$treat)) rats_table ## poison treat ## 1 I A ## 2 II A ## 3 III A ## 4 I B ## 5 II B ## 6 III B ## 7 I C ## 8 II C ## 9 III C ## 10 I D ## 11 II D ## 12 III D matrix(paste0(rats_table$poison, &quot;-&quot;, rats_table$treat) , 4, 3, byrow = TRUE) ## [,1] [,2] [,3] ## [1,] &quot;I-A&quot; &quot;II-A&quot; &quot;III-A&quot; ## [2,] &quot;I-B&quot; &quot;II-B&quot; &quot;III-B&quot; ## [3,] &quot;I-C&quot; &quot;II-C&quot; &quot;III-C&quot; ## [4,] &quot;I-D&quot; &quot;II-D&quot; &quot;III-D&quot; Since we’ll be repeating ourselves a number of times, we write a function to perform the prediction. Some housekeeping is done to keep the estimates in order, and provide row and column names. Above, we’ve shown where each of the estimates will be placed in the resulting matrix. get_est_means = function(model, table) { mat = matrix(predict(model, table), nrow = 4, ncol = 3, byrow = TRUE) colnames(mat) = c(&quot;I&quot;, &quot;II&quot;, &quot;III&quot;) rownames(mat) = c(&quot;A&quot;, &quot;B&quot;, &quot;C&quot;, &quot;D&quot;) mat } First, we obtain the estimates from the interaction model. Note that each cell has a different value. knitr::kable(get_est_means(model = rats_int, table = rats_table)) I II III A 0.4125 0.3200 0.210 B 0.8800 0.8150 0.335 C 0.5675 0.3750 0.235 D 0.6100 0.6675 0.325 Next, we obtain the estimates from the additive model. Again, each cell has a different value. We also see that these estimates are somewhat close to those from the interaction model. knitr::kable(get_est_means(model = rats_add, table = rats_table)) I II III A 0.4522917 0.3791667 0.1110417 B 0.8147917 0.7416667 0.4735417 C 0.5306250 0.4575000 0.1893750 D 0.6722917 0.5991667 0.3310417 To understand the difference, let’s consider the effect of the treatments. additive_means = get_est_means(model = rats_add, table = rats_table) additive_means[&quot;A&quot;,] - additive_means[&quot;B&quot;,] ## I II III ## -0.3625 -0.3625 -0.3625 interaction_means = get_est_means(model = rats_int, table = rats_table) interaction_means[&quot;A&quot;,] - interaction_means[&quot;B&quot;,] ## I II III ## -0.4675 -0.4950 -0.1250 This is the key difference between the interaction and additive models. The difference between the effect of treatments A and B is the same for each poison in the additive model. They are different in the interaction model. The remaining three models are much simpler, having either only row or only column effects. Or no effects in the case of the null model. knitr::kable(get_est_means(model = rats_pois, table = rats_table)) I II III A 0.6175 0.544375 0.27625 B 0.6175 0.544375 0.27625 C 0.6175 0.544375 0.27625 D 0.6175 0.544375 0.27625 knitr::kable(get_est_means(model = rats_treat, table = rats_table)) I II III A 0.3141667 0.3141667 0.3141667 B 0.6766667 0.6766667 0.6766667 C 0.3925000 0.3925000 0.3925000 D 0.5341667 0.5341667 0.5341667 knitr::kable(get_est_means(model = rats_null, table = rats_table)) I II III A 0.479375 0.479375 0.479375 B 0.479375 0.479375 0.479375 C 0.479375 0.479375 0.479375 D 0.479375 0.479375 0.479375 To perform the needed tests, we will need to create another ANOVA table. (We’ll skip the details of the sums of squares calculations and simply let R take care of them.) Source Sum of Squares Degrees of Freedom Mean Square \\(F\\) Factor A SSA \\(I -1\\) SSA / DFA MSA / MSE Factor B SSB \\(J -1\\) SSB / DFB MSB / MSE AB Interaction SSAB \\((I -1)(J -1)\\) SSAB / DFAB MSAB / MSE Error SSE \\(IJ(K - 1)\\) SSE / DFE Total SST \\(IJK - 1\\) The row for AB Interaction tests: \\[ H_0: \\text{ All }(\\alpha\\beta)_{ij} = 0. \\quad \\text{vs} \\quad H_1: \\text{ Not all } (\\alpha\\beta)_{ij} \\text{ are } 0. \\] Null Model: \\(y_{ijk} = \\mu + \\alpha_i + \\beta_j + \\epsilon_{ijk}.\\) (Additive Model.) Alternative Model: \\(y_{ijk} = \\mu + \\alpha_i + \\beta_j + (\\alpha \\beta)_{ij} + \\epsilon_{ijk}.\\) (Interaction Model.) We reject the null when the \\(F\\) statistic is large. Under the null hypothesis, the distribution of the test statistic is \\(F\\) with degrees of freedom \\((I -1)(J -1)\\) and \\(IJ(K - 1)\\). The row for Factor B tests: \\[ H_0: \\text{ All }\\beta_{j} = 0. \\quad \\text{vs} \\quad H_1: \\text{ Not all } \\beta_{j} \\text{ are } 0. \\] Null Model: \\(y_{ijk} = \\mu + \\alpha_i + \\epsilon_{ijk}.\\) (Only Factor A Model.) Alternative Model: \\(y_{ijk} = \\mu + \\alpha_i + \\beta_j + \\epsilon_{ijk}.\\) (Additive Model.) We reject the null when the \\(F\\) statistic is large. Under the null hypothesis, the distribution of the test statistic is \\(F\\) with degrees of freedom \\(J - 1\\) and \\(IJ(K - 1)\\). The row for Factor A tests: \\[ H_0: \\text{ All }\\alpha_{i} = 0. \\quad \\text{vs} \\quad H_1: \\text{ Not all } \\alpha_{i} \\text{ are } 0. \\] Null Model: \\(y_{ijk} = \\mu + \\beta_j + \\epsilon_{ijk}.\\) (Only Factor B Model.) Alternative Model: \\(y_{ijk} = \\mu + \\alpha_i + \\beta_j + \\epsilon_{ijk}.\\) (Additive Model.) We reject the null when the \\(F\\) statistic is large. Under the null hypothesis, the distribution of the test statistic is \\(F\\) with degrees of freedom \\(I - 1\\) and \\(IJ(K - 1)\\). These tests should be performed according to the model hierarchy. First consider the test of interaction. If it is significant, we select the interaction model and perform no further testing. If interaction is not significant, we then consider the necessity of the individual factors of the additive model. Model Hierarchy summary(aov(time ~ poison * treat, data = rats)) ## Df Sum Sq Mean Sq F value Pr(&gt;F) ## poison 2 1.0330 0.5165 23.222 3.33e-07 *** ## treat 3 0.9212 0.3071 13.806 3.78e-06 *** ## poison:treat 6 0.2501 0.0417 1.874 0.112 ## Residuals 36 0.8007 0.0222 ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 Using a significance level of \\(\\alpha = 0.05\\), we see that the interaction is not significant. Within the additive model, both factors are significant, so we select the additive model. Within the additive model, we could do further testing about the main effects. TukeyHSD(aov(time ~ poison + treat, data = rats)) ## Tukey multiple comparisons of means ## 95% family-wise confidence level ## ## Fit: aov(formula = time ~ poison + treat, data = rats) ## ## $poison ## diff lwr upr p adj ## II-I -0.073125 -0.2089936 0.0627436 0.3989657 ## III-I -0.341250 -0.4771186 -0.2053814 0.0000008 ## III-II -0.268125 -0.4039936 -0.1322564 0.0000606 ## ## $treat ## diff lwr upr p adj ## B-A 0.36250000 0.18976135 0.53523865 0.0000083 ## C-A 0.07833333 -0.09440532 0.25107198 0.6221729 ## D-A 0.22000000 0.04726135 0.39273865 0.0076661 ## C-B -0.28416667 -0.45690532 -0.11142802 0.0004090 ## D-B -0.14250000 -0.31523865 0.03023865 0.1380432 ## D-C 0.14166667 -0.03107198 0.31440532 0.1416151 For an example with interaction, we investigate the warpbreaks dataset, a default dataset in R. par(mfrow = c(1, 2)) with(warpbreaks, interaction.plot(wool, tension, breaks, lwd = 2, col = 2:4)) with(warpbreaks, interaction.plot(tension, wool, breaks, lwd = 2, col = 2:3)) Either plot makes it rather clear that the wool and tensions factors interact. summary(aov(breaks ~ wool * tension, data = warpbreaks)) ## Df Sum Sq Mean Sq F value Pr(&gt;F) ## wool 1 451 450.7 3.765 0.058213 . ## tension 2 2034 1017.1 8.498 0.000693 *** ## wool:tension 2 1003 501.4 4.189 0.021044 * ## Residuals 48 5745 119.7 ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 Using an \\(\\alpha\\) of \\(0.05\\), the ANOVA test finds that the interaction is significant, so we use the interaction model here. 12.6 R Markdown The R Markdown file for this chapter can be found here: anova.Rmd The file was created using R version 4.4.1. "],["model-diagnostics.html", "Chapter 13 Model Diagnostics 13.1 Model Assumptions 13.2 Checking Assumptions 13.3 Unusual Observations 13.4 Data Analysis Examples 13.5 R Markdown", " Chapter 13 Model Diagnostics “Your assumptions are your windows on the world. Scrub them off every once in a while, or the light won’t come in.” — Isaac Asimov After reading this chapter you will be able to: Understand the assumptions of a regression model. Assess regression model assumptions using visualizations and tests. Understand leverage, outliers, and influential points. Be able to identify unusual observations in regression models. 13.1 Model Assumptions Recall the multiple linear regression model that we have defined. \\[ Y_i = \\beta_0 + \\beta_1 x_{i1} + \\beta_2 x_{i2} + \\cdots + \\beta_{p-1} x_{i(p-1)} + \\epsilon_i, \\qquad i = 1, 2, \\ldots, n. \\] Using matrix notation, this model can be written much more succinctly as \\[ Y = X \\beta + \\epsilon. \\] Given data, we found the estimates for the \\(\\beta\\) parameters using \\[ \\hat{\\beta} = \\left( X^\\top X \\right)^{-1}X^\\top y. \\] We then noted that these estimates had mean \\[ \\text{E}[\\hat{\\beta}] = \\beta, \\] and variance \\[ \\text{Var}[\\hat{\\beta}] = \\sigma^2 \\left( X^\\top X \\right)^{-1}. \\] In particular, an individual parameter, say \\(\\hat{\\beta}_j\\) had a normal distribution \\[ \\hat{\\beta}_j \\sim N\\left(\\beta_j, \\sigma^2 C_{jj} \\right) \\] where \\(C\\) was the matrix defined as \\[ C = \\left(X^\\top X\\right)^{-1}. \\] We then used this fact to define \\[ \\frac{\\hat{\\beta}_j - \\beta_j}{s_e \\sqrt{C_{jj}}} \\sim t_{n-p}, \\] which we used to perform hypothesis testing. So far we have looked at various metrics such as RMSE, RSE, and \\(R^2\\) to determine how well our model fits our data. Each of these in some way considers the expression \\[ \\sum_{i = 1}^n (y_i - \\hat{y}_i)^2. \\] So, essentially each of these looks at how close the data points are to the model. However is that all we care about? It could be that the errors are made in a systematic way, which means that our model is misspecified. We may need additional interaction terms, or polynomial terms which we will see later. It is also possible that at a particular set of predictor values, the errors are very small, but at a different set of predictor values, the errors are large. Perhaps most of the errors are very small, but some are very large. This would suggest that the errors do not follow a normal distribution. Are these issues that we care about? If all we would like to do is predict, possibly not, since we would only care about the size of our errors. However, if we would like to perform inference, for example to determine if a particular predictor is important, we care a great deal. All of the distributional results, such as a \\(t\\)-test for a single predictor, are derived under the assumptions of our model. Technically, the assumptions of the model are encoded directly in a model statement such as, \\[ Y_i = \\beta_0 + \\beta_1 x_{i1} + \\beta_2 x_{i2} + \\cdots + \\beta_{p-1} x_{i(p-1)} + \\epsilon_i \\] where \\(\\epsilon_i \\sim N(0, \\sigma^2).\\) Often, the assumptions of linear regression, are stated as, Linearity: the response can be written as a linear combination of the predictors. (With noise about this true linear relationship.) Independence: the errors are independent. Normality: the distribution of the errors should follow a normal distribution. Equal Variance: the error variance is the same at any set of predictor values. The linearity assumption is encoded as \\[ \\beta_0 + \\beta_1 x_{i1} + \\beta_2 x_{i2} + \\cdots + \\beta_{p-1} x_{i(p-1)}, \\] while the remaining three, are all encoded in \\[ \\epsilon_i \\sim N(0, \\sigma^2), \\] since the \\(\\epsilon_i\\) are \\(iid\\) normal random variables with constant variance. If these assumptions are met, great! We can perform inference, and it is valid. If these assumptions are not met, we can still “perform” a \\(t\\)-test using R, but the results are not valid. The distributions of the parameter estimates will not be what we expect. Hypothesis tests will then accept or reject incorrectly. Essentially, garbage in, garbage out. 13.2 Checking Assumptions We’ll now look at a number of tools for checking the assumptions of a linear model. To test these tools, we’ll use data simulated from three models: \\[ \\text{Model 1:} \\quad Y = 3 + 5x + \\epsilon, \\quad \\epsilon \\sim N(0, 1) \\] \\[ \\text{Model 2:} \\quad Y = 3 + 5x + \\epsilon, \\quad \\epsilon \\sim N(0, x^2) \\] \\[ \\text{Model 3:} \\quad Y = 3 + 5x^2 + \\epsilon, \\quad \\epsilon \\sim N(0, 25) \\] sim_1 = function(sample_size = 500) { x = runif(n = sample_size) * 5 y = 3 + 5 * x + rnorm(n = sample_size, mean = 0, sd = 1) data.frame(x, y) } sim_2 = function(sample_size = 500) { x = runif(n = sample_size) * 5 y = 3 + 5 * x + rnorm(n = sample_size, mean = 0, sd = x) data.frame(x, y) } sim_3 = function(sample_size = 500) { x = runif(n = sample_size) * 5 y = 3 + 5 * x ^ 2 + rnorm(n = sample_size, mean = 0, sd = 5) data.frame(x, y) } 13.2.1 Fitted versus Residuals Plot Probably our most useful tool will be a Fitted versus Residuals Plot. It will be useful for checking both the linearity and constant variance assumptions. Data generated from Model 1 above should not show any signs of violating assumptions, so we’ll use this to see what a good fitted versus residuals plot should look like. First, we’ll simulate observations from this model. set.seed(42) sim_data_1 = sim_1() head(sim_data_1) ## x y ## 1 4.574030 24.773995 ## 2 4.685377 26.475936 ## 3 1.430698 8.954993 ## 4 4.152238 23.951210 ## 5 3.208728 20.341344 ## 6 2.595480 14.943525 We then fit the model and add the fitted line to a scatterplot. plot(y ~ x, data = sim_data_1, col = &quot;grey&quot;, pch = 20, main = &quot;Data from Model 1&quot;) fit_1 = lm(y ~ x, data = sim_data_1) abline(fit_1, col = &quot;darkorange&quot;, lwd = 3) We now plot a fitted versus residuals plot. Note, this is residuals on the \\(y\\)-axis despite the ordering in the name. Sometimes you will see this called a residuals versus fitted, or residuals versus predicted plot. plot(fitted(fit_1), resid(fit_1), col = &quot;grey&quot;, pch = 20, xlab = &quot;Fitted&quot;, ylab = &quot;Residuals&quot;, main = &quot;Data from Model 1&quot;) abline(h = 0, col = &quot;darkorange&quot;, lwd = 2) We should look for two things in this plot. At any fitted value, the mean of the residuals should be roughly 0. If this is the case, the linearity assumption is valid. For this reason, we generally add a horizontal line at \\(y = 0\\) to emphasize this point. At every fitted value, the spread of the residuals should be roughly the same. If this is the case, the constant variance assumption is valid. Here we see this is the case for both. To get a better idea of how a fitted versus residuals plot can be useful, we will simulate from models with violated assumptions. Model 2 is an example of non-constant variance. In this case, the variance is larger for larger values of the predictor variable \\(x\\). set.seed(42) sim_data_2 = sim_2() fit_2 = lm(y ~ x, data = sim_data_2) plot(y ~ x, data = sim_data_2, col = &quot;grey&quot;, pch = 20, main = &quot;Data from Model 2&quot;) abline(fit_2, col = &quot;darkorange&quot;, lwd = 3) This actually is rather easy to see here by adding the fitted line to a scatterplot. This is because we are only performing simple linear regression. With multiple regression, a fitted versus residuals plot is a necessity, since adding a fitted regression to a scatterplot isn’t exactly possible. plot(fitted(fit_2), resid(fit_2), col = &quot;grey&quot;, pch = 20, xlab = &quot;Fitted&quot;, ylab = &quot;Residuals&quot;, main = &quot;Data from Model 2&quot;) abline(h = 0, col = &quot;darkorange&quot;, lwd = 2) On the fitted versus residuals plot, we see two things very clearly. For any fitted value, the residuals seem roughly centered at 0. This is good! The linearity assumption is not violated. However, we also see very clearly, that for larger fitted values, the spread of the residuals is larger. This is bad! The constant variance assumption is violated here. Now we will demonstrate a model which does not meet the linearity assumption. Model 3 is an example of a model where \\(Y\\) is not a linear combination of the predictors. In this case the predictor is \\(x\\), but the model uses \\(x^2\\). (We’ll see later that this is something that a “linear” model can deal with. The fix is simple, just make \\(x^2\\) a predictor!) set.seed(42) sim_data_3 = sim_3() fit_3 = lm(y ~ x, data = sim_data_3) plot(y ~ x, data = sim_data_3, col = &quot;grey&quot;, pch = 20, main = &quot;Data from Model 3&quot;) abline(fit_3, col = &quot;darkorange&quot;, lwd = 3) Again, this is rather clear on the scatterplot, but again, we wouldn’t be able to check this plot for multiple regression. plot(fitted(fit_3), resid(fit_3), col = &quot;grey&quot;, pch = 20, xlab = &quot;Fitted&quot;, ylab = &quot;Residuals&quot;, main = &quot;Data from Model 3&quot;) abline(h = 0, col = &quot;darkorange&quot;, lwd = 2) This time on the fitted versus residuals plot, for any fitted value, the spread of the residuals is about the same. However, they are not even close to centered at zero! At small and large fitted values the model is underestimating, while at medium fitted values, the model is overestimating. These are systematic errors, not random noise. So the constant variance assumption is met, but the linearity assumption is violated. The form of our model is simply wrong. We’re trying to fit a line to a curve! 13.2.2 Breusch-Pagan Test Constant variance is often called homoscedasticity. Conversely, non-constant variance is called heteroscedasticity. We’ve seen how we can use a fitted versus residuals plot to look for these attributes. While a fitted versus residuals plot can give us an idea about homoscedasticity, sometimes we would prefer a more formal test. There are many tests for constant variance, but here we will present one, the Breusch-Pagan Test. The exact details of the test will be omitted here, but importantly the null and alternative can be considered to be, \\(H_0\\): Homoscedasticity. The errors have constant variance about the true model. \\(H_1\\): Heteroscedasticity. The errors have non-constant variance about the true model. Isn’t that convenient? A test that will specifically test the constant variance assumption. The Breusch-Pagan Test can not be performed by default in R, however the function bptest in the lmtest package implements the test. #install.packages(&quot;lmtest&quot;) library(lmtest) Let’s try it on the three models we fit above. Recall, fit_1 had no violation of assumptions, fit_2 violated the constant variance assumption, but not linearity, fit_3 violated linearity, but not constant variance. bptest(fit_1) ## ## studentized Breusch-Pagan test ## ## data: fit_1 ## BP = 1.0234, df = 1, p-value = 0.3117 For fit_1 we see a large p-value, so we do not reject the null of homoscedasticity, which is what we would expect. bptest(fit_2) ## ## studentized Breusch-Pagan test ## ## data: fit_2 ## BP = 76.693, df = 1, p-value &lt; 2.2e-16 For fit_2 we see a small p-value, so we reject the null of homoscedasticity. The constant variance assumption is violated. This matches our findings with a fitted versus residuals plot. bptest(fit_3) ## ## studentized Breusch-Pagan test ## ## data: fit_3 ## BP = 0.33466, df = 1, p-value = 0.5629 Lastly, for fit_3 we again see a large p-value, so we do not reject the null of homoscedasticity, which matches our findings with a fitted versus residuals plot. 13.2.3 Histograms We have a number of tools for assessing the normality assumption. The most obvious would be to make a histogram of the residuals. If it appears roughly normal, then we’ll believe the errors could truly be normal. par(mfrow = c(1, 3)) hist(resid(fit_1), xlab = &quot;Residuals&quot;, main = &quot;Histogram of Residuals, fit_1&quot;, col = &quot;darkorange&quot;, border = &quot;dodgerblue&quot;, breaks = 20) hist(resid(fit_2), xlab = &quot;Residuals&quot;, main = &quot;Histogram of Residuals, fit_2&quot;, col = &quot;darkorange&quot;, border = &quot;dodgerblue&quot;, breaks = 20) hist(resid(fit_3), xlab = &quot;Residuals&quot;, main = &quot;Histogram of Residuals, fit_3&quot;, col = &quot;darkorange&quot;, border = &quot;dodgerblue&quot;, breaks = 20) Above are histograms for each of the three regression models we have been considering. Notice that the first, for fit_1 appears very normal. The third, for fit_3, appears to be very non-normal. However fit_2 is not as clear. It does have a rough bell shape, however, it also has a very sharp peak. For this reason we will usually use more powerful tools such as Q-Q plots and the Shapiro-Wilk test for assessing the normality of errors. 13.2.4 Q-Q Plots Another visual method for assessing the normality of errors, which is more powerful than a histogram, is a normal quantile-quantile plot, or Q-Q plot for short. In R these are very easy to make. The qqnorm() function plots the points, and the qqline() function adds the necessary line. We create a Q-Q plot for the residuals of fit_1 to check if the errors could truly be normally distributed. qqnorm(resid(fit_1), main = &quot;Normal Q-Q Plot, fit_1&quot;, col = &quot;darkgrey&quot;) qqline(resid(fit_1), col = &quot;dodgerblue&quot;, lwd = 2) In short, if the points of the plot do not closely follow a straight line, this would suggest that the data do not come from a normal distribution. The calculations required to create the plot vary depending on the implementation, but essentially the \\(y\\)-axis is the sorted data (observed, or sample quantiles), and the \\(x\\)-axis is the values we would expect if the data did come from a normal distribution (theoretical quantiles). The Wikipedia page for Normal probability plots gives details on how this is implemented in R if you are interested. Also, to get a better idea of how Q-Q plots work, here is a quick function which creates a Q-Q plot: qq_plot = function(e) { n = length(e) normal_quantiles = qnorm(((1:n - 0.5) / n)) # normal_quantiles = qnorm(((1:n) / (n + 1))) # plot theoretical verus observed quantiles plot(normal_quantiles, sort(e), xlab = c(&quot;Theoretical Quantiles&quot;), ylab = c(&quot;Sample Quantiles&quot;), col = &quot;darkgrey&quot;) title(&quot;Normal Q-Q Plot&quot;) # calculate line through the first and third quartiles slope = (quantile(e, 0.75) - quantile(e, 0.25)) / (qnorm(0.75) - qnorm(0.25)) intercept = quantile(e, 0.25) - slope * qnorm(0.25) # add to existing plot abline(intercept, slope, lty = 2, lwd = 2, col = &quot;dodgerblue&quot;) } We can then verify that it is essentially equivalent to using qqnorm() and qqline() in R. set.seed(420) x = rnorm(100, mean = 0 , sd = 1) par(mfrow = c(1, 2)) qqnorm(x, col = &quot;darkgrey&quot;) qqline(x, lty = 2, lwd = 2, col = &quot;dodgerblue&quot;) qq_plot(x) To get a better idea of what “close to the line” means, we perform a number of simulations, and create Q-Q plots. First we simulate data from a normal distribution with different sample sizes, and each time create a Q-Q plot. par(mfrow = c(1, 3)) set.seed(420) qq_plot(rnorm(10)) qq_plot(rnorm(25)) qq_plot(rnorm(100)) Since this data is sampled from a normal distribution, these are all, by definition, good Q-Q plots. The points are “close to the line” and we would conclude that this data could have been sampled from a normal distribution. Notice in the first plot, one point is somewhat far from the line, but just one point, in combination with the small sample size, is not enough to make us worried. We see with the large sample size, all of the points are rather close to the line. Next, we simulate data from a \\(t\\) distribution with a small degrees of freedom, for different sample sizes. par(mfrow = c(1, 3)) set.seed(420) qq_plot(rt(10, df = 4)) qq_plot(rt(25, df = 4)) qq_plot(rt(100, df = 4)) Recall that as the degrees of freedom for a \\(t\\) distribution become larger, the distribution becomes more and more similar to a normal. Here, using 4 degrees of freedom, we have a distribution that is somewhat normal, it is symmetrical and roughly bell-shaped, however it has “fat tails.” This presents itself clearly in the third panel. While many of the points are close to the line, at the edges, there are large discrepancies. This indicates that the values are too small (negative) or too large (positive) compared to what we would expect for a normal distribution. So for the sample size of 100, we would conclude that the normality assumption is violated. (If these were residuals of a model.) For sample sizes of 10 and 25 we may be suspicious, but not entirely confident. Reading Q-Q plots, is a bit of an art, not completely a science. Next, we simulate data from an exponential distribution. par(mfrow = c(1, 3)) set.seed(420) qq_plot(rexp(10)) qq_plot(rexp(25)) qq_plot(rexp(100)) This is a distribution that is not very similar to a normal, so in all three cases, we see points that are far from the lines, so we would think that the normality assumption is violated. For a better understanding of which Q-Q plots are “good,” repeat the simulations above a number of times (without setting the seed) and pay attention to the differences between those that are simulated from normal, and those that are not. Also consider different sample sizes and distribution parameters. Returning to our three regressions, recall, fit_1 had no violation of assumptions, fit_2 violated the constant variance assumption, but not linearity, fit_3 violated linearity, but not constant variance. We’ll now create a Q-Q plot for each to assess normality of errors. qqnorm(resid(fit_1), main = &quot;Normal Q-Q Plot, fit_1&quot;, col = &quot;darkgrey&quot;) qqline(resid(fit_1), col = &quot;dodgerblue&quot;, lwd = 2) For fit_1, we have a near perfect Q-Q plot. We would believe the errors follow a normal distribution. qqnorm(resid(fit_2), main = &quot;Normal Q-Q Plot, fit_2&quot;, col = &quot;darkgrey&quot;) qqline(resid(fit_2), col = &quot;dodgerblue&quot;, lwd = 2) For fit_2, we have a suspect Q-Q plot. We would probably not believe the errors follow a normal distribution. qqnorm(resid(fit_3), main = &quot;Normal Q-Q Plot, fit_3&quot;, col = &quot;darkgrey&quot;) qqline(resid(fit_3), col = &quot;dodgerblue&quot;, lwd = 2) Lastly, for fit_3, we again have a suspect Q-Q plot. We would probably not believe the errors follow a normal distribution. 13.2.5 Shapiro-Wilk Test Histograms and Q-Q Plots give a nice visual representation of the residuals distribution, however if we are interested in formal testing, there are a number of options available. A commonly used test is the Shapiro–Wilk test, which is implemented in R. set.seed(42) shapiro.test(rnorm(25)) ## ## Shapiro-Wilk normality test ## ## data: rnorm(25) ## W = 0.9499, p-value = 0.2495 shapiro.test(rexp(25)) ## ## Shapiro-Wilk normality test ## ## data: rexp(25) ## W = 0.71164, p-value = 1.05e-05 This gives us the value of the test statistic and its p-value. The null hypothesis assumes the data were sampled from a normal distribution, thus a small p-value indicates we believe there is only a small probability the data could have been sampled from a normal distribution. For details, see: Wikipedia: Shapiro–Wilk test. In the above examples, we see we fail to reject for the data sampled from normal, and reject on the non-normal data, for any reasonable \\(\\alpha\\). Returning again to fit_1, fit_2, and fit_3, we see the result of running shapiro.test() on the residuals of each, returns a result for each that matches decisions based on the Q-Q plots. shapiro.test(resid(fit_1)) ## ## Shapiro-Wilk normality test ## ## data: resid(fit_1) ## W = 0.99858, p-value = 0.9622 shapiro.test(resid(fit_2)) ## ## Shapiro-Wilk normality test ## ## data: resid(fit_2) ## W = 0.93697, p-value = 1.056e-13 shapiro.test(resid(fit_3)) ## ## Shapiro-Wilk normality test ## ## data: resid(fit_3) ## W = 0.97643, p-value = 3.231e-07 13.3 Unusual Observations In addition to checking the assumptions of regression, we also look for any “unusual observations” in the data. Often a small number of data points can have an extremely large influence on a regression, sometimes so much so that the regression assumptions are violated as a result of these points. The following three plots are inspired by an example from Linear Models with R. par(mfrow = c(1, 3)) set.seed(42) ex_data = data.frame(x = 1:10, y = 10:1 + rnorm(n = 10)) ex_model = lm(y ~ x, data = ex_data) # low leverage, large residual, small influence point_1 = c(5.4, 11) ex_data_1 = rbind(ex_data, point_1) model_1 = lm(y ~ x, data = ex_data_1) plot(y ~ x, data = ex_data_1, cex = 2, pch = 20, col = &quot;grey&quot;, main = &quot;Low Leverage, Large Residual, Small Influence&quot;) points(x = point_1[1], y = point_1[2], pch = 1, cex = 4, col = &quot;black&quot;, lwd = 2) abline(ex_model, col = &quot;dodgerblue&quot;, lwd = 2) abline(model_1, lty = 2, col = &quot;darkorange&quot;, lwd = 2) legend(&quot;bottomleft&quot;, c(&quot;Original Data&quot;, &quot;Added Point&quot;), lty = c(1, 2), col = c(&quot;dodgerblue&quot;, &quot;darkorange&quot;)) # high leverage, small residual, small influence point_2 = c(18, -5.7) ex_data_2 = rbind(ex_data, point_2) model_2 = lm(y ~ x, data = ex_data_2) plot(y ~ x, data = ex_data_2, cex = 2, pch = 20, col = &quot;grey&quot;, main = &quot;High Leverage, Small Residual, Small Influence&quot;) points(x = point_2[1], y = point_2[2], pch = 1, cex = 4, col = &quot;black&quot;, lwd = 2) abline(ex_model, col = &quot;dodgerblue&quot;, lwd = 2) abline(model_2, lty = 2, col = &quot;darkorange&quot;, lwd = 2) legend(&quot;bottomleft&quot;, c(&quot;Original Data&quot;, &quot;Added Point&quot;), lty = c(1, 2), col = c(&quot;dodgerblue&quot;, &quot;darkorange&quot;)) # high leverage, large residual, large influence point_3 = c(14, 5.1) ex_data_3 = rbind(ex_data, point_3) model_3 = lm(y ~ x, data = ex_data_3) plot(y ~ x, data = ex_data_3, cex = 2, pch = 20, col = &quot;grey&quot;, ylim = c(-3, 12), main = &quot;High Leverage, Large Residual, Large Influence&quot;) points(x = point_3[1], y = point_3[2], pch = 1, cex = 4, col = &quot;black&quot;, lwd = 2) abline(ex_model, col = &quot;dodgerblue&quot;, lwd = 2) abline(model_3, lty = 2, col = &quot;darkorange&quot;, lwd = 2) legend(&quot;bottomleft&quot;, c(&quot;Original Data&quot;, &quot;Added Point&quot;), lty = c(1, 2), col = c(&quot;dodgerblue&quot;, &quot;darkorange&quot;)) The blue solid line in each plot is a regression fit to the 10 original data points stored in ex_data. The dashed orange line in each plot is the result of adding a single point to the original data in ex_data. This additional point is indicated by the circled point. The slope of the regression for the original ten points, the solid blue line, is given by: coef(ex_model)[2] ## x ## -0.9696033 The added point in the first plot has a small effect on the slope, which becomes: coef(model_1)[2] ## x ## -0.9749534 We will say that this point has low leverage, is an outlier due to its large residual, but has small influence. The added point in the second plot also has a small effect on the slope, which is: coef(model_2)[2] ## x ## -0.9507397 We will say that this point has high leverage, is not an outlier due to its small residual, and has a very small influence. Lastly, the added point in the third plot has a large effect on the slope, which is now: coef(model_3)[2] ## x ## -0.5892241 This added point is influential. It both has high leverage, and is an outlier due to its large residual. We’ve now mentioned three new concepts: leverage, outliers, and influential points, each of which we will discuss in detail. 13.3.1 Leverage A data point with high leverage, is a data point that could have a large influence when fitting the model. Recall that, \\[ \\hat{\\beta} = \\left(X^\\top X \\right)^{-1} X^\\top y. \\] Thus, \\[ \\hat{y} = X \\hat{\\beta} = X \\left(X^\\top X \\right)^{-1} X^\\top y \\] Now we define, \\[ H = X \\left(X^\\top X\\right)^{-1} X^\\top \\] which we will refer to as the hat matrix. The hat matrix is used to project onto the subspace spanned by the columns of \\(X\\). It is also simply known as a projection matrix. The hat matrix is a matrix that takes the original \\(y\\) values, and adds a hat! \\[ \\hat{y} = H y \\] The diagonal elements of this matrix are called the leverages \\[ H_{ii} = h_i, \\] where \\(h_i\\) is the leverage for the \\(i\\)th observation. Large values of \\(h_i\\) indicate extreme values in \\(X\\), which may influence regression. Note that leverages only depend on \\(X\\). Here, \\(p\\), the number of \\(\\beta\\)s, is also the trace (and rank) of the hat matrix. \\[ \\sum_{i = 1}^n h_i = p \\] What is a value of \\(h_i\\) that would be considered large? There is no exact answer to this question. A common heuristic would be to compare each leverage to two times the average leverage. A leverage larger than this is considered an observation to be aware of. That is, if \\[ h_i &gt; 2 \\bar{h} \\] we say that observation \\(i\\) has large leverage. Here, \\[ \\bar{h} = \\frac{\\sum_{i = 1}^n h_i}{n} = \\frac{p}{n}. \\] For simple linear regression, the leverage for each point is given by \\[ h_i = \\frac{1}{n} + \\frac{(x_i - \\bar{x})^2}{S_{xx}}. \\] This expression should be familiar. (Think back to inference for SLR.) It suggests that the large leverages occur when \\(x\\) values are far from their mean. Recall that the regression goes through the point \\((\\bar{x},\\bar{y})\\). There are multiple ways to find leverages in R. lev_ex = data.frame( x1 = c(0, 11, 11, 7, 4, 10, 5, 8), x2 = c(1, 5, 4, 3, 1, 4, 4, 2), y = c(11, 15, 13, 14, 0, 19, 16, 8)) plot(x2 ~ x1, data = lev_ex, cex = 2) points(7, 3, pch = 20, col = &quot;red&quot;, cex = 2) Here we’ve created some multivariate data. Notice that we have plotted the \\(x\\) values, not the \\(y\\) values. The red point is \\((7, 3)\\) which is the mean of x1 and the mean of x2 respectively. We could calculate the leverages using the expressions defined above. We first create the \\(X\\) matrix, then calculate \\(H\\) as defined, and extract the diagonal elements. X = cbind(rep(1, 8), lev_ex$x1, lev_ex$x2) H = X %*% solve(t(X) %*% X) %*% t(X) diag(H) ## [1] 0.6000 0.3750 0.2875 0.1250 0.4000 0.2125 0.5875 0.4125 Notice here, we have two predictors, so the regression would have 3 \\(\\beta\\) parameters, so the sum of the diagonal elements is 3. sum(diag(H)) ## [1] 3 Alternatively, the method we will use more often, is to simply fit a regression, then use the hatvalues() function, which returns the leverages. lev_fit = lm(y ~ ., data = lev_ex) hatvalues(lev_fit) ## 1 2 3 4 5 6 7 8 ## 0.6000 0.3750 0.2875 0.1250 0.4000 0.2125 0.5875 0.4125 Again, note that here we have “used” the \\(y\\) values to fit the regression, but R still ignores them when calculating the leverages, as leverages only depend on the \\(x\\) values. coef(lev_fit) ## (Intercept) x1 x2 ## 3.7 -0.7 4.4 Let’s see what happens to these coefficients when we modify the y value of the point with the highest leverage. which.max(hatvalues(lev_fit)) ## 1 ## 1 lev_ex[which.max(hatvalues(lev_fit)),] ## x1 x2 y ## 1 0 1 11 We see that the original y value is 11. We’ll create a copy of the data, and modify this point to have a y value of 20. lev_ex_1 = lev_ex lev_ex_1$y[1] = 20 lm(y ~ ., data = lev_ex_1) ## ## Call: ## lm(formula = y ~ ., data = lev_ex_1) ## ## Coefficients: ## (Intercept) x1 x2 ## 8.875 -1.375 4.625 Notice the large changes in the coefficients. Also notice that each of the coefficients has changed in some way. Note that the leverages of the points would not have changed, as we have not modified any of the \\(x\\) values. Now let’s see what happens to these coefficients when we modify the y value of the point with the lowest leverage. which.min(hatvalues(lev_fit)) ## 4 ## 4 lev_ex[which.min(hatvalues(lev_fit)),] ## x1 x2 y ## 4 7 3 14 We see that the original y value is 14. We’ll again create a copy of the data, and modify this point to have a y value of 30. lev_ex_2 = lev_ex lev_ex_2$y[4] = 30 lm(y ~ ., data = lev_ex_2) ## ## Call: ## lm(formula = y ~ ., data = lev_ex_2) ## ## Coefficients: ## (Intercept) x1 x2 ## 5.7 -0.7 4.4 This time despite a large change in the y value, there is only small change in the coefficients. Also, only the intercept has changed! mean(lev_ex$x1) ## [1] 7 mean(lev_ex$x2) ## [1] 3 lev_ex[4,] ## x1 x2 y ## 4 7 3 14 Notice that this point was the mean of both of the predictors. Returning to our three plots, each with an added point, we can calculate the leverages for each. Note that the 11th data point each time is the added data point. hatvalues(model_1) ## 1 2 3 4 5 6 7 ## 0.33534597 0.23860732 0.16610842 0.11784927 0.09382988 0.09405024 0.11851036 ## 8 9 10 11 ## 0.16721022 0.24014985 0.33732922 0.09100926 hatvalues(model_2) ## 1 2 3 4 5 6 7 ## 0.23238866 0.18663968 0.14979757 0.12186235 0.10283401 0.09271255 0.09149798 ## 8 9 10 11 ## 0.09919028 0.11578947 0.14129555 0.66599190 hatvalues(model_3) ## 1 2 3 4 5 6 7 ## 0.27852761 0.21411043 0.16319018 0.12576687 0.10184049 0.09141104 0.09447853 ## 8 9 10 11 ## 0.11104294 0.14110429 0.18466258 0.49386503 Are any of these large? hatvalues(model_1) &gt; 2 * mean(hatvalues(model_1)) ## 1 2 3 4 5 6 7 8 9 10 11 ## FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE hatvalues(model_2) &gt; 2 * mean(hatvalues(model_2)) ## 1 2 3 4 5 6 7 8 9 10 11 ## FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE hatvalues(model_3) &gt; 2 * mean(hatvalues(model_3)) ## 1 2 3 4 5 6 7 8 9 10 11 ## FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE We see that in the second and third plots, the added point is a point of high leverage. Recall that only in the third plot did that have an influence on the regression. To understand why, we’ll need to discuss outliers. 13.3.2 Outliers Outliers are points which do not fit the model well. They may or may not have a large effect on the model. To identify outliers, we will look for observations with large residuals. Note, \\[ e = y - \\hat{y} = Iy - Hy = (I - H) y \\] Then, under the assumptions of linear regression, \\[ \\text{Var}(e_i) = (1 - h_i) \\sigma^2 \\] and thus estimating \\(\\sigma^2\\) with \\(s_e^2\\) gives \\[ \\text{SE}[e_i] = s_e \\sqrt{(1 - h_i)}. \\] We can then look at the standardized residual for each observation, \\(i = 1, 2, \\ldots n\\), \\[ r_i = \\frac{e_i}{s_e\\sqrt{1 - h_i}} \\overset{approx}{\\sim} N(\\mu = 0, \\sigma^ 2 = 1) \\] when \\(n\\) is large. We can use this fact to identify “large” residuals. For example, standardized residuals greater than 2 in magnitude should only happen approximately 5 percent of the time. Returning again to our three plots, each with an added point, we can calculate the residuals and standardized residuals for each. Standardized residuals can be obtained in R by using rstandard() where we would normally use resid(). resid(model_1) ## 1 2 3 4 5 6 7 ## 0.4949887 -1.4657145 -0.5629345 -0.3182468 -0.5718877 -1.1073271 0.4852728 ## 8 9 10 11 ## -1.1459548 0.9420814 -1.1641029 4.4138254 rstandard(model_1) ## 1 2 3 4 5 6 7 ## 0.3464701 -0.9585470 -0.3517802 -0.1933575 -0.3428264 -0.6638841 0.2949482 ## 8 9 10 11 ## -0.7165857 0.6167268 -0.8160389 2.6418234 rstandard(model_1)[abs(rstandard(model_1)) &gt; 2] ## 11 ## 2.641823 In the first plot, we see that the 11th point, the added point, is a large standardized residual. resid(model_2) ## 1 2 3 4 5 6 ## 1.03288292 -0.95203397 -0.07346766 0.14700626 -0.13084829 -0.69050140 ## 7 8 9 10 11 ## 0.87788484 -0.77755647 1.28626601 -0.84413207 0.12449986 rstandard(model_2) ## 1 2 3 4 5 6 ## 1.41447023 -1.26655590 -0.09559792 0.18822094 -0.16574677 -0.86977220 ## 7 8 9 10 11 ## 1.10506546 -0.98294409 1.64121833 -1.09295417 0.25846620 rstandard(model_2)[abs(rstandard(model_2)) &gt; 2] ## named numeric(0) In the second plot, we see that there are no points with large standardized residuals. resid(model_3) ## 1 2 3 4 5 6 ## 2.30296166 -0.04347087 0.47357980 0.33253808 -0.30683212 -1.22800087 ## 7 8 9 10 11 ## -0.02113027 -2.03808722 -0.33578039 -2.82769411 3.69191633 rstandard(model_3) ## 1 2 3 4 5 6 ## 1.41302755 -0.02555591 0.26980722 0.18535382 -0.16873216 -0.67141143 ## 7 8 9 10 11 ## -0.01157256 -1.12656475 -0.18882474 -1.63206526 2.70453408 rstandard(model_3)[abs(rstandard(model_3)) &gt; 2] ## 11 ## 2.704534 In the last plot, we see that the 11th point, the added point, is a large standardized residual. Recall that the added point in plots two and three were both high leverage, but now only the point in plot three has a large residual. We will now combine this information and discuss influence. 13.3.3 Influence As we have now seen in the three plots, some outliers only change the regression a small amount (plot one) and some outliers have a large effect on the regression (plot three). Observations that fall into the latter category, points with (some combination of) high leverage and large residual, we will call influential. A common measure of influence is Cook’s Distance, which is defined as \\[ D_i = \\frac{1}{p}r_i^2\\frac{h_i}{1-{h_i}}. \\] Notice that this is a function of both leverage and standardized residuals. A Cook’s Distance is often considered large if \\[ D_i &gt; \\frac{4}{n} \\] and an observation with a large Cook’s Distance is called influential. This is again simply a heuristic, and not an exact rule. The Cook’s distance for each point of a regression can be calculated using cooks.distance() which is a default function in R. Let’s look for influential points in the three plots we had been considering. Recall that the circled points in each plot have different characteristics: Plot One: low leverage, large residual. Plot Two: high leverage, small residual. Plot Three: high leverage, large residual. We’ll now directly check if each of these is influential. cooks.distance(model_1)[11] &gt; 4 / length(cooks.distance(model_1)) ## 11 ## FALSE cooks.distance(model_2)[11] &gt; 4 / length(cooks.distance(model_2)) ## 11 ## FALSE cooks.distance(model_3)[11] &gt; 4 / length(cooks.distance(model_3)) ## 11 ## TRUE And, as expected, the added point in the third plot, with high leverage and a large residual, is considered influential! 13.4 Data Analysis Examples 13.4.1 Good Diagnostics Last chapter we fit an additive regression to the mtcars data with mpg as the response and hp and am as predictors. Let’s perform some diagnostics on this model. First, fit the model as we did last chapter. mpg_hp_add = lm(mpg ~ hp + am, data = mtcars) plot(fitted(mpg_hp_add), resid(mpg_hp_add), col = &quot;grey&quot;, pch = 20, xlab = &quot;Fitted&quot;, ylab = &quot;Residual&quot;, main = &quot;mtcars: Fitted versus Residuals&quot;) abline(h = 0, col = &quot;darkorange&quot;, lwd = 2) The fitted versus residuals plot looks good. We don’t see any obvious pattern, and the variance looks roughly constant. (Maybe a little larger for large fitted values, but not enough to worry about.) bptest(mpg_hp_add) ## ## studentized Breusch-Pagan test ## ## data: mpg_hp_add ## BP = 7.5858, df = 2, p-value = 0.02253 The Breusch-Pagan test verifies this, at least for a small \\(\\alpha\\) value. qqnorm(resid(mpg_hp_add), col = &quot;darkgrey&quot;) qqline(resid(mpg_hp_add), col = &quot;dodgerblue&quot;, lwd = 2) The Q-Q plot looks extremely good and the Shapiro-Wilk test agrees. shapiro.test(resid(mpg_hp_add)) ## ## Shapiro-Wilk normality test ## ## data: resid(mpg_hp_add) ## W = 0.96485, p-value = 0.3706 sum(hatvalues(mpg_hp_add) &gt; 2 * mean(hatvalues(mpg_hp_add))) ## [1] 2 We see that there are two points of large leverage. sum(abs(rstandard(mpg_hp_add)) &gt; 2) ## [1] 1 There is also one point with a large residual. Do these result in any points that are considered influential? cd_mpg_hp_add = cooks.distance(mpg_hp_add) sum(cd_mpg_hp_add &gt; 4 / length(cd_mpg_hp_add)) ## [1] 2 large_cd_mpg = cd_mpg_hp_add &gt; 4 / length(cd_mpg_hp_add) cd_mpg_hp_add[large_cd_mpg] ## Toyota Corolla Maserati Bora ## 0.1772555 0.3447994 We find two influential points. Interestingly, they are very different cars. coef(mpg_hp_add) ## (Intercept) hp am ## 26.5849137 -0.0588878 5.2770853 Since the diagnostics looked good, there isn’t much need to worry about these two points, but let’s see how much the coefficients change if we remove them. mpg_hp_add_fix = lm(mpg ~ hp + am, data = mtcars, subset = cd_mpg_hp_add &lt;= 4 / length(cd_mpg_hp_add)) coef(mpg_hp_add_fix) ## (Intercept) hp am ## 27.22190933 -0.06286249 4.29765867 It seems there isn’t much of a change in the coefficients as a result of removing the supposed influential points. Notice we did not create a new dataset to accomplish this. We instead used the subset argument to lm(). Think about what the code cd_mpg_hp_add &lt;= 4 / length(cd_mpg_hp_add) does here. par(mfrow = c(2, 2)) plot(mpg_hp_add) Notice that calling plot() on a variable which stores an object created by lm() outputs four diagnostic plots by default. Use ?plot.lm to learn more. The first two should already be familiar. 13.4.2 Suspect Diagnostics Let’s consider the model big_model from last chapter which was fit to the autompg dataset. It used mpg as the response, and considered many interaction terms between the predictors disp, hp, and domestic. str(autompg) ## &#39;data.frame&#39;: 383 obs. of 9 variables: ## $ mpg : num 18 15 18 16 17 15 14 14 14 15 ... ## $ cyl : Factor w/ 3 levels &quot;4&quot;,&quot;6&quot;,&quot;8&quot;: 3 3 3 3 3 3 3 3 3 3 ... ## $ disp : num 307 350 318 304 302 429 454 440 455 390 ... ## $ hp : num 130 165 150 150 140 198 220 215 225 190 ... ## $ wt : num 3504 3693 3436 3433 3449 ... ## $ acc : num 12 11.5 11 12 10.5 10 9 8.5 10 8.5 ... ## $ year : int 70 70 70 70 70 70 70 70 70 70 ... ## $ origin : int 1 1 1 1 1 1 1 1 1 1 ... ## $ domestic: num 1 1 1 1 1 1 1 1 1 1 ... big_model = lm(mpg ~ disp * hp * domestic, data = autompg) qqnorm(resid(big_model), col = &quot;darkgrey&quot;) qqline(resid(big_model), col = &quot;dodgerblue&quot;, lwd = 2) shapiro.test(resid(big_model)) ## ## Shapiro-Wilk normality test ## ## data: resid(big_model) ## W = 0.96161, p-value = 1.824e-08 Here both the Q-Q plot, and the Shapiro-Wilk test suggest that the normality assumption is violated. big_mod_cd = cooks.distance(big_model) sum(big_mod_cd &gt; 4 / length(big_mod_cd)) ## [1] 31 Here, we find 31, so perhaps removing them will help! big_model_fix = lm(mpg ~ disp * hp * domestic, data = autompg, subset = big_mod_cd &lt; 4 / length(big_mod_cd)) qqnorm(resid(big_model_fix), col = &quot;grey&quot;) qqline(resid(big_model_fix), col = &quot;dodgerblue&quot;, lwd = 2) shapiro.test(resid(big_model_fix)) ## ## Shapiro-Wilk normality test ## ## data: resid(big_model_fix) ## W = 0.99035, p-value = 0.02068 Removing these points results in a much better Q-Q plot, and now Shapiro-Wilk fails to reject for a low \\(\\alpha\\). We’ve now seen that sometimes modifying the data can fix issues with regression. However, next chapter, instead of modifying the data, we will modify the model via transformations. 13.5 R Markdown The R Markdown file for this chapter can be found here: diagnostics.Rmd The file was created using R version 4.4.1. "],["transformations.html", "Chapter 14 Transformations 14.1 Response Transformation 14.2 Predictor Transformation Response Transformations Predictor Transformations 14.3 R Markdown", " Chapter 14 Transformations “Give me a lever long enough and a fulcrum on which to place it, and I shall move the world.” — Archimedes Please note: some data currently used in this chapter was used, changed, and passed around over the years in STAT 420 at UIUC. Its original sources, if they exist, are at this time unknown to the author. As a result, they should only be considered for use with STAT 420. Going forward they will likely be replaced with alternative sourceable data that illustrates the same concepts. At the end of this chapter you can find code seen in videos for Week 8 for STAT 420 in the MCS-DS program. It is currently in the process of being merged into the narrative of this chapter. After reading this chapter you will be able to: Understand the concept of a variance stabilizing transformation. Use transformations of the response to improve regression models. Use polynomial terms as predictors to fit more flexible regression models. Last chapter we checked the assumptions of regression models and looked at ways to diagnose possible issues. This chapter we will use transformations of both response and predictor variables in order to correct issues with model diagnostics, and to also potentially simply make a model fit data better. 14.1 Response Transformation Let’s look at some (fictional) salary data from the (fictional) company Initech. We will try to model salary as a function of years of experience. The data can be found in initech.csv. initech = read.csv(&quot;data/initech.csv&quot;) plot(salary ~ years, data = initech, col = &quot;grey&quot;, pch = 20, cex = 1.5, main = &quot;Salaries at Initech, By Seniority&quot;) We first fit a simple linear model. initech_fit = lm(salary ~ years, data = initech) summary(initech_fit) ## ## Call: ## lm(formula = salary ~ years, data = initech) ## ## Residuals: ## Min 1Q Median 3Q Max ## -57225 -18104 241 15589 91332 ## ## Coefficients: ## Estimate Std. Error t value Pr(&gt;|t|) ## (Intercept) 5302 5750 0.922 0.359 ## years 8637 389 22.200 &lt;2e-16 *** ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 ## ## Residual standard error: 27360 on 98 degrees of freedom ## Multiple R-squared: 0.8341, Adjusted R-squared: 0.8324 ## F-statistic: 492.8 on 1 and 98 DF, p-value: &lt; 2.2e-16 This model appears significant, but does it meet the model assumptions? plot(salary ~ years, data = initech, col = &quot;grey&quot;, pch = 20, cex = 1.5, main = &quot;Salaries at Initech, By Seniority&quot;) abline(initech_fit, col = &quot;darkorange&quot;, lwd = 2) Adding the fitted line to the plot, we see that the linear relationship appears correct. par(mfrow = c(1, 2)) plot(fitted(initech_fit), resid(initech_fit), col = &quot;grey&quot;, pch = 20, xlab = &quot;Fitted&quot;, ylab = &quot;Residuals&quot;, main = &quot;Fitted versus Residuals&quot;) abline(h = 0, col = &quot;darkorange&quot;, lwd = 2) qqnorm(resid(initech_fit), main = &quot;Normal Q-Q Plot&quot;, col = &quot;darkgrey&quot;) qqline(resid(initech_fit), col = &quot;dodgerblue&quot;, lwd = 2) However, from the fitted versus residuals plot it appears there is non-constant variance. Specifically, the variance increases as the fitted value increases. 14.1.1 Variance Stabilizing Transformations Recall the fitted value is our estimate of the mean at a particular value of \\(x\\). Under our usual assumptions, \\[ \\epsilon \\sim N(0,\\sigma^2) \\] and thus \\[ \\text{Var}[Y | X = x] = \\sigma^2 \\] which is a constant value for any value of \\(x\\). However, here we see that the variance is a function of the mean, \\[ \\text{Var}[Y \\mid X = x] = h(\\text{E}[Y \\mid X = x]). \\] In this case, \\(h\\) is some increasing function. In order to correct for this, we would like to find some function of \\(Y\\), \\(g(Y)\\) such that, \\[ \\text{Var}[g(Y) \\mid X = x] = c \\] where \\(c\\) is a constant that does not depend on the mean, \\(\\text{E}[Y \\mid X = x]\\). A transformation that accomplishes this is called a variance stabilizing transformation. A common variance stabilizing transformation (VST) when we see increasing variance in a fitted versus residuals plot is \\(\\log(Y)\\). Also, if the values of a variable range over more than one order of magnitude and the variable is strictly positive, then replacing the variable by its logarithm is likely to be helpful. A reminder, that for our purposes, \\(\\log\\) and \\(\\ln\\) are both the natural log. R uses log to mean the natural log, unless a different base is specified. We will now use a model with a log transformed response for the Initech data, \\[ \\log(Y_i) = \\beta_0 + \\beta_1 x_i + \\epsilon_i. \\] Note, if we re-scale the model from a log scale back to the original scale of the data, we now have \\[ Y_i = \\exp(\\beta_0 + \\beta_1 x_i) \\cdot \\exp(\\epsilon_i) \\] which has the errors entering the model in a multiplicative fashion. Fitting this model in R requires only a minor modification to our formula specification. initech_fit_log = lm(log(salary) ~ years, data = initech) Note that while log(y) is considered the new response variable, we do not actually create a new variable in R, but simply transform the variable inside the model formula. plot(log(salary) ~ years, data = initech, col = &quot;grey&quot;, pch = 20, cex = 1.5, main = &quot;Salaries at Initech, By Seniority&quot;) abline(initech_fit_log, col = &quot;darkorange&quot;, lwd = 2) Plotting the data on the transformed log scale and adding the fitted line, the relationship again appears linear, and we can already see that the variation about the fitted line looks constant. plot(salary ~ years, data = initech, col = &quot;grey&quot;, pch = 20, cex = 1.5, main = &quot;Salaries at Initech, By Seniority&quot;) curve(exp(initech_fit_log$coef[1] + initech_fit_log$coef[2] * x), from = 0, to = 30, add = TRUE, col = &quot;darkorange&quot;, lwd = 2) By plotting the data on the original scale, and adding the fitted regression, we see an exponential relationship. However, this is still a linear model, since the new transformed response, \\(\\log(y)\\), is still a linear combination of the predictors. par(mfrow = c(1, 2)) plot(fitted(initech_fit_log), resid(initech_fit_log), col = &quot;grey&quot;, pch = 20, xlab = &quot;Fitted&quot;, ylab = &quot;Residuals&quot;, main = &quot;Fitted versus Residuals&quot;) abline(h = 0, col = &quot;darkorange&quot;, lwd = 2) qqnorm(resid(initech_fit_log), main = &quot;Normal Q-Q Plot&quot;, col = &quot;darkgrey&quot;) qqline(resid(initech_fit_log), col = &quot;dodgerblue&quot;, lwd = 2) The fitted versus residuals plot looks much better. It appears the constant variance assumption is no longer violated. Comparing the RMSE using the original and transformed response, we also see that the log transformed model simply fits better, with a smaller average squared error. sqrt(mean(resid(initech_fit) ^ 2)) ## [1] 27080.16 sqrt(mean(resid(initech_fit_log) ^ 2)) ## [1] 0.1934907 But wait, that isn’t fair, this difference is simply due to the different scales being used. sqrt(mean((initech$salary - fitted(initech_fit)) ^ 2)) ## [1] 27080.16 sqrt(mean((initech$salary - exp(fitted(initech_fit_log))) ^ 2)) ## [1] 24280.36 Transforming the fitted values of the log model back to the data scale, we do indeed see that it fits better! summary(initech_fit_log) ## ## Call: ## lm(formula = log(salary) ~ years, data = initech) ## ## Residuals: ## Min 1Q Median 3Q Max ## -0.57022 -0.13560 0.03048 0.14157 0.41366 ## ## Coefficients: ## Estimate Std. Error t value Pr(&gt;|t|) ## (Intercept) 10.48381 0.04108 255.18 &lt;2e-16 *** ## years 0.07888 0.00278 28.38 &lt;2e-16 *** ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 ## ## Residual standard error: 0.1955 on 98 degrees of freedom ## Multiple R-squared: 0.8915, Adjusted R-squared: 0.8904 ## F-statistic: 805.2 on 1 and 98 DF, p-value: &lt; 2.2e-16 Again, the transformed response is a linear combination of the predictors, \\[ \\log(\\hat{y}(x)) = \\hat{\\beta}_0 + \\hat{\\beta}_1 x = 10.484 + 0.079x. \\] But now, if we re-scale the data from a log scale back to the original scale of the data, we now have \\[ \\hat{y}(x) = \\exp(\\hat{\\beta}_0) \\exp(\\hat{\\beta}_1 x) = \\exp(10.484)\\exp(0.079x). \\] We see that for every one additional year of experience, average salary increases \\(\\exp(0.079) = 1.0822\\) times. We are now multiplying, not adding. While using a \\(\\log\\) transform is possibly the most common response variable transformation, many others exist. We will now consider a family of transformations and choose the best from among them, which includes the \\(\\log\\) transform. 14.1.2 Box-Cox Transformations The Box-Cox method considers a family of transformations on strictly positive response variables, \\[ g_\\lambda(y) = \\left\\{ \\begin{array}{lr}\\displaystyle\\frac{y^\\lambda - 1}{\\lambda} &amp; \\lambda \\neq 0\\\\ &amp; \\\\ \\log(y) &amp; \\lambda = 0 \\end{array} \\right. \\] The \\(\\lambda\\) parameter is chosen by numerically maximizing the log-likelihood, \\[ L(\\lambda) = -\\frac{n}{2}\\log(RSS_\\lambda / n) + (\\lambda -1)\\sum \\log(y_i). \\] A \\(100(1 - \\alpha)\\%\\) confidence interval for \\(\\lambda\\) is, \\[ \\left\\{ \\lambda : L(\\lambda) &gt; L(\\hat{\\lambda}) - \\frac{1}{2}\\chi_{1,\\alpha}^2 \\right\\} \\] which R will plot for us to help quickly select an appropriate \\(\\lambda\\) value. We often choose a “nice” value from within the confidence interval, instead of the value of \\(\\lambda\\) that truly maximizes the likelihood. library(MASS) library(faraway) Here we need the MASS package for the boxcox() function, and we will consider a couple of datasets from the faraway package. First we will use the savings dataset as an example of using the Box-Cox method to justify the use of no transformation. We fit an additive multiple regression model with sr as the response and each of the other variables as predictors. savings_model = lm(sr ~ ., data = savings) We then use the boxcox() function to find the best transformation of the form considered by the Box-Cox method. boxcox(savings_model, plotit = TRUE) R automatically plots the log-Likelihood as a function of possible \\(\\lambda\\) values. It indicates both the value that maximizes the log-likelihood, as well as a confidence interval for the \\(\\lambda\\) value that maximizes the log-likelihood. boxcox(savings_model, plotit = TRUE, lambda = seq(0.5, 1.5, by = 0.1)) Note that we can specify a range of \\(\\lambda\\) values to consider and thus be plotted. We often specify a range that is more visually interesting. Here we see that \\(\\lambda = 1\\) is both in the confidence interval, and is extremely close to the maximum. This suggests a transformation of the form \\[ \\frac{y^\\lambda - 1}{\\lambda} = \\frac{y^1 - 1}{1} = y - 1. \\] This is essentially not a transformation. It would not change the variance or make the model fit better. By subtracting 1 from every value, we would only change the intercept of the model, and the resulting errors would be the same. plot(fitted(savings_model), resid(savings_model), col = &quot;dodgerblue&quot;, pch = 20, cex = 1.5, xlab = &quot;Fitted&quot;, ylab = &quot;Residuals&quot;) abline(h = 0, lty = 2, col = &quot;darkorange&quot;, lwd = 2) Looking at a fitted versus residuals plot verifies that there likely are not any issues with the assumptions of this model, which Breusch-Pagan and Shapiro-Wilk tests verify. library(lmtest) bptest(savings_model) ## ## studentized Breusch-Pagan test ## ## data: savings_model ## BP = 4.9852, df = 4, p-value = 0.2888 shapiro.test(resid(savings_model)) ## ## Shapiro-Wilk normality test ## ## data: resid(savings_model) ## W = 0.98698, p-value = 0.8524 Now we will use the gala dataset as an example of using the Box-Cox method to justify a transformation other than \\(\\log\\). We fit an additive multiple regression model with Species as the response and most of the other variables as predictors. gala_model = lm(Species ~ Area + Elevation + Nearest + Scruz + Adjacent, data = gala) plot(fitted(gala_model), resid(gala_model), col = &quot;dodgerblue&quot;, pch = 20, cex = 1.5, xlab = &quot;Fitted&quot;, ylab = &quot;Residuals&quot;) abline(h = 0, lty = 2, col = &quot;darkorange&quot;, lwd = 2) Even though there is not a lot of data for large fitted values, it still seems very clear that the constant variance assumption is violated. boxcox(gala_model, lambda = seq(-0.25, 0.75, by = 0.05), plotit = TRUE) Using the Box-Cox method, we see that \\(\\lambda = 0.3\\) is both in the confidence interval, and is extremely close to the maximum, which suggests a transformation of the form \\[ \\frac{y^\\lambda - 1}{\\lambda} = \\frac{y^{0.3} - 1}{0.3}. \\] We then fit a model with this transformation applied to the response. gala_model_cox = lm((((Species ^ 0.3) - 1) / 0.3) ~ Area + Elevation + Nearest + Scruz + Adjacent, data = gala) plot(fitted(gala_model_cox), resid(gala_model_cox), col = &quot;dodgerblue&quot;, pch = 20, cex = 1.5, xlab = &quot;Fitted&quot;, ylab = &quot;Residuals&quot;) abline(h = 0, lty = 2, col = &quot;darkorange&quot;, lwd = 2) The resulting fitted versus residuals plot looks much better! Lastly, we return to the initech data, and the initech_fit model we had used earlier. Recall that this was the untransformed model, that we used a \\(\\log\\) transform to fix. boxcox(initech_fit) Using the Box-Cox method, we see that \\(\\lambda = 0\\) is both in the interval, and extremely close to the maximum, which suggests a transformation of the form \\[ \\log(y). \\] So the Box-Cox method justifies our previous choice of a \\(\\log\\) transform! 14.2 Predictor Transformation In addition to transformation of the response variable, we can also consider transformations of predictor variables. Sometimes these transformations can help with violation of model assumptions, and other times they can be used to simply fit a more flexible model. str(autompg) ## &#39;data.frame&#39;: 383 obs. of 9 variables: ## $ mpg : num 18 15 18 16 17 15 14 14 14 15 ... ## $ cyl : Factor w/ 3 levels &quot;4&quot;,&quot;6&quot;,&quot;8&quot;: 3 3 3 3 3 3 3 3 3 3 ... ## $ disp : num 307 350 318 304 302 429 454 440 455 390 ... ## $ hp : num 130 165 150 150 140 198 220 215 225 190 ... ## $ wt : num 3504 3693 3436 3433 3449 ... ## $ acc : num 12 11.5 11 12 10.5 10 9 8.5 10 8.5 ... ## $ year : int 70 70 70 70 70 70 70 70 70 70 ... ## $ origin : int 1 1 1 1 1 1 1 1 1 1 ... ## $ domestic: num 1 1 1 1 1 1 1 1 1 1 ... Recall the autompg dataset from the previous chapter. Here we will attempt to model mpg as a function of hp. par(mfrow = c(1, 2)) plot(mpg ~ hp, data = autompg, col = &quot;dodgerblue&quot;, pch = 20, cex = 1.5) mpg_hp = lm(mpg ~ hp, data = autompg) abline(mpg_hp, col = &quot;darkorange&quot;, lwd = 2) plot(fitted(mpg_hp), resid(mpg_hp), col = &quot;dodgerblue&quot;, pch = 20, cex = 1.5, xlab = &quot;Fitted&quot;, ylab = &quot;Residuals&quot;) abline(h = 0, lty = 2, col = &quot;darkorange&quot;, lwd = 2) We first attempt SLR, but we see a rather obvious pattern in the fitted versus residuals plot, which includes increasing variance, so we attempt a \\(\\log\\) transform of the response. par(mfrow = c(1, 2)) plot(log(mpg) ~ hp, data = autompg, col = &quot;dodgerblue&quot;, pch = 20, cex = 1.5) mpg_hp_log = lm(log(mpg) ~ hp, data = autompg) abline(mpg_hp_log, col = &quot;darkorange&quot;, lwd = 2) plot(fitted(mpg_hp_log), resid(mpg_hp_log), col = &quot;dodgerblue&quot;, pch = 20, cex = 1.5, xlab = &quot;Fitted&quot;, ylab = &quot;Residuals&quot;) abline(h = 0, lty = 2, col = &quot;darkorange&quot;, lwd = 2) After performing the \\(\\log\\) transform of the response, we still have some of the same issues with the fitted versus response. Now, we will try also \\(\\log\\) transforming the predictor. par(mfrow = c(1, 2)) plot(log(mpg) ~ log(hp), data = autompg, col = &quot;dodgerblue&quot;, pch = 20, cex = 1.5) mpg_hp_loglog = lm(log(mpg) ~ log(hp), data = autompg) abline(mpg_hp_loglog, col = &quot;darkorange&quot;, lwd = 2) plot(fitted(mpg_hp_loglog), resid(mpg_hp_loglog), col = &quot;dodgerblue&quot;, pch = 20, cex = 1.5, xlab = &quot;Fitted&quot;, ylab = &quot;Residuals&quot;) abline(h = 0, lty = 2, col = &quot;darkorange&quot;, lwd = 2) Here, our fitted versus residuals plot looks good. 14.2.1 Polynomials Another very common “transformation” of a predictor variable is the use of polynomial transformations. They are extremely useful as they allow for more flexible models, but do not change the units of the variables. It should come as no surprise that sales of a product are related to the advertising budget for the product, but there are diminishing returns. A company cannot always expect linear returns based on an increased advertising budget. Consider monthly data for the sales of Initech widgets, \\(y\\), as a function of Initech’s advertising expenditure for said widget, \\(x\\), both in ten thousand dollars. The data can be found in marketing.csv. marketing = read.csv(&quot;data/marketing.csv&quot;) plot(sales ~ advert, data = marketing, xlab = &quot;Advert Spending (in $10,000)&quot;, ylab = &quot;Sales (in $10,000)&quot;, pch = 20, cex = 2) We would like to fit the model, \\[ Y_i = \\beta_0 + \\beta_1 x_i + \\beta_2 x_i^2 + \\epsilon_i \\] where \\(\\epsilon_i \\sim N(0,\\sigma^2)\\) for \\(i = 1, 2, \\cdots 21.\\) The response \\(y\\) is now a linear function of “two” variables which now allows \\(y\\) to be a non-linear function of the original single predictor \\(x\\). We consider this a transformation, although we have actually in some sense added another predictor. Thus, our \\(X\\) matrix is, \\[ \\begin{bmatrix} 1 &amp; x_1 &amp; x_1^2 \\\\[3pt] 1 &amp; x_2 &amp; x_2^2 \\\\[3pt] 1 &amp; x_3 &amp; x_3^2 \\\\[3pt] \\vdots &amp; \\vdots &amp; \\vdots \\\\[3pt] 1 &amp; x_{n} &amp; x_{n}^2 \\\\ \\end{bmatrix} \\] We can then proceed to fit the model as we have in the past for multiple linear regression. \\[ \\hat{\\beta} = \\left( X^\\top X \\right)^{-1}X^\\top y. \\] Our estimates will have the usual properties. The mean is still \\[ E[\\hat{\\beta}] = \\beta, \\] and variance \\[ \\text{Var}[\\hat{\\beta}] = \\sigma^2 \\left( X^\\top X \\right)^{-1}. \\] We also maintain the same distributional results \\[ \\hat{\\beta}_j \\sim N\\left(\\beta_j, \\sigma^2 C_{jj} \\right). \\] mark_mod = lm(sales ~ advert, data = marketing) summary(mark_mod) ## ## Call: ## lm(formula = sales ~ advert, data = marketing) ## ## Residuals: ## Min 1Q Median 3Q Max ## -2.7845 -1.4762 -0.5103 1.2361 3.1869 ## ## Coefficients: ## Estimate Std. Error t value Pr(&gt;|t|) ## (Intercept) 9.4502 0.6806 13.88 2.13e-11 *** ## advert 1.1918 0.0937 12.72 9.65e-11 *** ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 ## ## Residual standard error: 1.907 on 19 degrees of freedom ## Multiple R-squared: 0.8949, Adjusted R-squared: 0.8894 ## F-statistic: 161.8 on 1 and 19 DF, p-value: 9.646e-11 While the SLR model is significant, the fitted versus residuals plot would have a very clear pattern. mark_mod_poly2 = lm(sales ~ advert + I(advert ^ 2), data = marketing) summary(mark_mod_poly2) ## ## Call: ## lm(formula = sales ~ advert + I(advert^2), data = marketing) ## ## Residuals: ## Min 1Q Median 3Q Max ## -1.9175 -0.8333 -0.1948 0.9292 2.1385 ## ## Coefficients: ## Estimate Std. Error t value Pr(&gt;|t|) ## (Intercept) 6.76161 0.67219 10.059 8.16e-09 *** ## advert 2.46231 0.24830 9.917 1.02e-08 *** ## I(advert^2) -0.08745 0.01658 -5.275 5.14e-05 *** ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 ## ## Residual standard error: 1.228 on 18 degrees of freedom ## Multiple R-squared: 0.9587, Adjusted R-squared: 0.9541 ## F-statistic: 209 on 2 and 18 DF, p-value: 3.486e-13 To add the second order term we need to use the I() function in the model specification around our newly created predictor. We see that with the first order term in the model, the quadratic term is also significant. n = length(marketing$advert) X = cbind(rep(1, n), marketing$advert, marketing$advert ^ 2) t(X) %*% X ## [,1] [,2] [,3] ## [1,] 21.00 120.70 1107.95 ## [2,] 120.70 1107.95 12385.86 ## [3,] 1107.95 12385.86 151369.12 solve(t(X) %*% X) %*% t(X) %*% marketing$sales ## [,1] ## [1,] 6.76161045 ## [2,] 2.46230964 ## [3,] -0.08745394 Here we verify the parameter estimates were found as we would expect. We could also add higher order terms, such as a third degree predictor. This is easy to do. Our \\(X\\) matrix simply becomes larger again. \\[ Y_i = \\beta_0 + \\beta_1 x_i + \\beta_2 x_i^2 + \\beta_3 x_i^3 + \\epsilon_i \\] \\[ \\begin{bmatrix} 1 &amp; x_1 &amp; x_1^2 &amp; x_1^3 \\\\[3pt] 1 &amp; x_2 &amp; x_2^2 &amp; x_2^3 \\\\[3pt] 1 &amp; x_3 &amp; x_3^2 &amp; x_3^3 \\\\[3pt] \\vdots &amp; \\vdots &amp; \\vdots &amp; \\vdots \\\\[3pt] 1 &amp; x_{n} &amp; x_{n}^2 &amp; x_{n}^3 \\\\ \\end{bmatrix} \\] mark_mod_poly3 = lm(sales ~ advert + I(advert ^ 2) + I(advert ^ 3), data = marketing) summary(mark_mod_poly3) ## ## Call: ## lm(formula = sales ~ advert + I(advert^2) + I(advert^3), data = marketing) ## ## Residuals: ## Min 1Q Median 3Q Max ## -1.44322 -0.61310 -0.01527 0.68131 1.22517 ## ## Coefficients: ## Estimate Std. Error t value Pr(&gt;|t|) ## (Intercept) 3.890070 0.761956 5.105 8.79e-05 *** ## advert 4.681864 0.501032 9.344 4.14e-08 *** ## I(advert^2) -0.455152 0.078977 -5.763 2.30e-05 *** ## I(advert^3) 0.016131 0.003429 4.704 0.000205 *** ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 ## ## Residual standard error: 0.8329 on 17 degrees of freedom ## Multiple R-squared: 0.9821, Adjusted R-squared: 0.9789 ## F-statistic: 310.2 on 3 and 17 DF, p-value: 4.892e-15 Now we see that with the first and second order terms in the model, the third order term is also significant. But does this make sense practically? The following plot should gives hints as to why it doesn’t. (The model with the third order term doesn’t have diminishing returns!) plot(sales ~ advert, data = marketing, xlab = &quot;Advert Spending (in $10,000)&quot;, ylab = &quot;Sales (in $10,000)&quot;, pch = 20, cex = 2) abline(mark_mod, lty = 2, col = &quot;green&quot;, lwd = 2) xplot = seq(0, 16, by = 0.01) lines(xplot, predict(mark_mod_poly2, newdata = data.frame(advert = xplot)), col = &quot;blue&quot;, lwd = 2) lines(xplot, predict(mark_mod_poly3, newdata = data.frame(advert = xplot)), col = &quot;red&quot;, lty = 3, lwd = 3) The previous plot was made using base graphics in R. The next plot was made using the package ggplot2, an increasingly popular plotting method in R. library(ggplot2) ggplot(data = marketing, aes(x = advert, y = sales)) + stat_smooth(method = &quot;lm&quot;, se = FALSE, color = &quot;green&quot;, formula = y ~ x) + stat_smooth(method = &quot;lm&quot;, se = FALSE, color = &quot;blue&quot;, formula = y ~ x + I(x ^ 2)) + stat_smooth(method = &quot;lm&quot;, se = FALSE, color = &quot;red&quot;, formula = y ~ x + I(x ^ 2)+ I(x ^ 3)) + geom_point(colour = &quot;black&quot;, size = 3) Note we could fit a polynomial of an arbitrary order, \\[ Y_i = \\beta_0 + \\beta_1 x_i + \\beta_2 x_i^2 + \\cdots + \\beta_{p-1}x_i^{p-1} + \\epsilon_i \\] However, we should be careful about over-fitting, since with a polynomial of degree one less than the number of observations, it is sometimes possible to fit a model perfectly. set.seed(1234) x = seq(0, 10) y = 3 + x + 4 * x ^ 2 + rnorm(11, 0, 20) plot(x, y, ylim = c(-300, 400), cex = 2, pch = 20) fit = lm(y ~ x + I(x ^ 2)) #summary(fit) fit_perf = lm(y ~ x + I(x ^ 2) + I(x ^ 3) + I(x ^ 4) + I(x ^ 5) + I(x ^ 6) + I(x ^ 7) + I(x ^ 8) + I(x ^ 9) + I(x ^ 10)) summary(fit_perf) ## ## Call: ## lm(formula = y ~ x + I(x^2) + I(x^3) + I(x^4) + I(x^5) + I(x^6) + ## I(x^7) + I(x^8) + I(x^9) + I(x^10)) ## ## Residuals: ## ALL 11 residuals are 0: no residual degrees of freedom! ## ## Coefficients: ## Estimate Std. Error t value Pr(&gt;|t|) ## (Intercept) -2.114e+01 NaN NaN NaN ## x -1.918e+03 NaN NaN NaN ## I(x^2) 4.969e+03 NaN NaN NaN ## I(x^3) -4.932e+03 NaN NaN NaN ## I(x^4) 2.581e+03 NaN NaN NaN ## I(x^5) -8.035e+02 NaN NaN NaN ## I(x^6) 1.570e+02 NaN NaN NaN ## I(x^7) -1.947e+01 NaN NaN NaN ## I(x^8) 1.490e+00 NaN NaN NaN ## I(x^9) -6.424e-02 NaN NaN NaN ## I(x^10) 1.195e-03 NaN NaN NaN ## ## Residual standard error: NaN on 0 degrees of freedom ## Multiple R-squared: 1, Adjusted R-squared: NaN ## F-statistic: NaN on 10 and 0 DF, p-value: NA xplot = seq(0, 10, by = 0.1) lines(xplot, predict(fit, newdata = data.frame(x = xplot)), col = &quot;dodgerblue&quot;, lwd = 2, lty = 1) lines(xplot, predict(fit_perf, newdata = data.frame(x = xplot)), col = &quot;darkorange&quot;, lwd = 2, lty = 2) Notice in the summary, R could not calculate standard errors. This is a result of being “out” of degrees of freedom. With 11 \\(\\beta\\) parameters and 11 data points, we use up all the degrees of freedom before we can estimate \\(\\sigma\\). In this example, the true relationship is quadratic, but the order 10 polynomial’s fit is “perfect”. Next chapter we will focus on the trade-off between goodness of fit (minimizing errors) and complexity of model. Suppose you work for an automobile manufacturer which makes a large luxury sedan. You would like to know how the car performs from a fuel efficiency standpoint when it is driven at various speeds. Instead of testing the car at every conceivable speed (which would be impossible) you create an experiment where the car is driven at speeds of interest in increments of 5 miles per hour. Our goal then, is to fit a model to this data in order to be able to predict fuel efficiency when driving at certain speeds. The data from this example can be found in fuel_econ.csv. econ = read.csv(&quot;data/fuel_econ.csv&quot;) In this example, we will be frequently looking at the fitted versus residuals plot, so we should write a function to make our life easier, but this is left as an exercise for homework. We will also be adding fitted curves to scatterplots repeatedly, so smartly we will write a function to do so. plot_econ_curve = function(model) { plot(mpg ~ mph, data = econ, xlab = &quot;Speed (Miles per Hour)&quot;, ylab = &quot;Fuel Efficiency (Miles per Gallon)&quot;, col = &quot;dodgerblue&quot;, pch = 20, cex = 2) xplot = seq(10, 75, by = 0.1) lines(xplot, predict(model, newdata = data.frame(mph = xplot)), col = &quot;darkorange&quot;, lwd = 2, lty = 1) } So now we first fit a simple linear regression to this data. fit1 = lm(mpg ~ mph, data = econ) par(mfrow = c(1, 2)) plot_econ_curve(fit1) plot(fitted(fit1), resid(fit1), xlab = &quot;Fitted&quot;, ylab = &quot;Residuals&quot;, col = &quot;dodgerblue&quot;, pch = 20, cex = 2) abline(h = 0, col = &quot;darkorange&quot;, lwd = 2) Pretty clearly we can do better. Yes fuel efficiency does increase as speed increases, but only up to a certain point. We will now add polynomial terms until we fit a suitable fit. fit2 = lm(mpg ~ mph + I(mph ^ 2), data = econ) summary(fit2) ## ## Call: ## lm(formula = mpg ~ mph + I(mph^2), data = econ) ## ## Residuals: ## Min 1Q Median 3Q Max ## -2.8411 -0.9694 0.0017 1.0181 3.3900 ## ## Coefficients: ## Estimate Std. Error t value Pr(&gt;|t|) ## (Intercept) 2.4444505 1.4241091 1.716 0.0984 . ## mph 1.2716937 0.0757321 16.792 3.99e-15 *** ## I(mph^2) -0.0145014 0.0008719 -16.633 4.97e-15 *** ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 ## ## Residual standard error: 1.663 on 25 degrees of freedom ## Multiple R-squared: 0.9188, Adjusted R-squared: 0.9123 ## F-statistic: 141.5 on 2 and 25 DF, p-value: 2.338e-14 par(mfrow = c(1, 2)) plot_econ_curve(fit2) plot(fitted(fit2), resid(fit2), xlab = &quot;Fitted&quot;, ylab = &quot;Residuals&quot;, col = &quot;dodgerblue&quot;, pch = 20, cex = 2) abline(h = 0, col = &quot;darkorange&quot;, lwd = 2) While this model clearly fits much better, and the second order term is significant, we still see a pattern in the fitted versus residuals plot which suggests higher order terms will help. Also, we would expect the curve to flatten as speed increases or decreases, not go sharply downward as we see here. fit3 = lm(mpg ~ mph + I(mph ^ 2) + I(mph ^ 3), data = econ) summary(fit3) ## ## Call: ## lm(formula = mpg ~ mph + I(mph^2) + I(mph^3), data = econ) ## ## Residuals: ## Min 1Q Median 3Q Max ## -2.8112 -0.9677 0.0264 1.0345 3.3827 ## ## Coefficients: ## Estimate Std. Error t value Pr(&gt;|t|) ## (Intercept) 2.258e+00 2.768e+00 0.816 0.4227 ## mph 1.291e+00 2.529e-01 5.103 3.2e-05 *** ## I(mph^2) -1.502e-02 6.604e-03 -2.274 0.0322 * ## I(mph^3) 4.066e-06 5.132e-05 0.079 0.9375 ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 ## ## Residual standard error: 1.697 on 24 degrees of freedom ## Multiple R-squared: 0.9188, Adjusted R-squared: 0.9087 ## F-statistic: 90.56 on 3 and 24 DF, p-value: 3.17e-13 par(mfrow = c(1, 2)) plot_econ_curve(fit3) plot(fitted(fit3), resid(fit3), xlab = &quot;Fitted&quot;, ylab = &quot;Residuals&quot;, col = &quot;dodgerblue&quot;, pch = 20, cex = 2) abline(h = 0, col = &quot;darkorange&quot;, lwd = 2) Adding the third order term doesn’t seem to help at all. The fitted curve hardly changes. This makes sense, since what we would like is for the curve to flatten at the extremes. For this we will need an even degree polynomial term. fit4 = lm(mpg ~ mph + I(mph ^ 2) + I(mph ^ 3) + I(mph ^ 4), data = econ) summary(fit4) ## ## Call: ## lm(formula = mpg ~ mph + I(mph^2) + I(mph^3) + I(mph^4), data = econ) ## ## Residuals: ## Min 1Q Median 3Q Max ## -1.57410 -0.60308 0.04236 0.74481 1.93038 ## ## Coefficients: ## Estimate Std. Error t value Pr(&gt;|t|) ## (Intercept) 2.146e+01 2.965e+00 7.238 2.28e-07 *** ## mph -1.468e+00 3.913e-01 -3.751 0.00104 ** ## I(mph^2) 1.081e-01 1.673e-02 6.463 1.35e-06 *** ## I(mph^3) -2.130e-03 2.844e-04 -7.488 1.31e-07 *** ## I(mph^4) 1.255e-05 1.665e-06 7.539 1.17e-07 *** ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 ## ## Residual standard error: 0.9307 on 23 degrees of freedom ## Multiple R-squared: 0.9766, Adjusted R-squared: 0.9726 ## F-statistic: 240.2 on 4 and 23 DF, p-value: &lt; 2.2e-16 par(mfrow = c(1, 2)) plot_econ_curve(fit4) plot(fitted(fit4), resid(fit4), xlab = &quot;Fitted&quot;, ylab = &quot;Residuals&quot;, col = &quot;dodgerblue&quot;, pch = 20, cex = 2) abline(h = 0, col = &quot;darkorange&quot;, lwd = 2) Now we are making progress. The fourth order term is significant with the other terms in the model. Also we are starting to see what we expected for low and high speed. However, there still seems to be a bit of a pattern in the residuals, so we will again try more higher order terms. We will add the fifth and sixth together, since adding the fifth will be similar to adding the third. fit6 = lm(mpg ~ mph + I(mph ^ 2) + I(mph ^ 3) + I(mph ^ 4) + I(mph ^ 5) + I(mph^6), data = econ) summary(fit6) ## ## Call: ## lm(formula = mpg ~ mph + I(mph^2) + I(mph^3) + I(mph^4) + I(mph^5) + ## I(mph^6), data = econ) ## ## Residuals: ## Min 1Q Median 3Q Max ## -1.1129 -0.5717 -0.1707 0.5026 1.5288 ## ## Coefficients: ## Estimate Std. Error t value Pr(&gt;|t|) ## (Intercept) -4.206e+00 1.204e+01 -0.349 0.7304 ## mph 4.203e+00 2.553e+00 1.646 0.1146 ## I(mph^2) -3.521e-01 2.012e-01 -1.750 0.0947 . ## I(mph^3) 1.579e-02 7.691e-03 2.053 0.0527 . ## I(mph^4) -3.473e-04 1.529e-04 -2.271 0.0338 * ## I(mph^5) 3.585e-06 1.518e-06 2.362 0.0279 * ## I(mph^6) -1.402e-08 5.941e-09 -2.360 0.0280 * ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 ## ## Residual standard error: 0.8657 on 21 degrees of freedom ## Multiple R-squared: 0.9815, Adjusted R-squared: 0.9762 ## F-statistic: 186 on 6 and 21 DF, p-value: &lt; 2.2e-16 par(mfrow = c(1, 2)) plot_econ_curve(fit6) plot(fitted(fit6), resid(fit6), xlab = &quot;Fitted&quot;, ylab = &quot;Residuals&quot;, col = &quot;dodgerblue&quot;, pch = 20, cex = 2) abline(h = 0, col = &quot;darkorange&quot;, lwd = 2) Again the sixth order term is significant with the other terms in the model and here we see less pattern in the residuals plot. Let’s now test which of the previous two models we prefer. We will test \\[ H_0: \\beta_5 = \\beta_6 = 0. \\] anova(fit4, fit6) ## Analysis of Variance Table ## ## Model 1: mpg ~ mph + I(mph^2) + I(mph^3) + I(mph^4) ## Model 2: mpg ~ mph + I(mph^2) + I(mph^3) + I(mph^4) + I(mph^5) + I(mph^6) ## Res.Df RSS Df Sum of Sq F Pr(&gt;F) ## 1 23 19.922 ## 2 21 15.739 2 4.1828 2.7905 0.0842 . ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 So, this test does not reject the null hypothesis at a level of significance of \\(\\alpha = 0.05\\), however the p-value is still rather small, and the fitted versus residuals plot is much better for the model with the sixth order term. This makes the sixth order model a good choice. We could repeat this process one more time. fit8 = lm(mpg ~ mph + I(mph ^ 2) + I(mph ^ 3) + I(mph ^ 4) + I(mph ^ 5) + I(mph ^ 6) + I(mph ^ 7) + I(mph ^ 8), data = econ) summary(fit8) ## ## Call: ## lm(formula = mpg ~ mph + I(mph^2) + I(mph^3) + I(mph^4) + I(mph^5) + ## I(mph^6) + I(mph^7) + I(mph^8), data = econ) ## ## Residuals: ## Min 1Q Median 3Q Max ## -1.21938 -0.50464 -0.09105 0.49029 1.45440 ## ## Coefficients: ## Estimate Std. Error t value Pr(&gt;|t|) ## (Intercept) -1.202e+01 7.045e+01 -0.171 0.866 ## mph 6.021e+00 2.014e+01 0.299 0.768 ## I(mph^2) -5.037e-01 2.313e+00 -0.218 0.830 ## I(mph^3) 2.121e-02 1.408e-01 0.151 0.882 ## I(mph^4) -4.008e-04 5.017e-03 -0.080 0.937 ## I(mph^5) 1.789e-06 1.080e-04 0.017 0.987 ## I(mph^6) 4.486e-08 1.381e-06 0.032 0.974 ## I(mph^7) -6.456e-10 9.649e-09 -0.067 0.947 ## I(mph^8) 2.530e-12 2.835e-11 0.089 0.930 ## ## Residual standard error: 0.9034 on 19 degrees of freedom ## Multiple R-squared: 0.9818, Adjusted R-squared: 0.9741 ## F-statistic: 128.1 on 8 and 19 DF, p-value: 7.074e-15 par(mfrow = c(1, 2)) plot_econ_curve(fit8) plot(fitted(fit8), resid(fit8), xlab = &quot;Fitted&quot;, ylab = &quot;Residuals&quot;, col = &quot;dodgerblue&quot;, pch = 20, cex = 2) abline(h = 0, col = &quot;darkorange&quot;, lwd = 2) summary(fit8) ## ## Call: ## lm(formula = mpg ~ mph + I(mph^2) + I(mph^3) + I(mph^4) + I(mph^5) + ## I(mph^6) + I(mph^7) + I(mph^8), data = econ) ## ## Residuals: ## Min 1Q Median 3Q Max ## -1.21938 -0.50464 -0.09105 0.49029 1.45440 ## ## Coefficients: ## Estimate Std. Error t value Pr(&gt;|t|) ## (Intercept) -1.202e+01 7.045e+01 -0.171 0.866 ## mph 6.021e+00 2.014e+01 0.299 0.768 ## I(mph^2) -5.037e-01 2.313e+00 -0.218 0.830 ## I(mph^3) 2.121e-02 1.408e-01 0.151 0.882 ## I(mph^4) -4.008e-04 5.017e-03 -0.080 0.937 ## I(mph^5) 1.789e-06 1.080e-04 0.017 0.987 ## I(mph^6) 4.486e-08 1.381e-06 0.032 0.974 ## I(mph^7) -6.456e-10 9.649e-09 -0.067 0.947 ## I(mph^8) 2.530e-12 2.835e-11 0.089 0.930 ## ## Residual standard error: 0.9034 on 19 degrees of freedom ## Multiple R-squared: 0.9818, Adjusted R-squared: 0.9741 ## F-statistic: 128.1 on 8 and 19 DF, p-value: 7.074e-15 anova(fit6, fit8) ## Analysis of Variance Table ## ## Model 1: mpg ~ mph + I(mph^2) + I(mph^3) + I(mph^4) + I(mph^5) + I(mph^6) ## Model 2: mpg ~ mph + I(mph^2) + I(mph^3) + I(mph^4) + I(mph^5) + I(mph^6) + ## I(mph^7) + I(mph^8) ## Res.Df RSS Df Sum of Sq F Pr(&gt;F) ## 1 21 15.739 ## 2 19 15.506 2 0.2324 0.1424 0.8682 Here we would clearly stick with fit6. The eighth order term is not significant with the other terms in the model and the F-test does not reject. As an aside, be aware that there is a quicker way to specify a model with many higher order terms. fit6_alt = lm(mpg ~ poly(mph, 6), data = econ) all.equal(fitted(fit6), fitted(fit6_alt)) ## [1] TRUE We first verify that this method produces the same fitted values. However, the estimated coefficients are different. coef(fit6) ## (Intercept) mph I(mph^2) I(mph^3) I(mph^4) ## -4.206224e+00 4.203382e+00 -3.521452e-01 1.579340e-02 -3.472665e-04 ## I(mph^5) I(mph^6) ## 3.585201e-06 -1.401995e-08 coef(fit6_alt) ## (Intercept) poly(mph, 6)1 poly(mph, 6)2 poly(mph, 6)3 poly(mph, 6)4 ## 24.40714286 4.16769628 -27.66685755 0.13446747 7.01671480 ## poly(mph, 6)5 poly(mph, 6)6 ## 0.09288754 -2.04307796 This is because poly() uses orthogonal polynomials, which solves an issue we will discuss in the next chapter. summary(fit6) ## ## Call: ## lm(formula = mpg ~ mph + I(mph^2) + I(mph^3) + I(mph^4) + I(mph^5) + ## I(mph^6), data = econ) ## ## Residuals: ## Min 1Q Median 3Q Max ## -1.1129 -0.5717 -0.1707 0.5026 1.5288 ## ## Coefficients: ## Estimate Std. Error t value Pr(&gt;|t|) ## (Intercept) -4.206e+00 1.204e+01 -0.349 0.7304 ## mph 4.203e+00 2.553e+00 1.646 0.1146 ## I(mph^2) -3.521e-01 2.012e-01 -1.750 0.0947 . ## I(mph^3) 1.579e-02 7.691e-03 2.053 0.0527 . ## I(mph^4) -3.473e-04 1.529e-04 -2.271 0.0338 * ## I(mph^5) 3.585e-06 1.518e-06 2.362 0.0279 * ## I(mph^6) -1.402e-08 5.941e-09 -2.360 0.0280 * ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 ## ## Residual standard error: 0.8657 on 21 degrees of freedom ## Multiple R-squared: 0.9815, Adjusted R-squared: 0.9762 ## F-statistic: 186 on 6 and 21 DF, p-value: &lt; 2.2e-16 summary(fit6_alt) ## ## Call: ## lm(formula = mpg ~ poly(mph, 6), data = econ) ## ## Residuals: ## Min 1Q Median 3Q Max ## -1.1129 -0.5717 -0.1707 0.5026 1.5288 ## ## Coefficients: ## Estimate Std. Error t value Pr(&gt;|t|) ## (Intercept) 24.40714 0.16360 149.184 &lt; 2e-16 *** ## poly(mph, 6)1 4.16770 0.86571 4.814 9.31e-05 *** ## poly(mph, 6)2 -27.66686 0.86571 -31.958 &lt; 2e-16 *** ## poly(mph, 6)3 0.13447 0.86571 0.155 0.878 ## poly(mph, 6)4 7.01671 0.86571 8.105 6.68e-08 *** ## poly(mph, 6)5 0.09289 0.86571 0.107 0.916 ## poly(mph, 6)6 -2.04308 0.86571 -2.360 0.028 * ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 ## ## Residual standard error: 0.8657 on 21 degrees of freedom ## Multiple R-squared: 0.9815, Adjusted R-squared: 0.9762 ## F-statistic: 186 on 6 and 21 DF, p-value: &lt; 2.2e-16 Notice though that the p-value for testing the degree 6 term is the same. Because of this, for the most part we can use these interchangeably. To use poly() to obtain the same results as using I() repeatedly, we would need to set raw = TRUE. fit6_alt2 = lm(mpg ~ poly(mph, 6, raw = TRUE), data = econ) coef(fit6_alt2) ## (Intercept) poly(mph, 6, raw = TRUE)1 poly(mph, 6, raw = TRUE)2 ## -4.206224e+00 4.203382e+00 -3.521452e-01 ## poly(mph, 6, raw = TRUE)3 poly(mph, 6, raw = TRUE)4 poly(mph, 6, raw = TRUE)5 ## 1.579340e-02 -3.472665e-04 3.585201e-06 ## poly(mph, 6, raw = TRUE)6 ## -1.401995e-08 We’ve now seen how to transform predictor and response variables. In this chapter we have mostly focused on using this in the context of fixing SLR models. However, these concepts can easily be used together with categorical variables and interactions to build larger, more flexible models. In the next chapter, we will discuss how to choose a good model from a collection of possible models. Material below here is currently being merged into the content above. Response Transformations initech = read.csv(&quot;data/initech.csv&quot;) plot(salary ~ years, data = initech, col = &quot;grey&quot;, pch = 20, cex = 1.5, main = &quot;Salaries at Initech, By Seniority&quot;) initech_fit = lm(salary ~ years, data = initech) summary(initech_fit) ## ## Call: ## lm(formula = salary ~ years, data = initech) ## ## Residuals: ## Min 1Q Median 3Q Max ## -57225 -18104 241 15589 91332 ## ## Coefficients: ## Estimate Std. Error t value Pr(&gt;|t|) ## (Intercept) 5302 5750 0.922 0.359 ## years 8637 389 22.200 &lt;2e-16 *** ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 ## ## Residual standard error: 27360 on 98 degrees of freedom ## Multiple R-squared: 0.8341, Adjusted R-squared: 0.8324 ## F-statistic: 492.8 on 1 and 98 DF, p-value: &lt; 2.2e-16 plot(salary ~ years, data = initech, col = &quot;grey&quot;, pch = 20, cex = 1.5, main = &quot;Salaries at Initech, By Seniority&quot;) abline(initech_fit, col = &quot;darkorange&quot;, lwd = 2) par(mfrow = c(1, 2)) plot(fitted(initech_fit), resid(initech_fit), col = &quot;grey&quot;, pch = 20, xlab = &quot;Fitted&quot;, ylab = &quot;Residuals&quot;, main = &quot;Fitted versus Residuals&quot;) abline(h = 0, col = &quot;darkorange&quot;, lwd = 2) qqnorm(resid(initech_fit), main = &quot;Normal Q-Q Plot&quot;, col = &quot;darkgrey&quot;) qqline(resid(initech_fit), col = &quot;dodgerblue&quot;, lwd = 2) initech_fit_log = lm(log(salary) ~ years, data = initech) \\[ \\log(Y_i) = \\beta_0 + \\beta_1 x_i + \\epsilon_i \\] plot(log(salary) ~ years, data = initech, col = &quot;grey&quot;, pch = 20, cex = 1.5, main = &quot;Salaries at Initech, By Seniority&quot;) abline(initech_fit_log, col = &quot;darkorange&quot;, lwd = 2) \\[ Y_i = \\exp(\\beta_0 + \\beta_1 x_i) \\cdot \\exp(\\epsilon_i) \\] plot(salary ~ years, data = initech, col = &quot;grey&quot;, pch = 20, cex = 1.5, main = &quot;Salaries at Initech, By Seniority&quot;) curve(exp(initech_fit_log$coef[1] + initech_fit_log$coef[2] * x), from = 0, to = 30, add = TRUE, col = &quot;darkorange&quot;, lwd = 2) par(mfrow = c(1, 2)) plot(fitted(initech_fit_log), resid(initech_fit_log), col = &quot;grey&quot;, pch = 20, xlab = &quot;Fitted&quot;, ylab = &quot;Residuals&quot;, main = &quot;Fitted versus Residuals&quot;) abline(h = 0, col = &quot;darkorange&quot;, lwd = 2) qqnorm(resid(initech_fit_log), main = &quot;Normal Q-Q Plot&quot;, col = &quot;darkgrey&quot;) qqline(resid(initech_fit_log), col = &quot;dodgerblue&quot;, lwd = 2) sqrt(mean(resid(initech_fit) ^ 2)) ## [1] 27080.16 sqrt(mean(resid(initech_fit_log) ^ 2)) ## [1] 0.1934907 sqrt(mean((initech$salary - fitted(initech_fit)) ^ 2)) ## [1] 27080.16 sqrt(mean((initech$salary - exp(fitted(initech_fit_log))) ^ 2)) ## [1] 24280.36 Predictor Transformations 14.2.2 A Quadratic Model sim_quad = function(sample_size = 500) { x = runif(n = sample_size) * 5 y = 3 + 5 * x ^ 2 + rnorm(n = sample_size, mean = 0, sd = 5) data.frame(x, y) } set.seed(314) quad_data = sim_quad(sample_size = 200) lin_fit = lm(y ~ x, data = quad_data) summary(lin_fit) ## ## Call: ## lm(formula = y ~ x, data = quad_data) ## ## Residuals: ## Min 1Q Median 3Q Max ## -20.363 -7.550 -3.416 8.472 26.181 ## ## Coefficients: ## Estimate Std. Error t value Pr(&gt;|t|) ## (Intercept) -18.3271 1.5494 -11.83 &lt;2e-16 *** ## x 24.8716 0.5343 46.55 &lt;2e-16 *** ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 ## ## Residual standard error: 10.79 on 198 degrees of freedom ## Multiple R-squared: 0.9163, Adjusted R-squared: 0.9158 ## F-statistic: 2167 on 1 and 198 DF, p-value: &lt; 2.2e-16 plot(y ~ x, data = quad_data, col = &quot;grey&quot;, pch = 20, cex = 1.5, main = &quot;Simulated Quadratic Data&quot;) abline(lin_fit, col = &quot;darkorange&quot;, lwd = 2) par(mfrow = c(1, 2)) plot(fitted(lin_fit), resid(lin_fit), col = &quot;grey&quot;, pch = 20, xlab = &quot;Fitted&quot;, ylab = &quot;Residuals&quot;, main = &quot;Fitted versus Residuals&quot;) abline(h = 0, col = &quot;darkorange&quot;, lwd = 2) qqnorm(resid(lin_fit), main = &quot;Normal Q-Q Plot&quot;, col = &quot;darkgrey&quot;) qqline(resid(lin_fit), col = &quot;dodgerblue&quot;, lwd = 2) \\[ Y_i = \\beta_0 + \\beta_1 x_i + \\beta_2 x_i^2 + \\epsilon_i \\] quad_fit = lm(y ~ x + I(x^2), data = quad_data) summary(quad_fit) ## ## Call: ## lm(formula = y ~ x + I(x^2), data = quad_data) ## ## Residuals: ## Min 1Q Median 3Q Max ## -11.4167 -3.0581 0.2297 3.1024 12.1256 ## ## Coefficients: ## Estimate Std. Error t value Pr(&gt;|t|) ## (Intercept) 3.0649 0.9577 3.200 0.0016 ** ## x -0.5108 0.8637 -0.591 0.5549 ## I(x^2) 5.0740 0.1667 30.433 &lt;2e-16 *** ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 ## ## Residual standard error: 4.531 on 197 degrees of freedom ## Multiple R-squared: 0.9853, Adjusted R-squared: 0.9852 ## F-statistic: 6608 on 2 and 197 DF, p-value: &lt; 2.2e-16 plot(y ~ x, data = quad_data, col = &quot;grey&quot;, pch = 20, cex = 1.5, main = &quot;Simulated Quadratic Data&quot;) curve(quad_fit$coef[1] + quad_fit$coef[2] * x + quad_fit$coef[3] * x ^ 2, from = -5, to = 30, add = TRUE, col = &quot;darkorange&quot;, lwd = 2) par(mfrow = c(1, 2)) plot(fitted(quad_fit), resid(quad_fit), col = &quot;grey&quot;, pch = 20, xlab = &quot;Fitted&quot;, ylab = &quot;Residuals&quot;, main = &quot;Fitted versus Residuals&quot;) abline(h = 0, col = &quot;darkorange&quot;, lwd = 2) qqnorm(resid(quad_fit), main = &quot;Normal Q-Q Plot&quot;, col = &quot;darkgrey&quot;) qqline(resid(quad_fit), col = &quot;dodgerblue&quot;, lwd = 2) 14.2.3 Overfitting and Extrapolation sim_for_perf = function() { x = seq(0, 10) y = 3 + x - 4 * x ^ 2 + rnorm(n = 11, mean = 0, sd = 25) data.frame(x, y) } set.seed(1234) data_for_perf = sim_for_perf() fit_correct = lm(y ~ x + I(x ^ 2), data = data_for_perf) fit_perfect = lm(y ~ x + I(x ^ 2) + I(x ^ 3) + I(x ^ 4) + I(x ^ 5) + I(x ^ 6) + I(x ^ 7) + I(x ^ 8) + I(x ^ 9) + I(x ^ 10), data = data_for_perf) x_plot = seq(-5, 15, by = 0.1) plot(y ~ x, data = data_for_perf, ylim = c(-450, 100), cex = 2, pch = 20) lines(x_plot, predict(fit_correct, newdata = data.frame(x = x_plot)), col = &quot;dodgerblue&quot;, lwd = 2, lty = 1) lines(x_plot, predict(fit_perfect, newdata = data.frame(x = x_plot)), col = &quot;darkorange&quot;, lwd = 2, lty = 2) 14.2.4 Comparing Polynomial Models sim_higher = function(sample_size = 250) { x = runif(n = sample_size, min = -1, max = 1) * 2 y = 3 + -6 * x ^ 2 + 1 * x ^ 4 + rnorm(n = sample_size, mean = 0, sd = 3) data.frame(x, y) } \\[ Y_i = \\beta_0 + \\beta_1 x_i + \\beta_2 x_i^2 + \\epsilon_i \\] \\[ Y_i = \\beta_0 + \\beta_1 x_i + \\beta_2 x_i^2 + \\beta_3 x_i^3 + \\beta_4 x_i^4 + \\epsilon_i \\] \\[ Y_i = \\beta_0 + \\beta_1 x_i + \\beta_2 x_i^2 + \\beta_3 x_i^3 + \\beta_4 x_i^4 + \\beta_5 x_i^5 + \\beta_6 x_i^6 + \\epsilon_i \\] set.seed(42) data_higher = sim_higher() plot(y ~ x, data = data_higher, col = &quot;grey&quot;, pch = 20, cex = 1.5, main = &quot;Simulated Quartic Data&quot;) fit_2 = lm(y ~ poly(x, 2), data = data_higher) fit_4 = lm(y ~ poly(x, 4), data = data_higher) plot(y ~ x, data = data_higher, col = &quot;grey&quot;, pch = 20, cex = 1.5, main = &quot;Simulated Quartic Data&quot;) x_plot = seq(-5, 5, by = 0.05) lines(x_plot, predict(fit_2, newdata = data.frame(x = x_plot)), col = &quot;dodgerblue&quot;, lwd = 2, lty = 1) lines(x_plot, predict(fit_4, newdata = data.frame(x = x_plot)), col = &quot;darkorange&quot;, lwd = 2, lty = 2) par(mfrow = c(1, 2)) plot(fitted(fit_2), resid(fit_2), col = &quot;grey&quot;, pch = 20, xlab = &quot;Fitted&quot;, ylab = &quot;Residuals&quot;, main = &quot;Fitted versus Residuals&quot;) abline(h = 0, col = &quot;darkorange&quot;, lwd = 2) qqnorm(resid(fit_2), main = &quot;Normal Q-Q Plot&quot;, col = &quot;darkgrey&quot;) qqline(resid(fit_2), col = &quot;dodgerblue&quot;, lwd = 2) par(mfrow = c(1, 2)) plot(fitted(fit_4), resid(fit_4), col = &quot;grey&quot;, pch = 20, xlab = &quot;Fitted&quot;, ylab = &quot;Residuals&quot;, main = &quot;Fitted versus Residuals&quot;) abline(h = 0, col = &quot;darkorange&quot;, lwd = 2) qqnorm(resid(fit_4), main = &quot;Normal Q-Q Plot&quot;, col = &quot;darkgrey&quot;) qqline(resid(fit_4), col = &quot;dodgerblue&quot;, lwd = 2) anova(fit_2, fit_4) ## Analysis of Variance Table ## ## Model 1: y ~ poly(x, 2) ## Model 2: y ~ poly(x, 4) ## Res.Df RSS Df Sum of Sq F Pr(&gt;F) ## 1 247 2334.1 ## 2 245 1912.6 2 421.51 26.997 2.536e-11 *** ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 fit_6 = lm(y ~ poly(x, 6), data = data_higher) anova(fit_4, fit_6) ## Analysis of Variance Table ## ## Model 1: y ~ poly(x, 4) ## Model 2: y ~ poly(x, 6) ## Res.Df RSS Df Sum of Sq F Pr(&gt;F) ## 1 245 1912.6 ## 2 243 1904.4 2 8.1889 0.5224 0.5937 14.2.5 poly() Function and Orthogonal Polynomials \\[ Y_i = \\beta_0 + \\beta_1 x_i + \\beta_2 x_i^2 + \\beta_3 x_i^3 + \\beta_4 x_i^4 + \\epsilon_i \\] fit_4a = lm(y ~ poly(x, degree = 4), data = data_higher) fit_4b = lm(y ~ poly(x, degree = 4, raw = TRUE), data = data_higher) fit_4c = lm(y ~ x + I(x^2) + I(x^3) + I(x^4), data = data_higher) coef(fit_4a) ## (Intercept) poly(x, degree = 4)1 poly(x, degree = 4)2 ## -1.980036 -2.053929 -49.344752 ## poly(x, degree = 4)3 poly(x, degree = 4)4 ## 0.669874 20.519759 coef(fit_4b) ## (Intercept) poly(x, degree = 4, raw = TRUE)1 ## 2.9996256 -0.3880250 ## poly(x, degree = 4, raw = TRUE)2 poly(x, degree = 4, raw = TRUE)3 ## -6.1511166 0.1269046 ## poly(x, degree = 4, raw = TRUE)4 ## 1.0282139 coef(fit_4c) ## (Intercept) x I(x^2) I(x^3) I(x^4) ## 2.9996256 -0.3880250 -6.1511166 0.1269046 1.0282139 unname(coef(fit_4a)) ## [1] -1.980036 -2.053929 -49.344752 0.669874 20.519759 unname(coef(fit_4b)) ## [1] 2.9996256 -0.3880250 -6.1511166 0.1269046 1.0282139 unname(coef(fit_4c)) ## [1] 2.9996256 -0.3880250 -6.1511166 0.1269046 1.0282139 all.equal(fitted(fit_4a), fitted(fit_4b)) ## [1] TRUE all.equal(resid(fit_4a), resid(fit_4b)) ## [1] TRUE summary(fit_4a) ## ## Call: ## lm(formula = y ~ poly(x, degree = 4), data = data_higher) ## ## Residuals: ## Min 1Q Median 3Q Max ## -7.6982 -2.0334 0.0042 1.9532 7.4626 ## ## Coefficients: ## Estimate Std. Error t value Pr(&gt;|t|) ## (Intercept) -1.9800 0.1767 -11.205 &lt; 2e-16 *** ## poly(x, degree = 4)1 -2.0539 2.7940 -0.735 0.463 ## poly(x, degree = 4)2 -49.3448 2.7940 -17.661 &lt; 2e-16 *** ## poly(x, degree = 4)3 0.6699 2.7940 0.240 0.811 ## poly(x, degree = 4)4 20.5198 2.7940 7.344 3.06e-12 *** ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 ## ## Residual standard error: 2.794 on 245 degrees of freedom ## Multiple R-squared: 0.5993, Adjusted R-squared: 0.5928 ## F-statistic: 91.61 on 4 and 245 DF, p-value: &lt; 2.2e-16 summary(fit_4c) ## ## Call: ## lm(formula = y ~ x + I(x^2) + I(x^3) + I(x^4), data = data_higher) ## ## Residuals: ## Min 1Q Median 3Q Max ## -7.6982 -2.0334 0.0042 1.9532 7.4626 ## ## Coefficients: ## Estimate Std. Error t value Pr(&gt;|t|) ## (Intercept) 2.9996 0.3315 9.048 &lt; 2e-16 *** ## x -0.3880 0.3828 -1.014 0.312 ## I(x^2) -6.1511 0.5049 -12.183 &lt; 2e-16 *** ## I(x^3) 0.1269 0.1456 0.871 0.384 ## I(x^4) 1.0282 0.1400 7.344 3.06e-12 *** ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 ## ## Residual standard error: 2.794 on 245 degrees of freedom ## Multiple R-squared: 0.5993, Adjusted R-squared: 0.5928 ## F-statistic: 91.61 on 4 and 245 DF, p-value: &lt; 2.2e-16 14.2.6 Inhibit Function coef(lm(y ~ x + x ^ 2, data = quad_data)) ## (Intercept) x ## -18.32715 24.87163 coef(lm(y ~ x + I(x ^ 2), data = quad_data)) ## (Intercept) x I(x^2) ## 3.0649446 -0.5108131 5.0739805 coef(lm(y ~ x + x:x, data = quad_data)) ## (Intercept) x ## -18.32715 24.87163 coef(lm(y ~ x * x, data = quad_data)) ## (Intercept) x ## -18.32715 24.87163 coef(lm(y ~ x ^ 2, data = quad_data)) ## (Intercept) x ## -18.32715 24.87163 coef(lm(y ~ x + x ^ 2, data = quad_data)) ## (Intercept) x ## -18.32715 24.87163 coef(lm(y ~ I(x + x), data = quad_data)) ## (Intercept) I(x + x) ## -18.32715 12.43582 coef(lm(y ~ x + x, data = quad_data)) ## (Intercept) x ## -18.32715 24.87163 14.2.7 Data Example pairs(autompg) mpg_hp = lm(mpg ~ hp, data = autompg) par(mfrow = c(1, 2)) plot(mpg ~ hp, data = autompg, col = &quot;dodgerblue&quot;, pch = 20, cex = 1.5) abline(mpg_hp, col = &quot;darkorange&quot;, lwd = 2) plot(fitted(mpg_hp), resid(mpg_hp), col = &quot;dodgerblue&quot;, pch = 20, cex = 1.5, xlab = &quot;Fitted&quot;, ylab = &quot;Residuals&quot;) abline(h = 0, lty = 2, col = &quot;darkorange&quot;, lwd = 2) mpg_hp_log = lm(mpg ~ hp + I(hp ^ 2), data = autompg) par(mfrow = c(1, 2)) plot(mpg ~ hp, data = autompg, col = &quot;dodgerblue&quot;, pch = 20, cex = 1.5) xplot = seq(min(autompg$hp), max(autompg$hp), by = 0.1) lines(xplot, predict(mpg_hp_log, newdata = data.frame(hp = xplot)), col = &quot;darkorange&quot;, lwd = 2, lty = 1) plot(fitted(mpg_hp_log), resid(mpg_hp_log), col = &quot;dodgerblue&quot;, pch = 20, cex = 1.5, xlab = &quot;Fitted&quot;, ylab = &quot;Residuals&quot;) abline(h = 0, lty = 2, col = &quot;darkorange&quot;, lwd = 2) mpg_hp_log = lm(log(mpg) ~ hp + I(hp ^ 2), data = autompg) par(mfrow = c(1, 2)) plot(log(mpg) ~ hp, data = autompg, col = &quot;dodgerblue&quot;, pch = 20, cex = 1.5) xplot = seq(min(autompg$hp), max(autompg$hp), by = 0.1) lines(xplot, predict(mpg_hp_log, newdata = data.frame(hp = xplot)), col = &quot;darkorange&quot;, lwd = 2, lty = 1) plot(fitted(mpg_hp_log), resid(mpg_hp_log), col = &quot;dodgerblue&quot;, pch = 20, cex = 1.5, xlab = &quot;Fitted&quot;, ylab = &quot;Residuals&quot;) abline(h = 0, lty = 2, col = &quot;darkorange&quot;, lwd = 2) mpg_hp_loglog = lm(log(mpg) ~ log(hp), data = autompg) par(mfrow = c(1, 2)) plot(log(mpg) ~ log(hp), data = autompg, col = &quot;dodgerblue&quot;, pch = 20, cex = 1.5) abline(mpg_hp_loglog, col = &quot;darkorange&quot;, lwd = 2) plot(fitted(mpg_hp_loglog), resid(mpg_hp_loglog), col = &quot;dodgerblue&quot;, pch = 20, cex = 1.5, xlab = &quot;Fitted&quot;, ylab = &quot;Residuals&quot;) abline(h = 0, lty = 2, col = &quot;darkorange&quot;, lwd = 2) big_model = lm(mpg ~ disp * hp * domestic, data = autompg) qqnorm(resid(big_model), col = &quot;darkgrey&quot;) qqline(resid(big_model), col = &quot;dodgerblue&quot;, lwd = 2) bigger_model = lm(log(mpg) ~ disp * hp * domestic + I(disp ^ 2) + I(hp ^ 2), data = autompg) summary(bigger_model) ## ## Call: ## lm(formula = log(mpg) ~ disp * hp * domestic + I(disp^2) + I(hp^2), ## data = autompg) ## ## Residuals: ## Min 1Q Median 3Q Max ## -0.40381 -0.08635 -0.01040 0.09995 0.71365 ## ## Coefficients: ## Estimate Std. Error t value Pr(&gt;|t|) ## (Intercept) 4.276e+00 2.564e-01 16.674 &lt;2e-16 *** ## disp -5.289e-03 2.565e-03 -2.062 0.0399 * ## hp -7.386e-03 3.309e-03 -2.232 0.0262 * ## domestic -2.496e-01 2.787e-01 -0.896 0.3710 ## I(disp^2) 8.552e-06 4.141e-06 2.065 0.0396 * ## I(hp^2) -1.565e-05 1.679e-05 -0.932 0.3519 ## disp:hp 2.685e-05 3.082e-05 0.871 0.3842 ## disp:domestic -1.101e-03 2.526e-03 -0.436 0.6631 ## hp:domestic 7.560e-03 3.689e-03 2.049 0.0411 * ## disp:hp:domestic -2.311e-05 2.662e-05 -0.868 0.3859 ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 ## ## Residual standard error: 0.1507 on 373 degrees of freedom ## Multiple R-squared: 0.8107, Adjusted R-squared: 0.8062 ## F-statistic: 177.5 on 9 and 373 DF, p-value: &lt; 2.2e-16 qqnorm(resid(bigger_model), col = &quot;darkgrey&quot;) qqline(resid(bigger_model), col = &quot;dodgerblue&quot;, lwd = 2) 14.3 R Markdown The R Markdown file for this chapter can be found here: transformations.Rmd The file was created using R version 4.4.1. "],["collinearity.html", "Chapter 15 Collinearity 15.1 Exact Collinearity 15.2 Collinearity 15.3 Simulation 15.4 R Markdown", " Chapter 15 Collinearity “If I look confused it is because I am thinking.” — Samuel Goldwyn After reading this chapter you will be able to: Identify collinearity in regression. Understand the effect of collinearity on regression models. 15.1 Exact Collinearity Let’s create a dataset where one of the predictors, \\(x_3\\), is a linear combination of the other predictors. gen_exact_collin_data = function(num_samples = 100) { x1 = rnorm(n = num_samples, mean = 80, sd = 10) x2 = rnorm(n = num_samples, mean = 70, sd = 5) x3 = 2 * x1 + 4 * x2 + 3 y = 3 + x1 + x2 + rnorm(n = num_samples, mean = 0, sd = 1) data.frame(y, x1, x2, x3) } Notice that the way we are generating this data, the response \\(y\\) only really depends on \\(x_1\\) and \\(x_2\\). set.seed(42) exact_collin_data = gen_exact_collin_data() head(exact_collin_data) ## y x1 x2 x3 ## 1 170.7135 93.70958 76.00483 494.4385 ## 2 152.9106 74.35302 75.22376 452.6011 ## 3 152.7866 83.63128 64.98396 430.1984 ## 4 170.6306 86.32863 79.24241 492.6269 ## 5 152.3320 84.04268 66.66613 437.7499 ## 6 151.3155 78.93875 70.52757 442.9878 What happens when we attempt to fit a regression model in R using all of the predictors? exact_collin_fit = lm(y ~ x1 + x2 + x3, data = exact_collin_data) summary(exact_collin_fit) ## ## Call: ## lm(formula = y ~ x1 + x2 + x3, data = exact_collin_data) ## ## Residuals: ## Min 1Q Median 3Q Max ## -2.57662 -0.66188 -0.08253 0.63706 2.52057 ## ## Coefficients: (1 not defined because of singularities) ## Estimate Std. Error t value Pr(&gt;|t|) ## (Intercept) 2.957336 1.735165 1.704 0.0915 . ## x1 0.985629 0.009788 100.702 &lt;2e-16 *** ## x2 1.017059 0.022545 45.112 &lt;2e-16 *** ## x3 NA NA NA NA ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 ## ## Residual standard error: 1.014 on 97 degrees of freedom ## Multiple R-squared: 0.9923, Adjusted R-squared: 0.9921 ## F-statistic: 6236 on 2 and 97 DF, p-value: &lt; 2.2e-16 We see that R simply decides to exclude a variable. Why is this happening? X = cbind(1, as.matrix(exact_collin_data[,-1])) solve(t(X) %*% X) If we attempt to find \\(\\boldsymbol{\\hat{\\beta}}\\) using \\(\\left( \\boldsymbol{X}^T \\boldsymbol{X} \\right)^{-1}\\), we see that this is not possible, due to the fact that the columns of \\(\\boldsymbol{X}\\) are linearly dependent. The previous lines of code were not run, because they produce an error! When this happens, we say there is exact collinearity in the dataset. As a result of this issue, R essentially chose to fit the model y ~ x1 + x2. However notice that two other models would accomplish exactly the same fit. fit1 = lm(y ~ x1 + x2, data = exact_collin_data) fit2 = lm(y ~ x1 + x3, data = exact_collin_data) fit3 = lm(y ~ x2 + x3, data = exact_collin_data) We see that the fitted values for each of the three models are exactly the same. This is a result of \\(x_3\\) containing all of the information from \\(x_1\\) and \\(x_2\\). As long as one of \\(x_1\\) or \\(x_2\\) is included in the model, \\(x_3\\) can be used to recover the information from the variable not included. all.equal(fitted(fit1), fitted(fit2)) ## [1] TRUE all.equal(fitted(fit2), fitted(fit3)) ## [1] TRUE While their fitted values are all the same, their estimated coefficients are wildly different. The sign of \\(x_2\\) is switched in two of the models! So only fit1 properly explains the relationship between the variables, fit2 and fit3 still predict as well as fit1, despite the coefficients having little to no meaning, a concept we will return to later. coef(fit1) ## (Intercept) x1 x2 ## 2.9573357 0.9856291 1.0170586 coef(fit2) ## (Intercept) x1 x3 ## 2.1945418 0.4770998 0.2542647 coef(fit3) ## (Intercept) x2 x3 ## 1.4788921 -0.9541995 0.4928145 15.2 Collinearity Exact collinearity is an extreme example of collinearity, which occurs in multiple regression when predictor variables are highly correlated. Collinearity is often called multicollinearity, since it is a phenomenon that really only occurs during multiple regression. Looking at the seatpos dataset from the faraway package, we will see an example of this concept. The predictors in this dataset are various attributes of car drivers, such as their height, weight and age. The response variable hipcenter measures the “horizontal distance of the midpoint of the hips from a fixed location in the car in mm.” Essentially, it measures the position of the seat for a given driver. This is potentially useful information for car manufacturers considering comfort and safety when designing vehicles. We will attempt to fit a model that predicts hipcenter. Two predictor variables are immediately interesting to us: HtShoes and Ht. We certainly expect a person’s height to be highly correlated to their height when wearing shoes. We’ll pay special attention to these two variables when fitting models. library(faraway) pairs(seatpos, col = &quot;dodgerblue&quot;) round(cor(seatpos), 2) ## Age Weight HtShoes Ht Seated Arm Thigh Leg hipcenter ## Age 1.00 0.08 -0.08 -0.09 -0.17 0.36 0.09 -0.04 0.21 ## Weight 0.08 1.00 0.83 0.83 0.78 0.70 0.57 0.78 -0.64 ## HtShoes -0.08 0.83 1.00 1.00 0.93 0.75 0.72 0.91 -0.80 ## Ht -0.09 0.83 1.00 1.00 0.93 0.75 0.73 0.91 -0.80 ## Seated -0.17 0.78 0.93 0.93 1.00 0.63 0.61 0.81 -0.73 ## Arm 0.36 0.70 0.75 0.75 0.63 1.00 0.67 0.75 -0.59 ## Thigh 0.09 0.57 0.72 0.73 0.61 0.67 1.00 0.65 -0.59 ## Leg -0.04 0.78 0.91 0.91 0.81 0.75 0.65 1.00 -0.79 ## hipcenter 0.21 -0.64 -0.80 -0.80 -0.73 -0.59 -0.59 -0.79 1.00 After loading the faraway package, we do some quick checks of correlation between the predictors. Visually, we can do this with the pairs() function, which plots all possible scatterplots between pairs of variables in the dataset. We can also do this numerically with the cor() function, which when applied to a dataset, returns all pairwise correlations. Notice this is a symmetric matrix. Recall that correlation measures strength and direction of the linear relationship between two variables. The correlation between Ht and HtShoes is extremely high. So high, that rounded to two decimal places, it appears to be 1! Unlike exact collinearity, here we can still fit a model with all of the predictors, but what effect does this have? hip_model = lm(hipcenter ~ ., data = seatpos) summary(hip_model) ## ## Call: ## lm(formula = hipcenter ~ ., data = seatpos) ## ## Residuals: ## Min 1Q Median 3Q Max ## -73.827 -22.833 -3.678 25.017 62.337 ## ## Coefficients: ## Estimate Std. Error t value Pr(&gt;|t|) ## (Intercept) 436.43213 166.57162 2.620 0.0138 * ## Age 0.77572 0.57033 1.360 0.1843 ## Weight 0.02631 0.33097 0.080 0.9372 ## HtShoes -2.69241 9.75304 -0.276 0.7845 ## Ht 0.60134 10.12987 0.059 0.9531 ## Seated 0.53375 3.76189 0.142 0.8882 ## Arm -1.32807 3.90020 -0.341 0.7359 ## Thigh -1.14312 2.66002 -0.430 0.6706 ## Leg -6.43905 4.71386 -1.366 0.1824 ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 ## ## Residual standard error: 37.72 on 29 degrees of freedom ## Multiple R-squared: 0.6866, Adjusted R-squared: 0.6001 ## F-statistic: 7.94 on 8 and 29 DF, p-value: 1.306e-05 One of the first things we should notice is that the \\(F\\)-test for the regression tells us that the regression is significant, however each individual predictor is not. Another interesting result is the opposite signs of the coefficients for Ht and HtShoes. This should seem rather counter-intuitive. Increasing Ht increases hipcenter, but increasing HtShoes decreases hipcenter? This happens as a result of the predictors being highly correlated. For example, the HtShoe variable explains a large amount of the variation in Ht. When they are both in the model, their effects on the response are lessened individually, but together they still explain a large portion of the variation of hipcenter. We define \\(R_j^2\\) to be the proportion of observed variation in the \\(j\\)-th predictor explained by the other predictors. In other words \\(R_j^2\\) is the multiple R-Squared for the regression of \\(x_j\\) on each of the other predictors. ht_shoes_model = lm(HtShoes ~ . - hipcenter, data = seatpos) summary(ht_shoes_model)$r.squared ## [1] 0.9967472 Here we see that the other predictors explain \\(99.67\\%\\) of the variation in HtShoe. When fitting this model, we removed hipcenter since it is not a predictor. 15.2.1 Variance Inflation Factor. Now note that the variance of \\(\\hat{\\beta_j}\\) can be written as \\[ \\text{Var}(\\hat{\\beta_j}) = \\sigma^2 C_{jj} = \\sigma^2 \\left( \\frac{1}{1 - R_j^2} \\right) \\frac{1}{S_{x_j x_j}} \\] where \\[ S_{x_j x_j} = \\sum(x_{ij}-\\bar{x}_j)^2. \\] This gives us a way to understand how collinearity affects our regression estimates. We will call, \\[ \\frac{1}{1 - R_j^2} \\] the variance inflation factor. The variance inflation factor quantifies the effect of collinearity on the variance of our regression estimates. When \\(R_j^2\\) is large, that is close to 1, \\(x_j\\) is well explained by the other predictors. With a large \\(R_j^2\\) the variance inflation factor becomes large. This tells us that when \\(x_j\\) is highly correlated with other predictors, our estimate of \\(\\beta_j\\) is highly variable. The vif function from the faraway package calculates the VIFs for each of the predictors of a model. vif(hip_model) ## Age Weight HtShoes Ht Seated Arm Thigh ## 1.997931 3.647030 307.429378 333.137832 8.951054 4.496368 2.762886 ## Leg ## 6.694291 In practice it is common to say that any VIF greater than \\(5\\) is cause for concern. So in this example we see there is a huge multicollinearity issue as many of the predictors have a VIF greater than 5. Let’s further investigate how the presence of collinearity actually affects a model. If we add a moderate amount of noise to the data, we see that the estimates of the coefficients change drastically. This is a rather undesirable effect. Adding random noise should not affect the coefficients of a model. set.seed(1337) noise = rnorm(n = nrow(seatpos), mean = 0, sd = 5) hip_model_noise = lm(hipcenter + noise ~ ., data = seatpos) Adding the noise had such a large effect, the sign of the coefficient for Ht has changed. coef(hip_model) ## (Intercept) Age Weight HtShoes Ht Seated ## 436.43212823 0.77571620 0.02631308 -2.69240774 0.60134458 0.53375170 ## Arm Thigh Leg ## -1.32806864 -1.14311888 -6.43904627 coef(hip_model_noise) ## (Intercept) Age Weight HtShoes Ht Seated ## 415.32909380 0.76578240 0.01910958 -2.90377584 -0.12068122 2.03241638 ## Arm Thigh Leg ## -1.02127944 -0.89034509 -5.61777220 This tells us that a model with collinearity is bad at explaining the relationship between the response and the predictors. We cannot even be confident in the direction of the relationship. However, does collinearity affect prediction? plot(fitted(hip_model), fitted(hip_model_noise), col = &quot;dodgerblue&quot;, pch = 20, xlab = &quot;Predicted, Without Noise&quot;, ylab = &quot;Predicted, With Noise&quot;, cex = 1.5) abline(a = 0, b = 1, col = &quot;darkorange&quot;, lwd = 2) We see that by plotting the predicted values using both models against each other, they are actually rather similar. Let’s now look at a smaller model, hip_model_small = lm(hipcenter ~ Age + Arm + Ht, data = seatpos) summary(hip_model_small) ## ## Call: ## lm(formula = hipcenter ~ Age + Arm + Ht, data = seatpos) ## ## Residuals: ## Min 1Q Median 3Q Max ## -82.347 -24.745 -0.094 23.555 58.314 ## ## Coefficients: ## Estimate Std. Error t value Pr(&gt;|t|) ## (Intercept) 493.2491 101.0724 4.880 2.46e-05 *** ## Age 0.7988 0.5111 1.563 0.12735 ## Arm -2.9385 3.5210 -0.835 0.40979 ## Ht -3.4991 0.9954 -3.515 0.00127 ** ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 ## ## Residual standard error: 36.12 on 34 degrees of freedom ## Multiple R-squared: 0.6631, Adjusted R-squared: 0.6333 ## F-statistic: 22.3 on 3 and 34 DF, p-value: 3.649e-08 vif(hip_model_small) ## Age Arm Ht ## 1.749943 3.996766 3.508693 Immediately we see that multicollinearity isn’t an issue here. anova(hip_model_small, hip_model) ## Analysis of Variance Table ## ## Model 1: hipcenter ~ Age + Arm + Ht ## Model 2: hipcenter ~ Age + Weight + HtShoes + Ht + Seated + Arm + Thigh + ## Leg ## Res.Df RSS Df Sum of Sq F Pr(&gt;F) ## 1 34 44354 ## 2 29 41262 5 3091.9 0.4346 0.8207 Also notice that using an \\(F\\)-test to compare the two models, we would prefer the smaller model. We now investigate the effect of adding another variable to this smaller model. Specifically we want to look at adding the variable HtShoes. So now our possible predictors are HtShoes, Age, Arm, and Ht. Our response is still hipcenter. To quantify this effect we will look at a variable added plot and a partial correlation coefficient. For both of these, we will look at the residuals of two models: Regressing the response (hipcenter) against all of the predictors except the predictor of interest (HtShoes). Regressing the predictor of interest (HtShoes) against the other predictors (Age, Arm, and Ht). ht_shoes_model_small = lm(HtShoes ~ Age + Arm + Ht, data = seatpos) So now, the residuals of hip_model_small give us the variation of hipcenter that is unexplained by Age, Arm, and Ht. Similarly, the residuals of ht_shoes_model_small give us the variation of HtShoes unexplained by Age, Arm, and Ht. The correlation of these two residuals gives us the partial correlation coefficient of HtShoes and hipcenter with the effects of Age, Arm, and Ht removed. cor(resid(ht_shoes_model_small), resid(hip_model_small)) ## [1] -0.03311061 Since this value is small, close to zero, it means that the variation of hipcenter that is unexplained by Age, Arm, and Ht shows very little correlation with the variation of HtShoes that is not explained by Age, Arm, and Ht. Thus adding HtShoes to the model would likely be of little benefit. Similarly a variable added plot visualizes these residuals against each other. It is also helpful to regress the residuals of the response against the residuals of the predictor and add the regression line to the plot. plot(resid(hip_model_small) ~ resid(ht_shoes_model_small), col = &quot;dodgerblue&quot;, pch = 20, xlab = &quot;Residuals, Added Predictor&quot;, ylab = &quot;Residuals, Original Model&quot;, main = &quot;Variable Added Plot&quot;) abline(h = 0, lty = 2) abline(v = 0, lty = 2) abline(lm(resid(hip_model_small) ~ resid(ht_shoes_model_small)), col = &quot;darkorange&quot;, lwd = 2) Here the variable added plot shows almost no linear relationship. This tells us that adding HtShoes to the model would probably not be worthwhile. Since its variation is largely explained by the other predictors, adding it to the model will not do much to improve the model. However it will increase the variation of the estimates and make the model much harder to interpret. Had there been a strong linear relationship here, thus a large partial correlation coefficient, it would likely have been useful to add the additional predictor to the model. This trade-off is mostly true in general. As a model gets more predictors, errors will get smaller and its prediction will be better, but it will be harder to interpret. This is why, if we are interested in explaining the relationship between the predictors and the response, we often want a model that fits well, but with a small number of predictors with little correlation. Next chapter we will learn about methods to find models that both fit well, but also have a small number of predictors. We will also discuss overfitting. Although, adding additional predictors will always make errors smaller, sometimes we will be “fitting the noise” and such a model will not generalize to additional observations well. 15.3 Simulation Here we simulate example data with and without collinearity. We will note the difference in the distribution of the estimates of the \\(\\beta\\) parameters, in particular their variance. However, we will also notice the similarity in their \\(MSE\\). We will use the model, \\[ Y = \\beta_0 + \\beta_1 x_1 + \\beta_2 x_2 + \\epsilon \\] where \\(\\epsilon \\sim N(\\mu = 0, \\sigma^2 = 25)\\) and the \\(\\beta\\) coefficients defined below. set.seed(42) beta_0 = 7 beta_1 = 3 beta_2 = 4 sigma = 5 We will use a sample size of 10, and 2500 simulations for both situations. sample_size = 10 num_sim = 2500 We’ll first consider the situation with a collinearity issue, so we manually create the two predictor variables. x1 = c(1, 2, 3, 4, 5, 6, 7, 8, 9, 10) x2 = c(1, 2, 3, 4, 5, 7, 6, 10, 9, 8) c(sd(x1), sd(x2)) ## [1] 3.02765 3.02765 cor(x1, x2) ## [1] 0.9393939 Notice that they have extremely high correlation. true_line_bad = beta_0 + beta_1 * x1 + beta_2 * x2 beta_hat_bad = matrix(0, num_sim, 2) mse_bad = rep(0, num_sim) We perform the simulation 2500 times, each time fitting a regression model, and storing the estimated coefficients and the MSE. for (s in 1:num_sim) { y = true_line_bad + rnorm(n = sample_size, mean = 0, sd = sigma) reg_out = lm(y ~ x1 + x2) beta_hat_bad[s, ] = coef(reg_out)[-1] mse_bad[s] = mean(resid(reg_out) ^ 2) } Now we move to the situation without a collinearity issue, so we again manually create the two predictor variables. z1 = c(1, 2, 3, 4, 5, 6, 7, 8, 9, 10) z2 = c(9, 2, 7, 4, 5, 6, 3, 8, 1, 10) Notice that the standard deviations of each are the same as before, however, now the correlation is extremely close to 0. c(sd(z1), sd(z2)) ## [1] 3.02765 3.02765 cor(z1, z2) ## [1] 0.03030303 true_line_good = beta_0 + beta_1 * z1 + beta_2 * z2 beta_hat_good = matrix(0, num_sim, 2) mse_good = rep(0, num_sim) We then perform simulations and store the same results. for (s in 1:num_sim) { y = true_line_good + rnorm(n = sample_size, mean = 0, sd = sigma) reg_out = lm(y ~ z1 + z2) beta_hat_good[s, ] = coef(reg_out)[-1] mse_good[s] = mean(resid(reg_out) ^ 2) } We’ll now investigate the differences. par(mfrow = c(1, 2)) hist(beta_hat_bad[, 1], col = &quot;darkorange&quot;, border = &quot;dodgerblue&quot;, main = expression(&quot;Histogram of &quot; *hat(beta)[1]* &quot; with Collinearity&quot;), xlab = expression(hat(beta)[1]), breaks = 20) hist(beta_hat_good[, 1], col = &quot;darkorange&quot;, border = &quot;dodgerblue&quot;, main = expression(&quot;Histogram of &quot; *hat(beta)[1]* &quot; without Collinearity&quot;), xlab = expression(hat(beta)[1]), breaks = 20) First, for \\(\\beta_1\\), which has a true value of \\(3\\), we see that both with and without collinearity, the simulated values are centered near \\(3\\). mean(beta_hat_bad[, 1]) ## [1] 2.963325 mean(beta_hat_good[, 1]) ## [1] 3.013414 The way the predictors were created, the \\(S_{x_j x_j}\\) portion of the variance is the same for the predictors in both cases, but the variance is still much larger in the simulations performed with collinearity. The variance is so large in the collinear case, that sometimes the estimated coefficient for \\(\\beta_1\\) is negative! sd(beta_hat_bad[, 1]) ## [1] 1.633294 sd(beta_hat_good[, 1]) ## [1] 0.5484684 par(mfrow = c(1, 2)) hist(beta_hat_bad[, 2], col = &quot;darkorange&quot;, border = &quot;dodgerblue&quot;, main = expression(&quot;Histogram of &quot; *hat(beta)[2]* &quot; with Collinearity&quot;), xlab = expression(hat(beta)[2]), breaks = 20) hist(beta_hat_good[, 2], col = &quot;darkorange&quot;, border = &quot;dodgerblue&quot;, main = expression(&quot;Histogram of &quot; *hat(beta)[2]* &quot; without Collinearity&quot;), xlab = expression(hat(beta)[2]), breaks = 20) We see the same issues with \\(\\beta_2\\). On average the estimates are correct, but the variance is again much larger with collinearity. mean(beta_hat_bad[, 2]) ## [1] 4.025059 mean(beta_hat_good[, 2]) ## [1] 4.004913 sd(beta_hat_bad[, 2]) ## [1] 1.642592 sd(beta_hat_good[, 2]) ## [1] 0.5470381 par(mfrow = c(1, 2)) hist(mse_bad, col = &quot;darkorange&quot;, border = &quot;dodgerblue&quot;, main = &quot;MSE, with Collinearity&quot;, xlab = &quot;MSE&quot;) hist(mse_good, col = &quot;darkorange&quot;, border = &quot;dodgerblue&quot;, main = &quot;MSE, without Collinearity&quot;, xlab = &quot;MSE&quot;) Interestingly, in both cases, the MSE is roughly the same on average. Again, this is because collinearity affects a model’s ability to explain, but not predict. mean(mse_bad) ## [1] 17.7186 mean(mse_good) ## [1] 17.70513 15.4 R Markdown The R Markdown file for this chapter can be found here: collinearity.Rmd The file was created using R version 4.4.1. "],["variable-selection-and-model-building.html", "Chapter 16 Variable Selection and Model Building 16.1 Quality Criterion 16.2 Selection Procedures 16.3 Higher Order Terms 16.4 Explanation versus Prediction 16.5 R Markdown", " Chapter 16 Variable Selection and Model Building “Choose well. Your choice is brief, and yet endless.” — Johann Wolfgang von Goethe After reading this chapter you will be able to: Understand the trade-off between goodness-of-fit and model complexity. Use variable selection procedures to find a good model from a set of possible models. Understand the two uses of models: explanation and prediction. Last chapter we saw how correlation between predictor variables can have undesirable effects on models. We used variance inflation factors to assess the severity of the collinearity issues caused by these correlations. We also saw how fitting a smaller model, leaving out some of the correlated predictors, results in a model which no longer suffers from collinearity issues. But how should we choose this smaller model? This chapter, we will discuss several criteria and procedures for choosing a “good” model from among a choice of many. 16.1 Quality Criterion So far, we have seen criteria such as \\(R^2\\) and \\(\\text{RMSE}\\) for assessing quality of fit. However, both of these have a fatal flaw. By increasing the size of a model, that is adding predictors, that can at worst not improve. It is impossible to add a predictor to a model and make \\(R^2\\) or \\(\\text{RMSE}\\) worse. That means, if we were to use either of these to choose between models, we would always simply choose the larger model. Eventually we would simply be fitting to noise. This suggests that we need a quality criterion that takes into account the size of the model, since our preference is for small models that still fit well. We are willing to sacrifice a small amount of “goodness-of-fit” for obtaining a smaller model. (Here we use “goodness-of-fit” to simply mean how far the data is from the model, the smaller the errors the better. Often in statistics, goodness-of-fit can have a more precise meaning.) We will look at three criteria that do this explicitly: \\(\\text{AIC}\\), \\(\\text{BIC}\\), and Adjusted \\(R^2\\). We will also look at one, Cross-Validated \\(\\text{RMSE}\\), which implicitly considers the size of the model. 16.1.1 Akaike Information Criterion The first criteria we will discuss is the Akaike Information Criterion, or \\(\\text{AIC}\\) for short. (Note that, when Akaike first introduced this metric, it was simply called An Information Criterion. The A has changed meaning over the years.) Recall, the maximized log-likelihood of a regression model can be written as \\[ \\log L(\\boldsymbol{\\hat{\\beta}}, \\hat{\\sigma}^2) = -\\frac{n}{2}\\log(2\\pi) - \\frac{n}{2}\\log\\left(\\frac{\\text{RSS}}{n}\\right) - \\frac{n}{2}, \\] where \\(\\text{RSS} = \\sum_{i=1}^n (y_i - \\hat{y}_i) ^ 2\\) and \\(\\boldsymbol{\\hat{\\beta}}\\) and \\(\\hat{\\sigma}^2\\) were chosen to maximize the likelihood. Then we can define \\(\\text{AIC}\\) as \\[ \\text{AIC} = -2 \\log L(\\boldsymbol{\\hat{\\beta}}, \\hat{\\sigma}^2) + 2p = n + n \\log(2\\pi) + n \\log\\left(\\frac{\\text{RSS}}{n}\\right) + 2p, \\] which is a measure of quality of the model. The smaller the \\(\\text{AIC}\\), the better. To see why, let’s talk about the two main components of \\(\\text{AIC}\\), the likelihood (which measures “goodness-of-fit”) and the penalty (which is a function of the size of the model). The likelihood portion of \\(\\text{AIC}\\) is given by \\[ -2 \\log L(\\boldsymbol{\\hat{\\beta}}, \\hat{\\sigma}^2) = n + n \\log(2\\pi) + n \\log\\left(\\frac{\\text{RSS}}{n}\\right). \\] For the sake of comparing models, the only term here that will change is \\(n \\log\\left(\\frac{\\text{RSS}}{n}\\right)\\), which is a function of \\(\\text{RSS}\\). The \\[ n + n \\log(2\\pi) \\] terms will be constant across all models applied to the same data. So, when a model fits well, that is, has a low \\(\\text{RSS}\\), then this likelihood component will be small. Similarly, we can discuss the penalty component of \\(\\text{AIC}\\) which is, \\[ 2p, \\] where \\(p\\) is the number of \\(\\beta\\) parameters in the model. We call this a penalty, because it is large when \\(p\\) is large, but we are seeking to find a small \\(\\text{AIC}\\) Thus, a good model, that is one with a small \\(\\text{AIC}\\), will have a good balance between fitting well, and using a small number of parameters. For comparing models \\[ \\text{AIC} = n\\log\\left(\\frac{\\text{RSS}}{n}\\right) + 2p \\] is a sufficient expression, as \\(n + n \\log(2\\pi)\\) is the same across all models for any particular dataset. 16.1.2 Bayesian Information Criterion The Bayesian Information Criterion, or \\(\\text{BIC}\\), is similar to \\(\\text{AIC}\\), but has a larger penalty. \\(\\text{BIC}\\) also quantifies the trade-off between a model which fits well and the number of model parameters, however for a reasonable sample size, generally picks a smaller model than \\(\\text{AIC}\\). Again, for model selection use the model with the smallest \\(\\text{BIC}\\). \\[ \\text{BIC} = -2 \\log L(\\boldsymbol{\\hat{\\beta}}, \\hat{\\sigma}^2) + \\log(n) p = n + n\\log(2\\pi) + n\\log\\left(\\frac{\\text{RSS}}{n}\\right) + \\log(n)p. \\] Notice that the \\(\\text{AIC}\\) penalty was \\[ 2p, \\] whereas for \\(\\text{BIC}\\), the penalty is \\[ \\log(n) p. \\] So, for any dataset where \\(log(n) &gt; 2\\) the \\(\\text{BIC}\\) penalty will be larger than the \\(\\text{AIC}\\) penalty, thus \\(\\text{BIC}\\) will likely prefer a smaller model. Note that sometimes the penalty is considered a general expression of the form \\[ k \\cdot p. \\] Then, for \\(\\text{AIC}\\) \\(k = 2\\), and for \\(\\text{BIC}\\) \\(k = \\log(n)\\). For comparing models \\[ \\text{BIC} = n\\log\\left(\\frac{\\text{RSS}}{n}\\right) + \\log(n)p \\] is again a sufficient expression, as \\(n + n \\log(2\\pi)\\) is the same across all models for any particular dataset. 16.1.3 Adjusted R-Squared Recall, \\[ R^2 = 1 - \\frac{\\text{SSE}}{\\text{SST}} = 1 - \\frac{\\sum_{i=1}^{n}(y_i - \\hat{y}_i)^2}{\\sum_{i=1}^{n}(y_i - \\bar{y})^2}. \\] We now define \\[ R_a^2 = 1 - \\frac{\\text{SSE}/(n-p)}{\\text{SST}/(n-1)} = 1 - \\left( \\frac{n-1}{n-p} \\right)(1-R^2) \\] which we call the Adjusted \\(R^2\\). Unlike \\(R^2\\) which can never become smaller with added predictors, Adjusted \\(R^2\\) effectively penalizes for additional predictors, and can decrease with added predictors. Like \\(R^2\\), larger is still better. 16.1.4 Cross-Validated RMSE Each of the previous three metrics explicitly used \\(p\\), the number of parameters, in their calculations. Thus, they all explicitly limit the size of models chosen when used to compare models. We’ll now briefly introduce overfitting and cross-validation. make_poly_data = function(sample_size = 11) { x = seq(0, 10) y = 3 + x + 4 * x ^ 2 + rnorm(n = sample_size, mean = 0, sd = 20) data.frame(x, y) } set.seed(1234) poly_data = make_poly_data() Here we have generated data where the mean of \\(Y\\) is a quadratic function of a single predictor \\(x\\), specifically, \\[ Y = 3 + x + 4 x ^ 2 + \\epsilon. \\] We’ll now fit two models to this data, one which has the correct form, quadratic, and one that is large, which includes terms up to and including an eighth degree. fit_quad = lm(y ~ poly(x, degree = 2), data = poly_data) fit_big = lm(y ~ poly(x, degree = 8), data = poly_data) We then plot the data and the results of the two models. plot(y ~ x, data = poly_data, ylim = c(-100, 400), cex = 2, pch = 20) xplot = seq(0, 10, by = 0.1) lines(xplot, predict(fit_quad, newdata = data.frame(x = xplot)), col = &quot;dodgerblue&quot;, lwd = 2, lty = 1) lines(xplot, predict(fit_big, newdata = data.frame(x = xplot)), col = &quot;darkorange&quot;, lwd = 2, lty = 2) We can see that the solid blue curve models this data rather nicely. The dashed orange curve fits the points better, making smaller errors, however it is unlikely that it is correctly modeling the true relationship between \\(x\\) and \\(y\\). It is fitting the random noise. This is an example of overfitting. We see that the larger model indeed has a lower \\(\\text{RMSE}\\). sqrt(mean(resid(fit_quad) ^ 2)) ## [1] 17.61812 sqrt(mean(resid(fit_big) ^ 2)) ## [1] 10.4197 To correct for this, we will introduce cross-validation. We define the leave-one-out cross-validated RMSE to be \\[ \\text{RMSE}_{\\text{LOOCV}} = \\sqrt{\\frac{1}{n} \\sum_{i=1}^n e_{[i]}^2}. \\] The \\(e_{[i]}\\) are the residual for the \\(i\\)th observation, when that observation is not used to fit the model. \\[ e_{[i]} = y_{i} - \\hat{y}_{[i]} \\] That is, the fitted value is calculated as \\[ \\hat{y}_{[i]} = \\boldsymbol{x}_i ^ \\top \\hat{\\beta}_{[i]} \\] where \\(\\hat{\\beta}_{[i]}\\) are the estimated coefficients when the \\(i\\)th observation is removed from the dataset. In general, to perform this calculation, we would be required to fit the model \\(n\\) times, once with each possible observation removed. However, for leave-one-out cross-validation and linear models, the equation can be rewritten as \\[ \\text{RMSE}_{\\text{LOOCV}} = \\sqrt{\\frac{1}{n}\\sum_{i=1}^n \\left(\\frac{e_{i}}{1-h_{i}}\\right)^2}, \\] where \\(h_i\\) are the leverages and \\(e_i\\) are the usual residuals. This is great, because now we can obtain the LOOCV \\(\\text{RMSE}\\) by fitting only one model! In practice 5 or 10 fold cross-validation are much more popular. For example, in 5-fold cross-validation, the model is fit 5 times, each time leaving out a fifth of the data, then predicting on those values. We’ll leave in-depth examination of cross-validation to a machine learning course, and simply use LOOCV here. Let’s calculate LOOCV \\(\\text{RMSE}\\) for both models, then discuss why we want to do so. We first write a function which calculates the LOOCV \\(\\text{RMSE}\\) as defined using the shortcut formula for linear models. calc_loocv_rmse = function(model) { sqrt(mean((resid(model) / (1 - hatvalues(model))) ^ 2)) } Then calculate the metric for both models. calc_loocv_rmse(fit_quad) ## [1] 23.57189 calc_loocv_rmse(fit_big) ## [1] 1334.357 Now we see that the quadratic model has a much smaller LOOCV \\(\\text{RMSE}\\), so we would prefer this quadratic model. This is because the large model has severely over-fit the data. By leaving a single data point out and fitting the large model, the resulting fit is much different than the fit using all of the data. For example, let’s leave out the third data point and fit both models, then plot the result. fit_quad_removed = lm(y ~ poly(x, degree = 2), data = poly_data[-3, ]) fit_big_removed = lm(y ~ poly(x, degree = 8), data = poly_data[-3, ]) plot(y ~ x, data = poly_data, ylim = c(-100, 400), cex = 2, pch = 20) xplot = seq(0, 10, by = 0.1) lines(xplot, predict(fit_quad_removed, newdata = data.frame(x = xplot)), col = &quot;dodgerblue&quot;, lwd = 2, lty = 1) lines(xplot, predict(fit_big_removed, newdata = data.frame(x = xplot)), col = &quot;darkorange&quot;, lwd = 2, lty = 2) We see that on average, the solid blue line for the quadratic model has similar errors as before. It has changed very slightly. However, the dashed orange line for the large model, has a huge error at the point that was removed and is much different than the previous fit. This is the purpose of cross-validation. By assessing how the model fits points that were not used to perform the regression, we get an idea of how well the model will work for future observations. It assesses how well the model works in general, not simply on the observed data. 16.2 Selection Procedures We’ve now seen a number of model quality criteria, but now we need to address which models to consider. Model selection involves both a quality criterion, plus a search procedure. library(faraway) hipcenter_mod = lm(hipcenter ~ ., data = seatpos) coef(hipcenter_mod) ## (Intercept) Age Weight HtShoes Ht Seated ## 436.43212823 0.77571620 0.02631308 -2.69240774 0.60134458 0.53375170 ## Arm Thigh Leg ## -1.32806864 -1.14311888 -6.43904627 Let’s return to the seatpos data from the faraway package. Now, let’s consider only models with first order terms, thus no interactions and no polynomials. There are eight predictors in this model. So if we consider all possible models, ranging from using 0 predictors, to all eight predictors, there are \\[ \\sum_{k = 0}^{p - 1} {{p - 1} \\choose {k}} = 2 ^ {p - 1} = 2 ^ 8 = 256 \\] possible models. If we had 10 or more predictors, we would already be considering over 1000 models! For this reason, we often search through possible models in an intelligent way, bypassing some models that are unlikely to be considered good. We will consider three search procedures: backwards, forwards, and stepwise. 16.2.1 Backward Search Backward selection procedures start with all possible predictors in the model, then consider how deleting a single predictor will affect a chosen metric. Let’s try this on the seatpos data. We will use the step() function in R which by default uses \\(\\text{AIC}\\) as its metric of choice. hipcenter_mod_back_aic = step(hipcenter_mod, direction = &quot;backward&quot;) ## Start: AIC=283.62 ## hipcenter ~ Age + Weight + HtShoes + Ht + Seated + Arm + Thigh + ## Leg ## ## Df Sum of Sq RSS AIC ## - Ht 1 5.01 41267 281.63 ## - Weight 1 8.99 41271 281.63 ## - Seated 1 28.64 41290 281.65 ## - HtShoes 1 108.43 41370 281.72 ## - Arm 1 164.97 41427 281.78 ## - Thigh 1 262.76 41525 281.87 ## &lt;none&gt; 41262 283.62 ## - Age 1 2632.12 43894 283.97 ## - Leg 1 2654.85 43917 283.99 ## ## Step: AIC=281.63 ## hipcenter ~ Age + Weight + HtShoes + Seated + Arm + Thigh + Leg ## ## Df Sum of Sq RSS AIC ## - Weight 1 11.10 41278 279.64 ## - Seated 1 30.52 41297 279.66 ## - Arm 1 160.50 41427 279.78 ## - Thigh 1 269.08 41536 279.88 ## - HtShoes 1 971.84 42239 280.51 ## &lt;none&gt; 41267 281.63 ## - Leg 1 2664.65 43931 282.01 ## - Age 1 2808.52 44075 282.13 ## ## Step: AIC=279.64 ## hipcenter ~ Age + HtShoes + Seated + Arm + Thigh + Leg ## ## Df Sum of Sq RSS AIC ## - Seated 1 35.10 41313 277.67 ## - Arm 1 156.47 41434 277.78 ## - Thigh 1 285.16 41563 277.90 ## - HtShoes 1 975.48 42253 278.53 ## &lt;none&gt; 41278 279.64 ## - Leg 1 2661.39 43939 280.01 ## - Age 1 3011.86 44290 280.31 ## ## Step: AIC=277.67 ## hipcenter ~ Age + HtShoes + Arm + Thigh + Leg ## ## Df Sum of Sq RSS AIC ## - Arm 1 172.02 41485 275.83 ## - Thigh 1 344.61 41658 275.99 ## - HtShoes 1 1853.43 43166 277.34 ## &lt;none&gt; 41313 277.67 ## - Leg 1 2871.07 44184 278.22 ## - Age 1 2976.77 44290 278.31 ## ## Step: AIC=275.83 ## hipcenter ~ Age + HtShoes + Thigh + Leg ## ## Df Sum of Sq RSS AIC ## - Thigh 1 472.8 41958 274.26 ## &lt;none&gt; 41485 275.83 ## - HtShoes 1 2340.7 43826 275.92 ## - Age 1 3501.0 44986 276.91 ## - Leg 1 3591.7 45077 276.98 ## ## Step: AIC=274.26 ## hipcenter ~ Age + HtShoes + Leg ## ## Df Sum of Sq RSS AIC ## &lt;none&gt; 41958 274.26 ## - Age 1 3108.8 45067 274.98 ## - Leg 1 3476.3 45434 275.28 ## - HtShoes 1 4218.6 46176 275.90 We start with the model hipcenter ~ ., which is otherwise known as hipcenter ~ Age + Weight + HtShoes + Ht + Seated + Arm + Thigh + Leg. R will then repeatedly attempt to delete a predictor until it stops, or reaches the model hipcenter ~ 1, which contains no predictors. At each “step,” R reports the current model, its \\(\\text{AIC}\\), and the possible steps with their \\(\\text{RSS}\\) and more importantly \\(\\text{AIC}\\). In this example, at the first step, the current model is hipcenter ~ Age + Weight + HtShoes + Ht + Seated + Arm + Thigh + Leg which has an AIC of 283.62. Note that when R is calculating this value, it is using extractAIC(), which uses the expression \\[ \\text{AIC} = n\\log\\left(\\frac{\\text{RSS}}{n}\\right) + 2p, \\] which we quickly verify. extractAIC(hipcenter_mod) # returns both p and AIC ## [1] 9.000 283.624 n = length(resid(hipcenter_mod)) (p = length(coef(hipcenter_mod))) ## [1] 9 n * log(mean(resid(hipcenter_mod) ^ 2)) + 2 * p ## [1] 283.624 Returning to the first step, R then gives us a row which shows the effect of deleting each of the current predictors. The - signs at the beginning of each row indicate we are considering removing a predictor. There is also a row with &lt;none&gt; which is a row for keeping the current model. Notice that this row has the smallest \\(\\text{RSS}\\), as it is the largest model. We see that every row above &lt;none&gt; has a smaller \\(\\text{AIC}\\) than the row for &lt;none&gt; with the one at the top, Ht, giving the lowest \\(\\text{AIC}\\). Thus we remove Ht from the model, and continue the process. Notice, in the second step, we start with the model hipcenter ~ Age + Weight + HtShoes + Seated + Arm + Thigh + Leg and the variable Ht is no longer considered. We continue the process until we reach the model hipcenter ~ Age + HtShoes + Leg. At this step, the row for &lt;none&gt; tops the list, as removing any additional variable will not improve the \\(\\text{AIC}\\) This is the model which is stored in hipcenter_mod_back_aic. coef(hipcenter_mod_back_aic) ## (Intercept) Age HtShoes Leg ## 456.2136538 0.5998327 -2.3022555 -6.8297461 We could also search through the possible models in a backwards fashion using \\(\\text{BIC}\\). To do so, we again use the step() function, but now specify k = log(n), where n stores the number of observations in the data. n = length(resid(hipcenter_mod)) hipcenter_mod_back_bic = step(hipcenter_mod, direction = &quot;backward&quot;, k = log(n)) ## Start: AIC=298.36 ## hipcenter ~ Age + Weight + HtShoes + Ht + Seated + Arm + Thigh + ## Leg ## ## Df Sum of Sq RSS AIC ## - Ht 1 5.01 41267 294.73 ## - Weight 1 8.99 41271 294.73 ## - Seated 1 28.64 41290 294.75 ## - HtShoes 1 108.43 41370 294.82 ## - Arm 1 164.97 41427 294.88 ## - Thigh 1 262.76 41525 294.97 ## - Age 1 2632.12 43894 297.07 ## - Leg 1 2654.85 43917 297.09 ## &lt;none&gt; 41262 298.36 ## ## Step: AIC=294.73 ## hipcenter ~ Age + Weight + HtShoes + Seated + Arm + Thigh + Leg ## ## Df Sum of Sq RSS AIC ## - Weight 1 11.10 41278 291.10 ## - Seated 1 30.52 41297 291.12 ## - Arm 1 160.50 41427 291.24 ## - Thigh 1 269.08 41536 291.34 ## - HtShoes 1 971.84 42239 291.98 ## - Leg 1 2664.65 43931 293.47 ## - Age 1 2808.52 44075 293.59 ## &lt;none&gt; 41267 294.73 ## ## Step: AIC=291.1 ## hipcenter ~ Age + HtShoes + Seated + Arm + Thigh + Leg ## ## Df Sum of Sq RSS AIC ## - Seated 1 35.10 41313 287.50 ## - Arm 1 156.47 41434 287.61 ## - Thigh 1 285.16 41563 287.73 ## - HtShoes 1 975.48 42253 288.35 ## - Leg 1 2661.39 43939 289.84 ## - Age 1 3011.86 44290 290.14 ## &lt;none&gt; 41278 291.10 ## ## Step: AIC=287.5 ## hipcenter ~ Age + HtShoes + Arm + Thigh + Leg ## ## Df Sum of Sq RSS AIC ## - Arm 1 172.02 41485 284.02 ## - Thigh 1 344.61 41658 284.18 ## - HtShoes 1 1853.43 43166 285.53 ## - Leg 1 2871.07 44184 286.41 ## - Age 1 2976.77 44290 286.50 ## &lt;none&gt; 41313 287.50 ## ## Step: AIC=284.02 ## hipcenter ~ Age + HtShoes + Thigh + Leg ## ## Df Sum of Sq RSS AIC ## - Thigh 1 472.8 41958 280.81 ## - HtShoes 1 2340.7 43826 282.46 ## - Age 1 3501.0 44986 283.46 ## - Leg 1 3591.7 45077 283.54 ## &lt;none&gt; 41485 284.02 ## ## Step: AIC=280.81 ## hipcenter ~ Age + HtShoes + Leg ## ## Df Sum of Sq RSS AIC ## - Age 1 3108.8 45067 279.89 ## - Leg 1 3476.3 45434 280.20 ## &lt;none&gt; 41958 280.81 ## - HtShoes 1 4218.6 46176 280.81 ## ## Step: AIC=279.89 ## hipcenter ~ HtShoes + Leg ## ## Df Sum of Sq RSS AIC ## - Leg 1 3038.8 48105 278.73 ## &lt;none&gt; 45067 279.89 ## - HtShoes 1 5004.4 50071 280.25 ## ## Step: AIC=278.73 ## hipcenter ~ HtShoes ## ## Df Sum of Sq RSS AIC ## &lt;none&gt; 48105 278.73 ## - HtShoes 1 83534 131639 313.35 The procedure is exactly the same, except at each step we look to improve the \\(\\text{BIC}\\), which R still labels \\(\\text{AIC}\\) in the output. The variable hipcenter_mod_back_bic stores the model chosen by this procedure. coef(hipcenter_mod_back_bic) ## (Intercept) HtShoes ## 565.592659 -4.262091 We note that this model is smaller, has fewer predictors, than the model chosen by \\(\\text{AIC}\\), which is what we would expect. Also note that while both models are different, neither uses both Ht and HtShoes which are extremely correlated. We can use information from the summary() function to compare their Adjusted \\(R^2\\) values. Note that either selected model performs better than the original full model. summary(hipcenter_mod)$adj.r.squared ## [1] 0.6000855 summary(hipcenter_mod_back_aic)$adj.r.squared ## [1] 0.6531427 summary(hipcenter_mod_back_bic)$adj.r.squared ## [1] 0.6244149 We can also calculate the LOOCV \\(\\text{RMSE}\\) for both selected models, as well as the full model. calc_loocv_rmse(hipcenter_mod) ## [1] 44.44564 calc_loocv_rmse(hipcenter_mod_back_aic) ## [1] 37.58473 calc_loocv_rmse(hipcenter_mod_back_bic) ## [1] 37.40564 We see that we would prefer the model chosen via \\(\\text{BIC}\\) if using LOOCV \\(\\text{RMSE}\\) as our metric. 16.2.2 Forward Search Forward selection is the exact opposite of backwards selection. Here we tell R to start with a model using no predictors, that is hipcenter ~ 1, then at each step R will attempt to add a predictor until it finds a good model or reaches hipcenter ~ Age + Weight + HtShoes + Ht + Seated + Arm + Thigh + Leg. hipcenter_mod_start = lm(hipcenter ~ 1, data = seatpos) hipcenter_mod_forw_aic = step( hipcenter_mod_start, scope = hipcenter ~ Age + Weight + HtShoes + Ht + Seated + Arm + Thigh + Leg, direction = &quot;forward&quot;) ## Start: AIC=311.71 ## hipcenter ~ 1 ## ## Df Sum of Sq RSS AIC ## + Ht 1 84023 47616 275.07 ## + HtShoes 1 83534 48105 275.45 ## + Leg 1 81568 50071 276.98 ## + Seated 1 70392 61247 284.63 ## + Weight 1 53975 77664 293.66 ## + Thigh 1 46010 85629 297.37 ## + Arm 1 45065 86574 297.78 ## &lt;none&gt; 131639 311.71 ## + Age 1 5541 126098 312.07 ## ## Step: AIC=275.07 ## hipcenter ~ Ht ## ## Df Sum of Sq RSS AIC ## + Leg 1 2781.10 44835 274.78 ## &lt;none&gt; 47616 275.07 ## + Age 1 2353.51 45262 275.14 ## + Weight 1 195.86 47420 276.91 ## + Seated 1 101.56 47514 276.99 ## + Arm 1 75.78 47540 277.01 ## + HtShoes 1 25.76 47590 277.05 ## + Thigh 1 4.63 47611 277.06 ## ## Step: AIC=274.78 ## hipcenter ~ Ht + Leg ## ## Df Sum of Sq RSS AIC ## + Age 1 2896.60 41938 274.24 ## &lt;none&gt; 44835 274.78 ## + Arm 1 522.72 44312 276.33 ## + Weight 1 445.10 44390 276.40 ## + HtShoes 1 34.11 44801 276.75 ## + Thigh 1 32.96 44802 276.75 ## + Seated 1 1.12 44834 276.78 ## ## Step: AIC=274.24 ## hipcenter ~ Ht + Leg + Age ## ## Df Sum of Sq RSS AIC ## &lt;none&gt; 41938 274.24 ## + Thigh 1 372.71 41565 275.90 ## + Arm 1 257.09 41681 276.01 ## + Seated 1 121.26 41817 276.13 ## + Weight 1 46.83 41891 276.20 ## + HtShoes 1 13.38 41925 276.23 Again, by default R uses \\(\\text{AIC}\\) as its quality metric when using the step() function. Also note that now the rows begin with a + which indicates addition of predictors to the current model from any step. hipcenter_mod_forw_bic = step( hipcenter_mod_start, scope = hipcenter ~ Age + Weight + HtShoes + Ht + Seated + Arm + Thigh + Leg, direction = &quot;forward&quot;, k = log(n)) ## Start: AIC=313.35 ## hipcenter ~ 1 ## ## Df Sum of Sq RSS AIC ## + Ht 1 84023 47616 278.34 ## + HtShoes 1 83534 48105 278.73 ## + Leg 1 81568 50071 280.25 ## + Seated 1 70392 61247 287.91 ## + Weight 1 53975 77664 296.93 ## + Thigh 1 46010 85629 300.64 ## + Arm 1 45065 86574 301.06 ## &lt;none&gt; 131639 313.35 ## + Age 1 5541 126098 315.35 ## ## Step: AIC=278.34 ## hipcenter ~ Ht ## ## Df Sum of Sq RSS AIC ## &lt;none&gt; 47616 278.34 ## + Leg 1 2781.10 44835 279.69 ## + Age 1 2353.51 45262 280.05 ## + Weight 1 195.86 47420 281.82 ## + Seated 1 101.56 47514 281.90 ## + Arm 1 75.78 47540 281.92 ## + HtShoes 1 25.76 47590 281.96 ## + Thigh 1 4.63 47611 281.98 We can make the same modification as last time to instead use \\(\\text{BIC}\\) with forward selection. summary(hipcenter_mod)$adj.r.squared ## [1] 0.6000855 summary(hipcenter_mod_forw_aic)$adj.r.squared ## [1] 0.6533055 summary(hipcenter_mod_forw_bic)$adj.r.squared ## [1] 0.6282374 We can compare the two selected models’ Adjusted \\(R^2\\) as well as their LOOCV \\(\\text{RMSE}\\) The results are very similar to those using backwards selection, although the models are not exactly the same. calc_loocv_rmse(hipcenter_mod) ## [1] 44.44564 calc_loocv_rmse(hipcenter_mod_forw_aic) ## [1] 37.62516 calc_loocv_rmse(hipcenter_mod_forw_bic) ## [1] 37.2511 16.2.3 Stepwise Search Stepwise search checks going both backwards and forwards at every step. It considers the addition of any variable not currently in the model, as well as the removal of any variable currently in the model. Here we perform stepwise search using \\(\\text{AIC}\\) as our metric. We start with the model hipcenter ~ 1 and search up to hipcenter ~ Age + Weight + HtShoes + Ht + Seated + Arm + Thigh + Leg. Notice that at many of the steps, some row begin with -, while others begin with +. hipcenter_mod_both_aic = step( hipcenter_mod_start, scope = hipcenter ~ Age + Weight + HtShoes + Ht + Seated + Arm + Thigh + Leg, direction = &quot;both&quot;) ## Start: AIC=311.71 ## hipcenter ~ 1 ## ## Df Sum of Sq RSS AIC ## + Ht 1 84023 47616 275.07 ## + HtShoes 1 83534 48105 275.45 ## + Leg 1 81568 50071 276.98 ## + Seated 1 70392 61247 284.63 ## + Weight 1 53975 77664 293.66 ## + Thigh 1 46010 85629 297.37 ## + Arm 1 45065 86574 297.78 ## &lt;none&gt; 131639 311.71 ## + Age 1 5541 126098 312.07 ## ## Step: AIC=275.07 ## hipcenter ~ Ht ## ## Df Sum of Sq RSS AIC ## + Leg 1 2781 44835 274.78 ## &lt;none&gt; 47616 275.07 ## + Age 1 2354 45262 275.14 ## + Weight 1 196 47420 276.91 ## + Seated 1 102 47514 276.99 ## + Arm 1 76 47540 277.01 ## + HtShoes 1 26 47590 277.05 ## + Thigh 1 5 47611 277.06 ## - Ht 1 84023 131639 311.71 ## ## Step: AIC=274.78 ## hipcenter ~ Ht + Leg ## ## Df Sum of Sq RSS AIC ## + Age 1 2896.6 41938 274.24 ## &lt;none&gt; 44835 274.78 ## - Leg 1 2781.1 47616 275.07 ## + Arm 1 522.7 44312 276.33 ## + Weight 1 445.1 44390 276.40 ## + HtShoes 1 34.1 44801 276.75 ## + Thigh 1 33.0 44802 276.75 ## + Seated 1 1.1 44834 276.78 ## - Ht 1 5236.3 50071 276.98 ## ## Step: AIC=274.24 ## hipcenter ~ Ht + Leg + Age ## ## Df Sum of Sq RSS AIC ## &lt;none&gt; 41938 274.24 ## - Age 1 2896.6 44835 274.78 ## - Leg 1 3324.2 45262 275.14 ## - Ht 1 4238.3 46176 275.90 ## + Thigh 1 372.7 41565 275.90 ## + Arm 1 257.1 41681 276.01 ## + Seated 1 121.3 41817 276.13 ## + Weight 1 46.8 41891 276.20 ## + HtShoes 1 13.4 41925 276.23 We could again instead use \\(\\text{BIC}\\) as our metric. hipcenter_mod_both_bic = step( hipcenter_mod_start, scope = hipcenter ~ Age + Weight + HtShoes + Ht + Seated + Arm + Thigh + Leg, direction = &quot;both&quot;, k = log(n)) ## Start: AIC=313.35 ## hipcenter ~ 1 ## ## Df Sum of Sq RSS AIC ## + Ht 1 84023 47616 278.34 ## + HtShoes 1 83534 48105 278.73 ## + Leg 1 81568 50071 280.25 ## + Seated 1 70392 61247 287.91 ## + Weight 1 53975 77664 296.93 ## + Thigh 1 46010 85629 300.64 ## + Arm 1 45065 86574 301.06 ## &lt;none&gt; 131639 313.35 ## + Age 1 5541 126098 315.35 ## ## Step: AIC=278.34 ## hipcenter ~ Ht ## ## Df Sum of Sq RSS AIC ## &lt;none&gt; 47616 278.34 ## + Leg 1 2781 44835 279.69 ## + Age 1 2354 45262 280.05 ## + Weight 1 196 47420 281.82 ## + Seated 1 102 47514 281.90 ## + Arm 1 76 47540 281.92 ## + HtShoes 1 26 47590 281.96 ## + Thigh 1 5 47611 281.98 ## - Ht 1 84023 131639 313.35 Adjusted \\(R^2\\) and LOOCV \\(\\text{RMSE}\\) comparisons are similar to backwards and forwards, which is not at all surprising, as some of the models selected are the same as before. summary(hipcenter_mod)$adj.r.squared ## [1] 0.6000855 summary(hipcenter_mod_both_aic)$adj.r.squared ## [1] 0.6533055 summary(hipcenter_mod_both_bic)$adj.r.squared ## [1] 0.6282374 calc_loocv_rmse(hipcenter_mod) ## [1] 44.44564 calc_loocv_rmse(hipcenter_mod_both_aic) ## [1] 37.62516 calc_loocv_rmse(hipcenter_mod_both_bic) ## [1] 37.2511 16.2.4 Exhaustive Search Backward, forward, and stepwise search are all useful, but do have an obvious issue. By not checking every possible model, sometimes they will miss the best possible model. With an extremely large number of predictors, sometimes this is necessary since checking every possible model would be rather time consuming, even with current computers. However, with a reasonably sized dataset, it isn’t too difficult to check all possible models. To do so, we will use the regsubsets() function in the R package leaps. library(leaps) all_hipcenter_mod = summary(regsubsets(hipcenter ~ ., data = seatpos)) A few points about this line of code. First, note that we immediately use summary() and store those results. That is simply the intended use of regsubsets(). Second, inside of regsubsets() we specify the model hipcenter ~ .. This will be the largest model considered, that is the model using all first-order predictors, and R will check all possible subsets. Third, regsubsets() has an argument of nvmax to set the maximum size of subsets to examine with \\(8\\) as default. You need to change it if you investigate the subsets with more than \\(8\\) predictors, although increasing nvmax will significantly increase computation time. We’ll now look at the information stored in all_hipcenter_mod. all_hipcenter_mod$which ## (Intercept) Age Weight HtShoes Ht Seated Arm Thigh Leg ## 1 TRUE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE ## 2 TRUE FALSE FALSE FALSE TRUE FALSE FALSE FALSE TRUE ## 3 TRUE TRUE FALSE FALSE TRUE FALSE FALSE FALSE TRUE ## 4 TRUE TRUE FALSE TRUE FALSE FALSE FALSE TRUE TRUE ## 5 TRUE TRUE FALSE TRUE FALSE FALSE TRUE TRUE TRUE ## 6 TRUE TRUE FALSE TRUE FALSE TRUE TRUE TRUE TRUE ## 7 TRUE TRUE TRUE TRUE FALSE TRUE TRUE TRUE TRUE ## 8 TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE Using $which gives us the best model, according to \\(\\text{RSS}\\), for a model of each possible size, in this case ranging from one to eight predictors. For example the best model with four predictors (\\(p = 5\\)) would use Age, HtShoes, Thigh, and Leg. all_hipcenter_mod$rss ## [1] 47615.79 44834.69 41938.09 41485.01 41313.00 41277.90 41266.80 41261.78 We can obtain the \\(\\text{RSS}\\) for each of these models using $rss. Notice that these are decreasing since the models range from small to large. Now that we have the \\(\\text{RSS}\\) for each of these models, it is rather easy to obtain \\(\\text{AIC}\\), \\(\\text{BIC}\\), and Adjusted \\(R^2\\) since they are all a function of \\(\\text{RSS}\\) Also, since we have the models with the best \\(\\text{RSS}\\) for each size, they will result in the models with the best \\(\\text{AIC}\\), \\(\\text{BIC}\\), and Adjusted \\(R^2\\) for each size. Then by picking from those, we can find the overall best \\(\\text{AIC}\\), \\(\\text{BIC}\\), and Adjusted \\(R^2\\). Conveniently, Adjusted \\(R^2\\) is automatically calculated. all_hipcenter_mod$adjr2 ## [1] 0.6282374 0.6399496 0.6533055 0.6466586 0.6371276 0.6257403 0.6133690 ## [8] 0.6000855 To find which model has the highest Adjusted \\(R^2\\) we can use the which.max() function. (best_r2_ind = which.max(all_hipcenter_mod$adjr2)) ## [1] 3 We can then extract the predictors of that model. all_hipcenter_mod$which[best_r2_ind, ] ## (Intercept) Age Weight HtShoes Ht Seated ## TRUE TRUE FALSE FALSE TRUE FALSE ## Arm Thigh Leg ## FALSE FALSE TRUE We’ll now calculate \\(\\text{AIC}\\) and \\(\\text{BIC}\\) for each of the models with the best \\(\\text{RSS}\\). To do so, we will need both \\(n\\) and the \\(p\\) for the largest possible model. p = length(coef(hipcenter_mod)) n = length(resid(hipcenter_mod)) We’ll use the form of \\(\\text{AIC}\\) which leaves out the constant term that is equal across all models. \\[ \\text{AIC} = n\\log\\left(\\frac{\\text{RSS}}{n}\\right) + 2p. \\] Since we have the \\(\\text{RSS}\\) of each model stored, this is easy to calculate. hipcenter_mod_aic = n * log(all_hipcenter_mod$rss / n) + 2 * (2:p) We can then extract the predictors of the model with the best \\(\\text{AIC}\\). best_aic_ind = which.min(hipcenter_mod_aic) all_hipcenter_mod$which[best_aic_ind,] ## (Intercept) Age Weight HtShoes Ht Seated ## TRUE TRUE FALSE FALSE TRUE FALSE ## Arm Thigh Leg ## FALSE FALSE TRUE Let’s fit this model so we can compare to our previously chosen models using \\(\\text{AIC}\\) and search procedures. hipcenter_mod_best_aic = lm(hipcenter ~ Age + Ht + Leg, data = seatpos) The extractAIC() function will calculate the \\(\\text{AIC}\\) defined above for a fitted model. extractAIC(hipcenter_mod_best_aic) ## [1] 4.0000 274.2418 extractAIC(hipcenter_mod_back_aic) ## [1] 4.0000 274.2597 extractAIC(hipcenter_mod_forw_aic) ## [1] 4.0000 274.2418 extractAIC(hipcenter_mod_both_aic) ## [1] 4.0000 274.2418 We see that two of the models chosen by search procedures have the best possible \\(\\text{AIC}\\), as they are the same model. This is however never guaranteed. We see that the model chosen using backwards selection does not achieve the smallest possible \\(\\text{AIC}\\). plot(hipcenter_mod_aic ~ I(2:p), ylab = &quot;AIC&quot;, xlab = &quot;p, number of parameters&quot;, pch = 20, col = &quot;dodgerblue&quot;, type = &quot;b&quot;, cex = 2, main = &quot;AIC vs Model Complexity&quot;) We could easily repeat this process for \\(\\text{BIC}\\). \\[ \\text{BIC} = n\\log\\left(\\frac{\\text{RSS}}{n}\\right) + \\log(n)p. \\] hipcenter_mod_bic = n * log(all_hipcenter_mod$rss / n) + log(n) * (2:p) which.min(hipcenter_mod_bic) ## [1] 1 all_hipcenter_mod$which[1,] ## (Intercept) Age Weight HtShoes Ht Seated ## TRUE FALSE FALSE FALSE TRUE FALSE ## Arm Thigh Leg ## FALSE FALSE FALSE hipcenter_mod_best_bic = lm(hipcenter ~ Ht, data = seatpos) extractAIC(hipcenter_mod_best_bic, k = log(n)) ## [1] 2.0000 278.3418 extractAIC(hipcenter_mod_back_bic, k = log(n)) ## [1] 2.0000 278.7306 extractAIC(hipcenter_mod_forw_bic, k = log(n)) ## [1] 2.0000 278.3418 extractAIC(hipcenter_mod_both_bic, k = log(n)) ## [1] 2.0000 278.3418 16.3 Higher Order Terms So far we have only allowed first-order terms in our models. Let’s return to the autompg dataset to explore higher-order terms. autompg = read.table( &quot;http://archive.ics.uci.edu/ml/machine-learning-databases/auto-mpg/auto-mpg.data&quot;, quote = &quot;\\&quot;&quot;, comment.char = &quot;&quot;, stringsAsFactors = FALSE) colnames(autompg) = c(&quot;mpg&quot;, &quot;cyl&quot;, &quot;disp&quot;, &quot;hp&quot;, &quot;wt&quot;, &quot;acc&quot;, &quot;year&quot;, &quot;origin&quot;, &quot;name&quot;) autompg = subset(autompg, autompg$hp != &quot;?&quot;) autompg = subset(autompg, autompg$name != &quot;plymouth reliant&quot;) rownames(autompg) = paste(autompg$cyl, &quot;cylinder&quot;, autompg$year, autompg$name) autompg$hp = as.numeric(autompg$hp) autompg$domestic = as.numeric(autompg$origin == 1) autompg = autompg[autompg$cyl != 5,] autompg = autompg[autompg$cyl != 3,] autompg$cyl = as.factor(autompg$cyl) autompg$domestic = as.factor(autompg$domestic) autompg = subset(autompg, select = c(&quot;mpg&quot;, &quot;cyl&quot;, &quot;disp&quot;, &quot;hp&quot;, &quot;wt&quot;, &quot;acc&quot;, &quot;year&quot;, &quot;domestic&quot;)) str(autompg) ## &#39;data.frame&#39;: 383 obs. of 8 variables: ## $ mpg : num 18 15 18 16 17 15 14 14 14 15 ... ## $ cyl : Factor w/ 3 levels &quot;4&quot;,&quot;6&quot;,&quot;8&quot;: 3 3 3 3 3 3 3 3 3 3 ... ## $ disp : num 307 350 318 304 302 429 454 440 455 390 ... ## $ hp : num 130 165 150 150 140 198 220 215 225 190 ... ## $ wt : num 3504 3693 3436 3433 3449 ... ## $ acc : num 12 11.5 11 12 10.5 10 9 8.5 10 8.5 ... ## $ year : int 70 70 70 70 70 70 70 70 70 70 ... ## $ domestic: Factor w/ 2 levels &quot;0&quot;,&quot;1&quot;: 2 2 2 2 2 2 2 2 2 2 ... Recall that we have two factor variables, cyl and domestic. The cyl variable has three levels, while the domestic variable has only two. Thus the cyl variable will be coded using two dummy variables, while the domestic variable will only need one. We will pay attention to this later. pairs(autompg, col = &quot;dodgerblue&quot;) We’ll use the pairs() plot to determine which variables may benefit from a quadratic relationship with the response. We’ll also consider all possible two-way interactions. We won’t consider any three-order or higher. For example, we won’t consider the interaction between first-order terms and the added quadratic terms. So now, we’ll fit this rather large model. We’ll use a log-transformed response. Notice that log(mpg) ~ . ^ 2 will automatically consider all first-order terms, as well as all two-way interactions. We use I(var_name ^ 2) to add quadratic terms for some variables. This generally works better than using poly() when performing variable selection. autompg_big_mod = lm( log(mpg) ~ . ^ 2 + I(disp ^ 2) + I(hp ^ 2) + I(wt ^ 2) + I(acc ^ 2), data = autompg) We think it is rather unlikely that we truly need all of these terms. There are quite a few! length(coef(autompg_big_mod)) ## [1] 40 We’ll try backwards search with both \\(\\text{AIC}\\) and \\(\\text{BIC}\\) to attempt to find a smaller, more reasonable model. autompg_mod_back_aic = step(autompg_big_mod, direction = &quot;backward&quot;, trace = 0) Notice that we used trace = 0 in the function call. This suppresses the output for each step, and simply stores the chosen model. This is useful, as this code would otherwise create a large amount of output. If we had viewed the output, which you can try on your own by removing trace = 0, we would see that R only considers the cyl variable as a single variable, despite the fact that it is coded using two dummy variables. So removing cyl would actually remove two parameters from the resulting model. You should also notice that R respects hierarchy when attempting to remove variables. That is, for example, R will not consider removing hp if hp:disp or I(hp ^ 2) are currently in the model. We also use \\(\\text{BIC}\\). n = length(resid(autompg_big_mod)) autompg_mod_back_bic = step(autompg_big_mod, direction = &quot;backward&quot;, k = log(n), trace = 0) Looking at the coefficients of the two chosen models, we see they are still rather large. coef(autompg_mod_back_aic) ## (Intercept) cyl6 cyl8 disp hp ## 3.671884e+00 -1.602563e-01 -8.581644e-01 -9.371971e-03 2.293534e-02 ## wt acc year domestic1 I(hp^2) ## -3.064497e-04 -1.393888e-01 -1.966361e-03 9.369324e-01 -1.497669e-05 ## cyl6:acc cyl8:acc disp:wt disp:year hp:acc ## 7.220298e-03 5.041915e-02 5.797816e-07 9.493770e-05 -5.062295e-04 ## hp:year acc:year acc:domestic1 year:domestic1 ## -1.838985e-04 2.345625e-03 -2.372468e-02 -7.332725e-03 coef(autompg_mod_back_bic) ## (Intercept) cyl6 cyl8 disp hp ## 4.657847e+00 -1.086165e-01 -7.611631e-01 -1.609316e-03 2.621266e-03 ## wt acc year domestic1 cyl6:acc ## -2.635972e-04 -1.670601e-01 -1.045646e-02 3.341579e-01 4.315493e-03 ## cyl8:acc disp:wt hp:acc acc:year acc:domestic1 ## 4.610095e-02 4.102804e-07 -3.386261e-04 2.500137e-03 -2.193294e-02 However, they are much smaller than the original full model. Also notice that the resulting models respect hierarchy. length(coef(autompg_big_mod)) ## [1] 40 length(coef(autompg_mod_back_aic)) ## [1] 19 length(coef(autompg_mod_back_bic)) ## [1] 15 Calculating the LOOCV \\(\\text{RMSE}\\) for each, we see that the model chosen using \\(\\text{BIC}\\) performs the best. That means that it is both the best model for prediction, since it achieves the best LOOCV \\(\\text{RMSE}\\), but also the best model for explanation, as it is also the smallest. calc_loocv_rmse(autompg_big_mod) ## [1] 0.1112024 calc_loocv_rmse(autompg_mod_back_aic) ## [1] 0.1032888 calc_loocv_rmse(autompg_mod_back_bic) ## [1] 0.103134 16.4 Explanation versus Prediction Throughout this chapter, we have attempted to find reasonably “small” models, which are good at explaining the relationship between the response and the predictors, that also have small errors which are thus good for making predictions. We’ll further discuss the model autompg_mod_back_bic to better explain the difference between using models for explaining and predicting. This is the model fit to the autompg data that was chosen using Backwards Search and \\(\\text{BIC}\\), which obtained the lowest LOOCV \\(\\text{RMSE}\\) of the models we considered. autompg_mod_back_bic ## ## Call: ## lm(formula = log(mpg) ~ cyl + disp + hp + wt + acc + year + domestic + ## cyl:acc + disp:wt + hp:acc + acc:year + acc:domestic, data = autompg) ## ## Coefficients: ## (Intercept) cyl6 cyl8 disp hp ## 4.658e+00 -1.086e-01 -7.612e-01 -1.609e-03 2.621e-03 ## wt acc year domestic1 cyl6:acc ## -2.636e-04 -1.671e-01 -1.046e-02 3.342e-01 4.315e-03 ## cyl8:acc disp:wt hp:acc acc:year acc:domestic1 ## 4.610e-02 4.103e-07 -3.386e-04 2.500e-03 -2.193e-02 Notice this is a somewhat “large” model, which uses 15 parameters, including several interaction terms. Do we care that this is a “large” model? The answer is, it depends. 16.4.1 Explanation Suppose we would like to use this model for explanation. Perhaps we are a car manufacturer trying to engineer a fuel efficient vehicle. If this is the case, we are interested in both what predictor variables are useful for explaining the car’s fuel efficiency, as well as how those variables affect fuel efficiency. By understanding this relationship, we can use this knowledge to our advantage when designing a car. To explain a relationship, we are interested in keeping models as small as possible, since smaller models are easy to interpret. The fewer predictors the fewer considerations we need to make in our design process. Also the fewer interactions and polynomial terms, the easier it is to interpret any one parameter, since the parameter interpretations are conditional on which parameters are in the model. Note that linear models are rather interpretable to begin with. Later in your data analysis careers, you will see more complicated models that may fit data better, but are much harder, if not impossible to interpret. These models aren’t very useful for explaining a relationship. To find small and interpretable models, we would use selection criteria that explicitly penalize larger models, such as AIC and BIC. In this case we still obtained a somewhat large model, but much smaller than the model we used to start the selection process. 16.4.1.1 Correlation and Causation A word of caution when using a model to explain a relationship. There are two terms often used to describe a relationship between two variables: causation and correlation. Correlation is often also referred to as association. Just because two variables are correlated does not necessarily mean that one causes the other. For example, considering modeling mpg as only a function of hp. plot(mpg ~ hp, data = autompg, col = &quot;dodgerblue&quot;, pch = 20, cex = 1.5) Does an increase in horsepower cause a drop in fuel efficiency? Or, perhaps the causality is reversed and an increase in fuel efficiency causes a decrease in horsepower. Or, perhaps there is a third variable that explains both! The issue here is that we have observational data. With observational data, we can only detect associations. To speak with confidence about causality, we would need to run experiments. This is a concept that you should encounter often in your statistics education. For some further reading, and some related fallacies, see: Wikipedia: Correlation does not imply causation. 16.4.2 Prediction Suppose now instead of the manufacturer who would like to build a car, we are a consumer who wishes to purchase a new car. However this particular car is so new, it has not been rigorously tested, so we are unsure of what fuel efficiency to expect. (And, as skeptics, we don’t trust what the manufacturer is telling us.) In this case, we would like to use the model to help predict the fuel efficiency of this car based on its attributes, which are the predictors of the model. The smaller the errors the model makes, the more confident we are in its prediction. Thus, to find models for prediction, we would use selection criteria that implicitly penalize larger models, such as LOOCV \\(\\text{RMSE}\\). So long as the model does not over-fit, we do not actually care how large the model becomes. Explaining the relationship between the variables is not our goal here, we simply want to know what kind of fuel efficiency we should expect! If we only care about prediction, we don’t need to worry about correlation vs causation, and we don’t need to worry about model assumptions. If a variable is correlated with the response, it doesn’t actually matter if it causes an effect on the response, it can still be useful for prediction. For example, in elementary school aged children their shoe size certainly doesn’t cause them to read at a higher level, however we could very easily use shoe size to make a prediction about a child’s reading ability. The larger their shoe size, the better they read. There’s a lurking variable here though, their age! (Don’t send your kids to school with size 14 shoes, it won’t make them read better!) We also don’t care about model assumptions. Least squares is least squares. For a specified model, it will find the values of the parameters which will minimize the squared error loss. Your results might be largely uninterpretable and useless for inference, but for prediction none of that matters. 16.5 R Markdown The R Markdown file for this chapter can be found here: selection.Rmd The file was created using R version 4.4.1. "],["logistic-regression.html", "Chapter 17 Logistic Regression 17.1 Generalized Linear Models 17.2 Binary Response 17.3 Working with Logistic Regression 17.4 Classification 17.5 R Markdown", " Chapter 17 Logistic Regression Note to current readers: This chapter is slightly less tested than previous chapters. Please do not hesitate to report any errors, or suggest sections that need better explanation! Also, as a result, this material is more likely to receive edits. After reading this chapter you will be able to: Understand how generalized linear models are a generalization of ordinary linear models. Use logistic regression to model a binary response. Apply concepts learned for ordinary linear models to logistic regression. Use logistic regression to perform classification. So far we have only considered models for numeric response variables. What about response variables that only take integer values? What about a response variable that is categorical? Can we use linear models in these situations? Yes! The model that we have been using, which we will call ordinary linear regression, is actually a specific case of the more general, generalized linear model. (Aren’t statisticians great at naming things?) 17.1 Generalized Linear Models So far, we’ve had response variables that, conditioned on the predictors, were modeled using a normal distribution with a mean that is some linear combination of the predictors. This linear combination is what made a linear model “linear.” \\[ Y \\mid {\\bf X} = {\\bf x} \\sim N(\\beta_0 + \\beta_1x_1 + \\ldots + \\beta_{p - 1}x_{p - 1}, \\ \\sigma^2) \\] Now we’ll allow for two modifications of this situation, which will let us use linear models in many more situations. Instead of using a normal distribution for the response conditioned on the predictors, we’ll allow for other distributions. Also, instead of the conditional mean being a linear combination of the predictors, it can be some function of a linear combination of the predictors. In general, a generalized linear model has three parts: A distribution of the response conditioned on the predictors. (Technically this distribution needs to be from the exponential family of distributions.) A linear combination of the \\(p - 1\\) predictors, \\(\\beta_0 + \\beta_1 x_1 + \\beta_2 x_2 + \\ldots + \\beta_{p - 1} x_{p - 1}\\), which we write as \\(\\eta({\\bf x})\\). That is, \\[\\eta({\\bf x}) = \\beta_0 + \\beta_1 x_1 + \\beta_2 x_2 + \\ldots + \\beta_{p - 1} x_{p - 1}\\] A link function, \\(g()\\), that defines how \\(\\eta({\\bf x})\\), the linear combination of the predictors, is related to the mean of the response conditioned on the predictors, \\(\\text{E}[Y \\mid {\\bf X} = {\\bf x}]\\). \\[ \\eta({\\bf x}) = g\\left(\\text{E}[Y \\mid {\\bf X} = {\\bf x}]\\right). \\] The following table summarizes three examples of a generalized linear model: Linear Regression Poisson Regression Logistic Regression \\(Y \\mid {\\bf X} = {\\bf x}\\) \\(N(\\mu({\\bf x}), \\sigma^2)\\) \\(\\text{Pois}(\\lambda({\\bf x}))\\) \\(\\text{Bern}(p({\\bf x}))\\) Distribution Name Normal Poisson Bernoulli (Binomial) \\(\\text{E}[Y \\mid {\\bf X} = {\\bf x}]\\) \\(\\mu({\\bf x})\\) \\(\\lambda({\\bf x})\\) \\(p({\\bf x})\\) Support Real: \\((-\\infty, \\infty)\\) Integer: \\(0, 1, 2, \\ldots\\) Integer: \\(0, 1\\) Usage Numeric Data Count (Integer) Data Binary (Class ) Data Link Name Identity Log Logit Link Function \\(\\eta({\\bf x}) = \\mu({\\bf x})\\) \\(\\eta({\\bf x}) = \\log(\\lambda({\\bf x}))\\) \\(\\eta({\\bf x}) = \\log \\left(\\frac{p({\\bf x})}{1 - p({\\bf x})} \\right)\\) Mean Function \\(\\mu({\\bf x}) = \\eta({\\bf x})\\) \\(\\lambda({\\bf x}) = e^{\\eta({\\bf x})}\\) \\(p({\\bf x}) = \\frac{e^{\\eta({\\bf x})}}{1 + e^{\\eta({\\bf x})}} = \\frac{1}{1 + e^{-\\eta({\\bf x})}}\\) Like ordinary linear regression, we will seek to “fit” the model by estimating the \\(\\beta\\) parameters. To do so, we will use the method of maximum likelihood. Note that a Bernoulli distribution is a specific case of a binomial distribution where the \\(n\\) parameter of a binomial is \\(1\\). Binomial regression is also possible, but we’ll focus on the much more popular Bernoulli case. So, in general, GLMs relate the mean of the response to a linear combination of the predictors, \\(\\eta({\\bf x})\\), through the use of a link function, \\(g()\\). That is, \\[ \\eta({\\bf x}) = g\\left(\\text{E}[Y \\mid {\\bf X} = {\\bf x}]\\right). \\] The mean is then \\[ \\text{E}[Y \\mid {\\bf X} = {\\bf x}] = g^{-1}(\\eta({\\bf x})). \\] 17.2 Binary Response To illustrate the use of a GLM we’ll focus on the case of binary responses variable coded using \\(0\\) and \\(1\\). In practice, these \\(0\\) and \\(1\\)s will code for two classes such as yes/no, cat/dog, sick/healthy, etc. \\[ Y = \\begin{cases} 1 &amp; \\text{yes} \\\\ 0 &amp; \\text{no} \\end{cases} \\] First, we define some notation that we will use throughout. \\[ p({\\bf x}) = P[Y = 1 \\mid {\\bf X} = {\\bf x}] \\] With a binary (Bernoulli) response, we’ll mostly focus on the case when \\(Y = 1\\), since with only two possibilities, it is trivial to obtain probabilities when \\(Y = 0\\). \\[ P[Y = 0 \\mid {\\bf X} = {\\bf x}] + P[Y = 1 \\mid {\\bf X} = {\\bf x}] = 1 \\] \\[ P[Y = 0 \\mid {\\bf X} = {\\bf x}] = 1 - p({\\bf x}) \\] We now define the logistic regression model. \\[ \\log\\left(\\frac{p({\\bf x})}{1 - p({\\bf x})}\\right) = \\beta_0 + \\beta_1 x_1 + \\ldots + \\beta_{p - 1} x_{p - 1} \\] Immediately we notice some similarities to ordinary linear regression, in particular, the right-hand side. This is our usual linear combination of the predictors. We have our usual \\(p - 1\\) predictors for a total of \\(p\\) \\(\\beta\\) parameters. (Note, many more machine learning focused texts will use \\(p\\) as the number of predictors. This is an arbitrary choice, but you should be aware of it.) The left-hand side is called the log odds, which is the log of the odds. The odds are the probability for a positive event \\((Y = 1)\\) divided by the probability of a negative event \\((Y = 0)\\). So when the odds are \\(1\\), the two events have equal probability. Odds greater than \\(1\\) favor a positive event. The opposite is true when the odds are less than \\(1\\). \\[ \\frac{p({\\bf x})}{1 - p({\\bf x})} = \\frac{P[Y = 1 \\mid {\\bf X} = {\\bf x}]}{P[Y = 0 \\mid {\\bf X} = {\\bf x}]} \\] Essentially, the log odds are the logit transform applied to \\(p({\\bf x})\\). \\[ \\text{logit}(\\xi) = \\log\\left(\\frac{\\xi}{1 - \\xi}\\right) \\] It will also be useful to define the inverse logit, otherwise known as the “logistic” or sigmoid function. \\[ \\text{logit}^{-1}(\\xi) = \\frac{e^\\xi}{1 + e^{\\xi}} = \\frac{1}{1 + e^{-\\xi}} \\] Note that for \\(x \\in (-\\infty, \\infty))\\), this function outputs values between 0 and 1. Students often ask, where is the error term? The answer is that it’s something that is specific to the normal model. First notice that the model with the error term, \\[ Y = \\beta_0 + \\beta_1x_1 + \\ldots + \\beta_qx_q + \\epsilon, \\ \\ \\epsilon \\sim N(0, \\sigma^2) \\] can instead be written as \\[ Y \\mid {\\bf X} = {\\bf x} \\sim N(\\beta_0 + \\beta_1x_1 + \\ldots + \\beta_qx_q, \\ \\sigma^2). \\] While our main focus is on estimating the mean, \\(\\beta_0 + \\beta_1x_1 + \\ldots + \\beta_qx_q\\), there is also another parameter, \\(\\sigma^2\\) which needs to be estimated. This is the result of the normal distribution having two parameters. With logistic regression, which uses the Bernoulli distribution, we only need to estimate the Bernoulli distribution’s single parameter \\(p({\\bf x})\\), which happens to be its mean. \\[ \\log\\left(\\frac{p({\\bf x})}{1 - p({\\bf x})}\\right) = \\beta_0 + \\beta_1 x_1 + \\ldots + \\beta_{q} x_{q} \\] So even though we introduced ordinary linear regression first, in some ways, logistic regression is actually simpler. Note that applying the inverse logit transformation allow us to obtain an expression for \\(p({\\bf x})\\). \\[ p({\\bf x}) = P[Y = 1 \\mid {\\bf X} = {\\bf x}] = \\frac{e^{\\beta_0 + \\beta_1 x_{1} + \\cdots + \\beta_{p-1} x_{(p-1)}}}{1 + e^{\\beta_0 + \\beta_1 x_{1} + \\cdots + \\beta_{p-1} x_{(p-1)}}} \\] 17.2.1 Fitting Logistic Regression With \\(n\\) observations, we write the model indexed with \\(i\\) to note that it is being applied to each observation. \\[ \\log\\left(\\frac{p({\\bf x_i})}{1 - p({\\bf x_i})}\\right) = \\beta_0 + \\beta_1 x_{i1} + \\cdots + \\beta_{p-1} x_{i(p-1)} \\] We can apply the inverse logit transformation to obtain \\(P[Y_i = 1 \\mid {\\bf X_i} = {\\bf x_i}]\\) for each observation. Since these are probabilities, it’s good that we used a function that returns values between \\(0\\) and \\(1\\). \\[ p({\\bf x_i}) = P[Y_i = 1 \\mid {\\bf X_i} = {\\bf x_i}] = \\frac{e^{\\beta_0 + \\beta_1 x_{i1} + \\cdots + \\beta_{p-1} x_{i(p-1)}}}{1 + e^{\\beta_0 + \\beta_1 x_{i1} + \\cdots + \\beta_{p-1} x_{i(p-1)}}} \\] \\[ 1 - p({\\bf x_i}) = P[Y_i = 0 \\mid {\\bf X} = {\\bf x_i}] = \\frac{1}{1 + e^{\\beta_0 + \\beta_1 x_{i1} + \\cdots + \\beta_{p-1} x_{i(p-1)}}} \\] To “fit” this model, that is estimate the \\(\\beta\\) parameters, we will use maximum likelihood. \\[ \\boldsymbol{{\\beta}} = [\\beta_0, \\beta_1, \\beta_2, \\beta_3, \\ldots, \\beta_{p - 1}] \\] We first write the likelihood given the observed data. \\[ L(\\boldsymbol{{\\beta}}) = \\prod_{i = 1}^{n} P[Y_i = y_i \\mid {\\bf X_i} = {\\bf x_i}] \\] This is already technically a function of the \\(\\beta\\) parameters, but we’ll do some rearrangement to make this more explicit. \\[ L(\\boldsymbol{{\\beta}}) = \\prod_{i = 1}^{n} p({\\bf x_i})^{y_i} (1 - p({\\bf x_i}))^{(1 - y_i)} \\] \\[ L(\\boldsymbol{{\\beta}}) = \\prod_{i : y_i = 1}^{n} p({\\bf x_i}) \\prod_{j : y_j = 0}^{n} (1 - p({\\bf x_j})) \\] \\[ L(\\boldsymbol{{\\beta}}) = \\prod_{i : y_i = 1}^{} \\frac{e^{\\beta_0 + \\beta_1 x_{i1} + \\cdots + \\beta_{p-1} x_{i(p-1)}}}{1 + e^{\\beta_0 + \\beta_1 x_{i1} + \\cdots + \\beta_{p-1} x_{i(p-1)}}} \\prod_{j : y_j = 0}^{} \\frac{1}{1 + e^{\\beta_0 + \\beta_1 x_{j1} + \\cdots + \\beta_{p-1} x_{j(p-1)}}} \\] Unfortunately, unlike ordinary linear regression, there is no analytical solution for this maximization problem. Instead, it will need to be solved numerically. Fortunately, R will take care of this for us using an iteratively reweighted least squares algorithm. (We’ll leave the details for a machine learning or optimization course, which would likely also discuss alternative optimization strategies.) 17.2.2 Fitting Issues We should note that, if there exists some \\(\\beta^*\\) such that \\[ {\\bf x_i}^{\\top} \\boldsymbol{{\\beta}^*} &gt; 0 \\implies y_i = 1 \\] and \\[ {\\bf x_i}^{\\top} \\boldsymbol{{\\beta}^*} &lt; 0 \\implies y_i = 0 \\] for all observations, then the MLE is not unique. Such data is said to be separable. This, and similar numeric issues related to estimated probabilities near 0 or 1, will return a warning in R: ## Warning: glm.fit: fitted probabilities numerically 0 or 1 occurred When this happens, the model is still “fit,” but there are consequences, namely, the estimated coefficients are highly suspect. This is an issue when then trying to interpret the model. When this happens, the model will often still be useful for creating a classifier, which will be discussed later. However, it is still subject to the usual evaluations for classifiers to determine how well it is performing. For details, see Modern Applied Statistics with S-PLUS, Chapter 7. 17.2.3 Simulation Examples sim_logistic_data = function(sample_size = 25, beta_0 = -2, beta_1 = 3) { x = rnorm(n = sample_size) eta = beta_0 + beta_1 * x p = 1 / (1 + exp(-eta)) y = rbinom(n = sample_size, size = 1, prob = p) data.frame(y, x) } You might think, why not simply use ordinary linear regression? Even with a binary response, our goal is still to model (some function of) \\(\\text{E}[Y \\mid {\\bf X} = {\\bf x}]\\). However, with a binary response coded as \\(0\\) and \\(1\\), \\(\\text{E}[Y \\mid {\\bf X} = {\\bf x}] = P[Y = 1 \\mid {\\bf X} = {\\bf x}]\\) since \\[ \\begin{aligned} \\text{E}[Y \\mid {\\bf X} = {\\bf x}] &amp;= 1 \\cdot P[Y = 1 \\mid {\\bf X} = {\\bf x}] + 0 \\cdot P[Y = 0 \\mid {\\bf X} = {\\bf x}] \\\\ &amp;= P[Y = 1 \\mid {\\bf X} = {\\bf x}] \\end{aligned} \\] Then why can’t we just use ordinary linear regression to estimate \\(\\text{E}[Y \\mid {\\bf X} = {\\bf x}]\\), and thus \\(P[Y = 1 \\mid {\\bf X} = {\\bf x}]\\)? To investigate, let’s simulate data from the following model: \\[ \\log\\left(\\frac{p({\\bf x})}{1 - p({\\bf x})}\\right) = -2 + 3 x \\] Another way to write this, which better matches the function we’re using to simulate the data: \\[ \\begin{aligned} Y_i \\mid {\\bf X_i} = {\\bf x_i} &amp;\\sim \\text{Bern}(p_i) \\\\ p_i &amp;= p({\\bf x_i}) = \\frac{1}{1 + e^{-\\eta({\\bf x_i})}} \\\\ \\eta({\\bf x_i}) &amp;= -2 + 3 x_i \\end{aligned} \\] set.seed(1) example_data = sim_logistic_data() head(example_data) ## y x ## 1 0 -0.6264538 ## 2 1 0.1836433 ## 3 0 -0.8356286 ## 4 1 1.5952808 ## 5 0 0.3295078 ## 6 0 -0.8204684 After simulating a dataset, we’ll then fit both ordinary linear regression and logistic regression. Notice that currently the responses variable y is a numeric variable that only takes values 0 and 1. Later we’ll see that we can also fit logistic regression when the response is a factor variable with only two levels. (Generally, having a factor response is preferred, but having a dummy response allows us to make the comparison to using ordinary linear regression.) # ordinary linear regression fit_lm = lm(y ~ x, data = example_data) # logistic regression fit_glm = glm(y ~ x, data = example_data, family = binomial) Notice that the syntax is extremely similar. What’s changed? lm() has become glm() We’ve added family = binomial argument In a lot of ways, lm() is just a more specific version of glm(). For example glm(y ~ x, data = example_data) would actually fit the ordinary linear regression that we have seen in the past. By default, glm() uses family = gaussian argument. That is, we’re fitting a GLM with a normally distributed response and the identity function as the link. The family argument to glm() actually specifies both the distribution and the link function. If not made explicit, the link function is chosen to be the canonical link function, which is essentially the most mathematical convenient link function. See ?glm and ?family for details. For example, the following code explicitly specifies the link function which was previously used by default. # more detailed call to glm for logistic regression fit_glm = glm(y ~ x, data = example_data, family = binomial(link = &quot;logit&quot;)) Making predictions with an object of type glm is slightly different than making predictions after fitting with lm(). In the case of logistic regression, with family = binomial, we have: type Returned \"link\" [default] \\(\\hat{\\eta}({\\bf x}) = \\log\\left(\\frac{\\hat{p}({\\bf x})}{1 - \\hat{p}({\\bf x})}\\right)\\) \"response\" \\(\\hat{p}({\\bf x}) = \\frac{e^{\\hat{\\eta}({\\bf x})}}{1 + e^{\\hat{\\eta}({\\bf x})}} = \\frac{1}{1 + e^{-\\hat{\\eta}({\\bf x})}}\\) That is, type = \"link\" will get you the log odds, while type = \"response\" will return the estimated mean, in this case, \\(P[Y = 1 \\mid {\\bf X} = {\\bf x}]\\) for each observation. predict(fit_glm, data.frame(x = 1.2)) # type = &quot;link&quot; as default ## 1 ## 2.076166 predict(fit_glm, data.frame(x = 1.2), type = &quot;link&quot;) ## 1 ## 2.076166 predict(fit_glm, data.frame(x = 1.2), type = &quot;response&quot;) ## 1 ## 0.888565 plot(y ~ x, data = example_data, pch = 20, ylab = &quot;Estimated Probability&quot;, main = &quot;Ordinary vs Logistic Regression&quot;) grid() abline(fit_lm, col = &quot;darkorange&quot;) curve(predict(fit_glm, data.frame(x), type = &quot;response&quot;), add = TRUE, col = &quot;dodgerblue&quot;, lty = 2) legend(&quot;topleft&quot;, c(&quot;Ordinary&quot;, &quot;Logistic&quot;, &quot;Data&quot;), lty = c(1, 2, 0), pch = c(NA, NA, 20), lwd = 2, col = c(&quot;darkorange&quot;, &quot;dodgerblue&quot;, &quot;black&quot;)) Since we only have a single predictor variable, we are able to graphically show this situation. First, note that the data, is plotted using black dots. The response y only takes values 0 and 1. Next, we need to discuss the two added lines to the plot. The first, the solid orange line, is the fitted ordinary linear regression. The dashed blue curve is the estimated logistic regression. It is helpful to realize that we are not plotting an estimate of \\(Y\\) for either. (Sometimes it might seem that way with ordinary linear regression, but that isn’t what is happening.) For both, we are plotting \\(\\hat{\\text{E}}[Y \\mid {\\bf X} = {\\bf x}]\\), the estimated mean, which for a binary response happens to be an estimate of \\(P[Y = 1 \\mid {\\bf X} = {\\bf x}]\\). We immediately see why ordinary linear regression is not a good idea. While it is estimating the mean, we see that it produces estimates that are less than 0! (And in other situations could produce estimates greater than 1!) If the mean is a probability, we don’t want probabilities less than 0 or greater than 1. Enter logistic regression. Since the output of the inverse logit function is restricted to be between 0 and 1, our estimates make much more sense as probabilities. Let’s look at our estimated coefficients. (With a lot of rounding, for simplicity.) round(coef(fit_glm), 1) ## (Intercept) x ## -2.3 3.7 Our estimated model is then: \\[ \\log\\left(\\frac{\\hat{p}({\\bf x})}{1 - \\hat{p}({\\bf x})}\\right) = -2.3 + 3.7 x \\] Because we’re not directly estimating the mean, but instead a function of the mean, we need to be careful with our interpretation of \\(\\hat{\\beta}_1 = 3.7\\). This means that, for a one unit increase in \\(x\\), the log odds change (in this case increase) by \\(3.7\\). Also, since \\(\\hat{\\beta}_1\\) is positive, as we increase \\(x\\) we also increase \\(\\hat{p}({\\bf x})\\). To see how much, we have to consider the inverse logistic function. For example, we have: \\[ \\hat{P}[Y = 1 \\mid X = -0.5] = \\frac{e^{-2.3 + 3.7 \\cdot (-0.5)}}{1 + e^{-2.3 + 3.7 \\cdot (-0.5)}} \\approx 0.016 \\] \\[ \\hat{P}[Y = 1 \\mid X = 0] = \\frac{e^{-2.3 + 3.7 \\cdot (0)}}{1 + e^{-2.3 + 3.7 \\cdot (0)}} \\approx 0.09112296 \\] \\[ \\hat{P}[Y = 1 \\mid X = 1] = \\frac{e^{-2.3 + 3.7 \\cdot (1)}}{1 + e^{-2.3 + 3.7 \\cdot (1)}} \\approx 0.8021839 \\] Now that we know we should use logistic regression, and not ordinary linear regression, let’s consider another example. This time, let’s consider the model \\[ \\log\\left(\\frac{p({\\bf x})}{1 - p({\\bf x})}\\right) = 1 + -4 x. \\] Again, we could re-write this to better match the function we’re using to simulate the data: \\[ \\begin{aligned} Y_i \\mid {\\bf X_i} = {\\bf x_i} &amp;\\sim \\text{Bern}(p_i) \\\\ p_i &amp;= p({\\bf x_i}) = \\frac{1}{1 + e^{-\\eta({\\bf x_i})}} \\\\ \\eta({\\bf x_i}) &amp;= 1 + -4 x_i \\end{aligned} \\] In this model, as \\(x\\) increases, the log odds decrease. set.seed(1) example_data = sim_logistic_data(sample_size = 50, beta_0 = 1, beta_1 = -4) We again simulate some observations form this model, then fit logistic regression. fit_glm = glm(y ~ x, data = example_data, family = binomial) plot(y ~ x, data = example_data, pch = 20, ylab = &quot;Estimated Probability&quot;, main = &quot;Logistic Regression, Decreasing Probability&quot;) grid() curve(predict(fit_glm, data.frame(x), type = &quot;response&quot;), add = TRUE, col = &quot;dodgerblue&quot;, lty = 2) curve(boot::inv.logit(1 - 4 * x), add = TRUE, col = &quot;darkorange&quot;, lty = 1) legend(&quot;bottomleft&quot;, c(&quot;True Probability&quot;, &quot;Estimated Probability&quot;, &quot;Data&quot;), lty = c(1, 2, 0), pch = c(NA, NA, 20), lwd = 2, col = c(&quot;darkorange&quot;, &quot;dodgerblue&quot;, &quot;black&quot;)) We see that this time, as \\(x\\) increases, \\(\\hat{p}({\\bf x})\\) decreases. Now let’s look at an example where the estimated probability doesn’t always simply increase or decrease. Much like ordinary linear regression, the linear combination of predictors can contain transformations of predictors (in this case a quadratic term) and interactions. sim_quadratic_logistic_data = function(sample_size = 25) { x = rnorm(n = sample_size) eta = -1.5 + 0.5 * x + x ^ 2 p = 1 / (1 + exp(-eta)) y = rbinom(n = sample_size, size = 1, prob = p) data.frame(y, x) } \\[ \\log\\left(\\frac{p({\\bf x})}{1 - p({\\bf x})}\\right) = -1.5 + 0.5x + x^2. \\] Again, we could re-write this to better match the function we’re using to simulate the data: \\[ \\begin{aligned} Y_i \\mid {\\bf X_i} = {\\bf x_i} &amp;\\sim \\text{Bern}(p_i) \\\\ p_i &amp;= p({\\bf x_i}) = \\frac{1}{1 + e^{-\\eta({\\bf x_i})}} \\\\ \\eta({\\bf x_i}) &amp;= -1.5 + 0.5x_i + x_i^2 \\end{aligned} \\] set.seed(42) example_data = sim_quadratic_logistic_data(sample_size = 50) fit_glm = glm(y ~ x + I(x^2), data = example_data, family = binomial) plot(y ~ x, data = example_data, pch = 20, ylab = &quot;Estimated Probability&quot;, main = &quot;Logistic Regression, Quadratic Relationship&quot;) grid() curve(predict(fit_glm, data.frame(x), type = &quot;response&quot;), add = TRUE, col = &quot;dodgerblue&quot;, lty = 2) curve(boot::inv.logit(-1.5 + 0.5 * x + x ^ 2), add = TRUE, col = &quot;darkorange&quot;, lty = 1) legend(&quot;bottomleft&quot;, c(&quot;True Probability&quot;, &quot;Estimated Probability&quot;, &quot;Data&quot;), lty = c(1, 2, 0), pch = c(NA, NA, 20), lwd = 2, col = c(&quot;darkorange&quot;, &quot;dodgerblue&quot;, &quot;black&quot;)) 17.3 Working with Logistic Regression While the logistic regression model isn’t exactly the same as the ordinary linear regression model, because they both use a linear combination of the predictors \\[ \\eta({\\bf x}) = \\beta_0 + \\beta_1 x_1 + \\beta_2 x_2 + \\ldots + \\beta_{p - 1} x_{p - 1} \\] working with logistic regression is very similar. Many of the things we did with ordinary linear regression can be done with logistic regression in a very similar fashion. For example, Testing for a single \\(\\beta\\) parameter Testing for a set of \\(\\beta\\) parameters Formula specification in R Interpreting parameters and estimates Confidence intervals for parameters Confidence intervals for mean response Variable selection After some introduction to the new tests, we’ll demonstrate each of these using an example. 17.3.1 Testing with GLMs Like ordinary linear regression, we’ll want to be able to perform hypothesis testing. We’ll again want both single parameter, and multiple parameter tests. 17.3.2 Wald Test In ordinary linear regression, we performed the test of \\[ H_0: \\beta_j = 0 \\quad \\text{vs} \\quad H_1: \\beta_j \\neq 0 \\] using a \\(t\\)-test. For the logistic regression model, \\[ \\log\\left(\\frac{p({\\bf x})}{1 - p({\\bf x})}\\right) = \\beta_0 + \\beta_1 x_1 + \\ldots + \\beta_{p - 1} x_{p - 1} \\] we can again perform a test of \\[ H_0: \\beta_j = 0 \\quad \\text{vs} \\quad H_1: \\beta_j \\neq 0 \\] however, the test statistic and its distribution are no longer \\(t\\). We see that the test statistic takes the same form \\[ z = \\frac{\\hat{\\beta}_j - \\beta_j}{\\text{SE}[\\hat{\\beta}_j]} \\overset{\\text{approx}}{\\sim} N(0, 1) \\] but now we are performing a \\(z\\)-test, as the test statistic is approximated by a standard normal distribution, provided we have a large enough sample. (The \\(t\\)-test for ordinary linear regression, assuming the assumptions were correct, had an exact distribution for any sample size.) We’ll skip some of the exact details of the calculations, as R will obtain the standard error for us. The use of this test will be extremely similar to the \\(t\\)-test for ordinary linear regression. Essentially the only thing that changes is the distribution of the test statistic. 17.3.3 Likelihood-Ratio Test Consider the following full model, \\[ \\log\\left(\\frac{p({\\bf x_i})}{1 - p({\\bf x_i})}\\right) = \\beta_0 + \\beta_1 x_{i1} + \\beta_2 x_{i2} + \\cdots + \\beta_{(p-1)} x_{i(p-1)} \\] This model has \\(p - 1\\) predictors, for a total of \\(p\\) \\(\\beta\\)-parameters. We will denote the MLE of these \\(\\beta\\)-parameters as \\(\\hat{\\beta}_{\\text{Full}}\\) Now consider a null (or reduced) model, \\[ \\log\\left(\\frac{p({\\bf x_i})}{1 - p({\\bf x_i})}\\right) = \\beta_0 + \\beta_1 x_{i1} + \\beta_2 x_{i2} + \\cdots + \\beta_{(q-1)} x_{i(q-1)} \\] where \\(q &lt; p\\). This model has \\(q - 1\\) predictors, for a total of \\(q\\) \\(\\beta\\)-parameters. We will denote the MLE of these \\(\\beta\\)-parameters as \\(\\hat{\\beta}_{\\text{Null}}\\) The difference between these two models can be codified by the null hypothesis of a test. \\[ H_0: \\beta_q = \\beta_{q+1} = \\cdots = \\beta_{p - 1} = 0. \\] This implies that the reduced model is nested inside the full model. We then define a test statistic, \\(D\\), \\[ D = -2 \\log \\left( \\frac{L(\\boldsymbol{\\hat{\\beta}_{\\text{Null}}})} {L(\\boldsymbol{\\hat{\\beta}_{\\text{Full}}})} \\right) = 2 \\log \\left( \\frac{L(\\boldsymbol{\\hat{\\beta}_{\\text{Full}}})} {L(\\boldsymbol{\\hat{\\beta}_{\\text{Null}}})} \\right) = 2 \\left( \\ell(\\hat{\\beta}_{\\text{Full}}) - \\ell(\\hat{\\beta}_{\\text{Null}})\\right) \\] where \\(L\\) denotes a likelihood and \\(\\ell\\) denotes a log-likelihood. For a large enough sample, this test statistic has an approximate Chi-square distribution \\[ D \\overset{\\text{approx}}{\\sim} \\chi^2_{k} \\] where \\(k = p - q\\), the difference in number of parameters of the two models. This test, which we will call the Likelihood-Ratio Test, will be the analogue to the ANOVA \\(F\\)-test for logistic regression. Interestingly, to perform the Likelihood-Ratio Test, we’ll actually again use the anova() function in R!. The Likelihood-Ratio Test is actually a rather general test, however, here we have presented a specific application to nested logistic regression models. 17.3.4 SAheart Example To illustrate the use of logistic regression, we will use the SAheart dataset from the bestglm package. # &quot;leaps&quot; is required for &quot;bestglm&quot; if (!require(&quot;leaps&quot;)) install.packages(&quot;leaps&quot;) if (!require(&quot;bestglm&quot;)) install.packages(&quot;bestglm&quot;) library(leaps) library(bestglm) data(&quot;SAheart&quot;) sbp tobacco ldl adiposity famhist typea obesity alcohol age chd 160 12.00 5.73 23.11 Present 49 25.30 97.20 52 1 144 0.01 4.41 28.61 Absent 55 28.87 2.06 63 1 118 0.08 3.48 32.28 Present 52 29.14 3.81 46 0 170 7.50 6.41 38.03 Present 51 31.99 24.26 58 1 134 13.60 3.50 27.78 Present 60 25.99 57.34 49 1 132 6.20 6.47 36.21 Present 62 30.77 14.14 45 0 This data comes from a retrospective sample of males in a heart-disease high-risk region of the Western Cape, South Africa. The chd variable, which we will use as a response, indicates whether or not coronary heart disease is present in an individual. Note that this is coded as a numeric 0 / 1 variable. Using this as a response with glm() it is important to indicate family = binomial, otherwise ordinary linear regression will be fit. Later, we will see the use of a factor variable response, which is actually preferred, as you cannot accidentally fit ordinary linear regression. The predictors are various measurements for each individual, many related to heart health. For example sbp, systolic blood pressure, and ldl, low density lipoprotein cholesterol. For full details, use ?SAheart. We’ll begin by attempting to model the probability of coronary heart disease based on low density lipoprotein cholesterol. That is, we will fit the model \\[ \\log\\left(\\frac{P[\\texttt{chd} = 1]}{1 - P[\\texttt{chd} = 1]}\\right) = \\beta_0 + \\beta_{\\texttt{ldl}} x_{\\texttt{ldl}} \\] chd_mod_ldl = glm(chd ~ ldl, data = SAheart, family = binomial) plot(jitter(chd, factor = 0.1) ~ ldl, data = SAheart, pch = 20, ylab = &quot;Probability of CHD&quot;, xlab = &quot;Low Density Lipoprotein Cholesterol&quot;) grid() curve(predict(chd_mod_ldl, data.frame(ldl = x), type = &quot;response&quot;), add = TRUE, col = &quot;dodgerblue&quot;, lty = 2) As before, we plot the data in addition to the estimated probabilities. Note that we have “jittered” the data to make it easier to visualize, but the data do only take values 0 and 1. As we would expect, this plot indicates that as ldl increases, so does the probability of chd. coef(summary(chd_mod_ldl)) ## Estimate Std. Error z value Pr(&gt;|z|) ## (Intercept) -1.9686681 0.27307908 -7.209150 5.630207e-13 ## ldl 0.2746613 0.05163983 5.318787 1.044615e-07 To perform the test \\[ H_0: \\beta_{\\texttt{ldl}} = 0 \\] we use the summary() function as we have done so many times before. Like the \\(t\\)-test for ordinary linear regression, this returns the estimate of the parameter, its standard error, the relevant test statistic (\\(z\\)), and its p-value. Here we have an incredibly low p-value, so we reject the null hypothesis. The ldl variable appears to be a significant predictor. When fitting logistic regression, we can use the same formula syntax as ordinary linear regression. So, to fit an additive model using all available predictors, we use: chd_mod_additive = glm(chd ~ ., data = SAheart, family = binomial) We can then use the likelihood-ratio test to compare the two models. Specifically, we are testing \\[ H_0: \\beta_{\\texttt{sbp}} = \\beta_{\\texttt{tobacco}} = \\beta_{\\texttt{adiposity}} = \\beta_{\\texttt{famhist}} = \\beta_{\\texttt{typea}} = \\beta_{\\texttt{obesity}} = \\beta_{\\texttt{alcohol}} = \\beta_{\\texttt{age}} = 0 \\] We could manually calculate the test statistic, -2 * as.numeric(logLik(chd_mod_ldl) - logLik(chd_mod_additive)) ## [1] 92.13879 Or we could utilize the anova() function. By specifying test = \"LRT\", R will use the likelihood-ratio test to compare the two models. anova(chd_mod_ldl, chd_mod_additive, test = &quot;LRT&quot;) ## Analysis of Deviance Table ## ## Model 1: chd ~ ldl ## Model 2: chd ~ sbp + tobacco + ldl + adiposity + famhist + typea + obesity + ## alcohol + age ## Resid. Df Resid. Dev Df Deviance Pr(&gt;Chi) ## 1 460 564.28 ## 2 452 472.14 8 92.139 &lt; 2.2e-16 *** ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 We see that the test statistic that we had just calculated appears in the output. The very small p-value suggests that we prefer the larger model. While we prefer the additive model compared to the model with only a single predictor, do we actually need all of the predictors in the additive model? To select a subset of predictors, we can use a stepwise procedure as we did with ordinary linear regression. Recall that AIC and BIC were defined in terms of likelihoods. Here we demonstrate using AIC with a backwards selection procedure. chd_mod_selected = step(chd_mod_additive, trace = 0) coef(chd_mod_selected) ## (Intercept) tobacco ldl famhistPresent typea ## -6.44644451 0.08037533 0.16199164 0.90817526 0.03711521 ## age ## 0.05046038 We could again compare this model to the additive models. \\[ H_0: \\beta_{\\texttt{sbp}} = \\beta_{\\texttt{adiposity}} = \\beta_{\\texttt{obesity}} = \\beta_{\\texttt{alcohol}} = 0 \\] anova(chd_mod_selected, chd_mod_additive, test = &quot;LRT&quot;) ## Analysis of Deviance Table ## ## Model 1: chd ~ tobacco + ldl + famhist + typea + age ## Model 2: chd ~ sbp + tobacco + ldl + adiposity + famhist + typea + obesity + ## alcohol + age ## Resid. Df Resid. Dev Df Deviance Pr(&gt;Chi) ## 1 456 475.69 ## 2 452 472.14 4 3.5455 0.471 Here it seems that we would prefer the selected model. 17.3.5 Confidence Intervals We can create confidence intervals for the \\(\\beta\\) parameters using the confint() function as we did with ordinary linear regression. confint(chd_mod_selected, level = 0.99) ## Waiting for profiling to be done... ## 0.5 % 99.5 % ## (Intercept) -8.941825274 -4.18278990 ## tobacco 0.015704975 0.14986616 ## ldl 0.022923610 0.30784590 ## famhistPresent 0.330033483 1.49603366 ## typea 0.006408724 0.06932612 ## age 0.024847330 0.07764277 Note that we could create intervals by rearranging the results of the Wald test to obtain the Wald confidence interval. This would be given by \\[ \\hat{\\beta}_j \\pm z_{\\alpha/2} \\cdot \\text{SE}[\\hat{\\beta}_j]. \\] However, R is using a slightly different approach based on a concept called the profile likelihood. (The details of which we will omit.) Ultimately the intervals reported will be similar, but the method used by R is more common in practice, probably at least partially because it is the default approach in R. Check to see how intervals using the formula above compare to those from the output of confint(). (Or, note that using confint.default() will return the results of calculating the Wald confidence interval.) 17.3.6 Confidence Intervals for Mean Response Confidence intervals for the mean response require some additional thought. With a “large enough” sample, we have \\[ \\frac{\\hat{\\eta}({\\bf x}) - \\eta({\\bf x})}{\\text{SE}[\\hat{\\eta}({\\bf x})]} \\overset{\\text{approx}}{\\sim} N(0, 1) \\] Then we can create an approximate \\((1 - \\alpha)\\%\\) confidence intervals for \\(\\eta({\\bf x})\\) using \\[ \\hat{\\eta}({\\bf x}) \\pm z_{\\alpha/2} \\cdot \\text{SE}[\\hat{\\eta}({\\bf x})] \\] where \\(z_{\\alpha/2}\\) is the critical value such that \\(P(Z &gt; z_{\\alpha/2}) = \\alpha/2\\). This isn’t a particularly interesting interval. Instead, what we really want is an interval for the mean response, \\(p({\\bf x})\\). To obtain an interval for \\(p({\\bf x})\\), we simply apply the inverse logit transform to the endpoints of the interval for \\(\\eta.\\) \\[ \\left(\\text{logit}^{-1}(\\hat{\\eta}({\\bf x}) - z_{\\alpha/2} \\cdot \\text{SE}[\\hat{\\eta}({\\bf x})] ), \\ \\text{logit}^{-1}(\\hat{\\eta}({\\bf x}) + z_{\\alpha/2} \\cdot \\text{SE}[\\hat{\\eta}({\\bf x})])\\right) \\] To demonstrate creating these intervals, we’ll consider a new observation. new_obs = data.frame( sbp = 148.0, tobacco = 5, ldl = 12, adiposity = 31.23, famhist = &quot;Present&quot;, typea = 47, obesity = 28.50, alcohol = 23.89, age = 60 ) First, we’ll use the predict() function to obtain \\(\\hat{\\eta}({\\bf x})\\) for this observation. eta_hat = predict(chd_mod_selected, new_obs, se.fit = TRUE, type = &quot;link&quot;) eta_hat ## $fit ## 1 ## 1.579545 ## ## $se.fit ## [1] 0.4114796 ## ## $residual.scale ## [1] 1 By setting se.fit = TRUE, R also computes \\(\\text{SE}[\\hat{\\eta}({\\bf x})]\\). Note that we used type = \"link\", but this is actually a default value. We added it here to stress that the output from predict() will be the value of the link function. z_crit = qnorm(0.975) round(z_crit, 2) ## [1] 1.96 After obtaining the correct critical value, we can easily create a \\(95\\%\\) confidence interval for \\(\\eta({\\bf x})\\). eta_hat$fit + c(-1, 1) * z_crit * eta_hat$se.fit ## [1] 0.7730599 2.3860302 Now we simply need to apply the correct transformation to make this a confidence interval for \\(p({\\bf x})\\), the probability of coronary heart disease for this observation. Note that the boot package contains functions logit() and inv.logit() which are the logit and inverse logit transformations, respectively. boot::inv.logit(eta_hat$fit + c(-1, 1) * z_crit * eta_hat$se.fit) ## [1] 0.6841824 0.9157558 Notice, as we would expect, the bounds of this interval are both between 0 and 1. Also, since both bounds of the interval for \\(\\eta({\\bf x})\\) are positive, both bounds of the interval for \\(p({\\bf x})\\) are greater than 0.5. 17.3.7 Formula Syntax Without really thinking about it, we’ve been using our previous knowledge of R’s model formula syntax to fit logistic regression. 17.3.7.1 Interactions Let’s add an interaction between LDL and family history for the model we selected. chd_mod_interaction = glm(chd ~ alcohol + ldl + famhist + typea + age + ldl:famhist, data = SAheart, family = binomial) summary(chd_mod_interaction) ## ## Call: ## glm(formula = chd ~ alcohol + ldl + famhist + typea + age + ldl:famhist, ## family = binomial, data = SAheart) ## ## Coefficients: ## Estimate Std. Error z value Pr(&gt;|z|) ## (Intercept) -6.043472 0.937186 -6.449 1.13e-10 *** ## alcohol 0.003800 0.004332 0.877 0.38033 ## ldl 0.035593 0.071448 0.498 0.61837 ## famhistPresent -0.733836 0.618131 -1.187 0.23515 ## typea 0.036253 0.012172 2.978 0.00290 ** ## age 0.062416 0.009723 6.419 1.37e-10 *** ## ldl:famhistPresent 0.314311 0.114922 2.735 0.00624 ** ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 ## ## (Dispersion parameter for binomial family taken to be 1) ## ## Null deviance: 596.11 on 461 degrees of freedom ## Residual deviance: 477.46 on 455 degrees of freedom ## AIC: 491.46 ## ## Number of Fisher Scoring iterations: 5 Based on the \\(z\\)-test seen in the above summary, this interaction is significant. The effect of LDL on the probability of CHD is different depending on family history. 17.3.7.2 Polynomial Terms Let’s take the previous model, and now add a polynomial term. chd_mod_int_quad = glm(chd ~ alcohol + ldl + famhist + typea + age + ldl:famhist + I(ldl^2), data = SAheart, family = binomial) summary(chd_mod_int_quad) ## ## Call: ## glm(formula = chd ~ alcohol + ldl + famhist + typea + age + ldl:famhist + ## I(ldl^2), family = binomial, data = SAheart) ## ## Coefficients: ## Estimate Std. Error z value Pr(&gt;|z|) ## (Intercept) -6.096747 1.065952 -5.720 1.07e-08 *** ## alcohol 0.003842 0.004350 0.883 0.37716 ## ldl 0.056876 0.214420 0.265 0.79081 ## famhistPresent -0.723769 0.625167 -1.158 0.24698 ## typea 0.036248 0.012171 2.978 0.00290 ** ## age 0.062299 0.009788 6.365 1.95e-10 *** ## I(ldl^2) -0.001587 0.015076 -0.105 0.91617 ## ldl:famhistPresent 0.311615 0.117559 2.651 0.00803 ** ## --- ## Signif. codes: 0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1 ## ## (Dispersion parameter for binomial family taken to be 1) ## ## Null deviance: 596.11 on 461 degrees of freedom ## Residual deviance: 477.45 on 454 degrees of freedom ## AIC: 493.45 ## ## Number of Fisher Scoring iterations: 5 Unsurprisingly, since this additional transformed variable wasn’t intelligently chosen, it is not significant. However, this does allow us to stress the fact that the syntax notation that we had been using with lm() works basically exactly the same for glm(), however now we understand that this is specifying the linear combination of predictions, \\(\\eta({\\bf x})\\). That is, the above fits the model \\[ \\log\\left(\\frac{p({\\bf x})}{1 - p({\\bf x})}\\right) = \\beta_0 + \\beta_{1}x_{\\texttt{alcohol}} + \\beta_{2}x_{\\texttt{ldl}} + \\beta_{3}x_{\\texttt{famhist}} + \\beta_{4}x_{\\texttt{typea}} + \\beta_{5}x_{\\texttt{age}} + \\beta_{6}x_{\\texttt{ldl}}x_{\\texttt{famhist}} + \\beta_{7}x_{\\texttt{ldl}}^2 \\] You may have realized this before we actually explicitly wrote it down! 17.3.8 Deviance You have probably noticed that the output from summary() is also very similar to that of ordinary linear regression. One difference, is the “deviance” being reported. The Null deviance is the deviance for the null model, that is, a model with no predictors. The Residual deviance is the deviance for the model that was fit. Deviance compares the model to a saturated model. (Without repeated observations, a saturated model is a model that fits perfectly, using a parameter for each observation.) Essentially, deviance is a generalized residual sum of squares for GLMs. Like RSS, deviance decreases as the model complexity increases. deviance(chd_mod_ldl) ## [1] 564.2788 deviance(chd_mod_selected) ## [1] 475.6856 deviance(chd_mod_additive) ## [1] 472.14 Note that these are nested, and we see that deviance does decrease as the model size becomes larger. So while a lower deviance is better, if the model becomes too big, it may be overfitting. Note that R also outputs AIC in the summary, which will penalize according to model size, to prevent overfitting. 17.4 Classification So far we’ve mostly used logistic regression to estimate class probabilities. The somewhat obvious next step is to use these probabilities to make “predictions,” which in this context, we would call classifications. Based on the values of the predictors, should an observation be classified as \\(Y = 1\\) or as \\(Y = 0\\)? Suppose we didn’t need to estimate probabilities from data, and instead, we actually knew both \\[ p({\\bf x}) = P[Y = 1 \\mid {\\bf X} = {\\bf x}] \\] and \\[ 1 - p({\\bf x}) = P[Y = 0 \\mid {\\bf X} = {\\bf x}]. \\] With this information, classifying observations based on the values of the predictors is actually extremely easy. Simply classify an observation to the class (\\(0\\) or \\(1\\)) with the larger probability. In general, this result is called the Bayes Classifier, \\[ C^B({\\bf x}) = \\underset{k}{\\mathrm{argmax}} \\ P[Y = k \\mid {\\bf X = x}]. \\] For a binary response, that is, \\[ \\hat{C}(\\bf x) = \\begin{cases} 1 &amp; p({\\bf x}) &gt; 0.5 \\\\ 0 &amp; p({\\bf x}) \\leq 0.5 \\end{cases} \\] Simply put, the Bayes classifier (not to be confused with the Naive Bayes Classifier) minimizes the probability of misclassification by classifying each observation to the class with the highest probability. Unfortunately, in practice, we won’t know the necessary probabilities to directly use the Bayes classifier. Instead we’ll have to use estimated probabilities. So to create a classifier that seeks to minimize misclassifications, we would use, \\[ \\hat{C}({\\bf x}) = \\underset{k}{\\mathrm{argmax}} \\ \\hat{P}[Y = k \\mid {\\bf X = x}]. \\] In the case of a binary response since \\(\\hat{p}({\\bf x}) = 1 - \\hat{p}({\\bf x})\\), this becomes \\[ \\hat{C}(\\bf x) = \\begin{cases} 1 &amp; \\hat{p}({\\bf x}) &gt; 0.5 \\\\ 0 &amp; \\hat{p}({\\bf x}) \\leq 0.5 \\end{cases} \\] Using this simple classification rule, we can turn logistic regression into a classifier. To use logistic regression for classification, we first use logistic regression to obtain estimated probabilities, \\(\\hat{p}({\\bf x})\\), then use these in conjunction with the above classification rule. Logistic regression is just one of many ways that these probabilities could be estimated. In a course completely focused on machine learning, you’ll learn many additional ways to do this, as well as methods to directly make classifications without needing to first estimate probabilities. But since we had already introduced logistic regression, it makes sense to discuss it in the context of classification. 17.4.1 spam Example To illustrate the use of logistic regression as a classifier, we will use the spam dataset from the kernlab package. # install.packages(&quot;kernlab&quot;) library(kernlab) data(&quot;spam&quot;) tibble::as.tibble(spam) ## Warning: `as.tibble()` was deprecated in tibble 2.0.0. ## ℹ Please use `as_tibble()` instead. ## ℹ The signature and semantics have changed, see `?as_tibble`. ## This warning is displayed once every 8 hours. ## Call `lifecycle::last_lifecycle_warnings()` to see where this warning was ## generated. ## # A tibble: 4,601 × 58 ## make address all num3d our over remove internet order mail receive ## &lt;dbl&gt; &lt;dbl&gt; &lt;dbl&gt; &lt;dbl&gt; &lt;dbl&gt; &lt;dbl&gt; &lt;dbl&gt; &lt;dbl&gt; &lt;dbl&gt; &lt;dbl&gt; &lt;dbl&gt; ## 1 0 0.64 0.64 0 0.32 0 0 0 0 0 0 ## 2 0.21 0.28 0.5 0 0.14 0.28 0.21 0.07 0 0.94 0.21 ## 3 0.06 0 0.71 0 1.23 0.19 0.19 0.12 0.64 0.25 0.38 ## 4 0 0 0 0 0.63 0 0.31 0.63 0.31 0.63 0.31 ## 5 0 0 0 0 0.63 0 0.31 0.63 0.31 0.63 0.31 ## 6 0 0 0 0 1.85 0 0 1.85 0 0 0 ## 7 0 0 0 0 1.92 0 0 0 0 0.64 0.96 ## 8 0 0 0 0 1.88 0 0 1.88 0 0 0 ## 9 0.15 0 0.46 0 0.61 0 0.3 0 0.92 0.76 0.76 ## 10 0.06 0.12 0.77 0 0.19 0.32 0.38 0 0.06 0 0 ## # ℹ 4,591 more rows ## # ℹ 47 more variables: will &lt;dbl&gt;, people &lt;dbl&gt;, report &lt;dbl&gt;, addresses &lt;dbl&gt;, ## # free &lt;dbl&gt;, business &lt;dbl&gt;, email &lt;dbl&gt;, you &lt;dbl&gt;, credit &lt;dbl&gt;, ## # your &lt;dbl&gt;, font &lt;dbl&gt;, num000 &lt;dbl&gt;, money &lt;dbl&gt;, hp &lt;dbl&gt;, hpl &lt;dbl&gt;, ## # george &lt;dbl&gt;, num650 &lt;dbl&gt;, lab &lt;dbl&gt;, labs &lt;dbl&gt;, telnet &lt;dbl&gt;, ## # num857 &lt;dbl&gt;, data &lt;dbl&gt;, num415 &lt;dbl&gt;, num85 &lt;dbl&gt;, technology &lt;dbl&gt;, ## # num1999 &lt;dbl&gt;, parts &lt;dbl&gt;, pm &lt;dbl&gt;, direct &lt;dbl&gt;, cs &lt;dbl&gt;, … This dataset, created in the late 1990s at Hewlett-Packard Labs, contains 4601 emails, of which 1813 are considered spam. The remaining are not spam. (Which for simplicity, we might call, ham.) Additional details can be obtained by using ?spam or by visiting the UCI Machine Learning Repository. The response variable, type, is a factor with levels that label each email as spam or nonspam. When fitting models, nonspam will be the reference level, \\(Y = 0\\), as it comes first alphabetically. is.factor(spam$type) ## [1] TRUE levels(spam$type) ## [1] &quot;nonspam&quot; &quot;spam&quot; Many of the predictors (often called features in machine learning) are engineered based on the emails. For example, charDollar is the number of times an email contains the $ character. Some variables are highly specific to this dataset, for example george and num650. (The name and area code for one of the researchers whose emails were used.) We should keep in mind that this dataset was created based on emails sent to academic type researchers in the 1990s. Any results we derive probably won’t generalize to modern emails for the general public. To get started, we’ll first test-train split the data. set.seed(42) # spam_idx = sample(nrow(spam), round(nrow(spam) / 2)) spam_idx = sample(nrow(spam), 1000) spam_trn = spam[spam_idx, ] spam_tst = spam[-spam_idx, ] We’ve used a somewhat small train set relative to the total size of the dataset. In practice it should likely be larger, but this is simply to keep training time low for illustration and rendering of this document. fit_caps = glm(type ~ capitalTotal, data = spam_trn, family = binomial) fit_selected = glm(type ~ edu + money + capitalTotal + charDollar, data = spam_trn, family = binomial) fit_additive = glm(type ~ ., data = spam_trn, family = binomial) fit_over = glm(type ~ capitalTotal * (.), data = spam_trn, family = binomial, maxit = 75) We’ll fit four logistic regressions, each more complex than the previous. Note that we’re suppressing two warnings. The first we briefly mentioned previously. ## Warning: glm.fit: fitted probabilities numerically 0 or 1 occurred Note that, when we receive this warning, we should be highly suspicious of the parameter estimates. coef(fit_selected) ## (Intercept) edu money capitalTotal charDollar ## -1.1199744712 -1.9837988840 0.9784675298 0.0007757011 11.5772904667 However, the model can still be used to create a classifier, and we will evaluate that classifier on its own merits. We also, “suppressed” the warning: ## Warning: glm.fit: algorithm did not converge In reality, we didn’t actually suppress it, but instead changed maxit to 75, when fitting the model fit_over. This was enough additional iterations to allow the iteratively reweighted least squares algorithm to converge when fitting the model. 17.4.2 Evaluating Classifiers The metric we’ll be most interested in for evaluating the overall performance of a classifier is the misclassification rate. (Sometimes, instead accuracy is reported, which is instead the proportion of correct classifications, so both metrics serve the same purpose.) \\[ \\text{Misclass}(\\hat{C}, \\text{Data}) = \\frac{1}{n}\\sum_{i = 1}^{n}I(y_i \\neq \\hat{C}({\\bf x_i})) \\] \\[ I(y_i \\neq \\hat{C}({\\bf x_i})) = \\begin{cases} 0 &amp; y_i = \\hat{C}({\\bf x_i}) \\\\ 1 &amp; y_i \\neq \\hat{C}({\\bf x_i}) \\\\ \\end{cases} \\] When using this metric on the training data, it will have the same issues as RSS did for ordinary linear regression, that is, it will only go down. # training misclassification rate mean(ifelse(predict(fit_caps) &gt; 0, &quot;spam&quot;, &quot;nonspam&quot;) != spam_trn$type) ## [1] 0.339 mean(ifelse(predict(fit_selected) &gt; 0, &quot;spam&quot;, &quot;nonspam&quot;) != spam_trn$type) ## [1] 0.224 mean(ifelse(predict(fit_additive) &gt; 0, &quot;spam&quot;, &quot;nonspam&quot;) != spam_trn$type) ## [1] 0.066 mean(ifelse(predict(fit_over) &gt; 0, &quot;spam&quot;, &quot;nonspam&quot;) != spam_trn$type) ## [1] 0.118 Because of this, training data isn’t useful for evaluating, as it would suggest that we should always use the largest possible model, when in reality, that model is likely overfitting. Recall, a model that is too complex will overfit. A model that is too simple will underfit. (We’re looking for something in the middle.) To overcome this, we’ll use cross-validation as we did with ordinary linear regression, but this time we’ll cross-validate the misclassification rate. To do so, we’ll use the cv.glm() function from the boot library. It takes arguments for the data (in this case training), a model fit via glm(), and K, the number of folds. It returns delta, a vector of the average mean-squared error (MSE) and the bias-corrected MSE. See ?cv.glm for details. Previously, for cross-validating RMSE in ordinary linear regression, we used LOOCV. We certainly could do that here. However, with logistic regression, we no longer have the clever trick that would allow us to obtain a LOOCV metric without needing to fit the model \\(n\\) times. So instead, we’ll use 5-fold cross-validation. (5 and 10 fold are the most common in practice.) Instead of leaving a single observation out repeatedly, we’ll leave out a fifth of the data. Essentially we’ll repeat the following process 5 times: Randomly set aside a fifth of the data (each observation will only be held-out once) Train model on remaining data Evaluate misclassification rate on held-out data The 5-fold cross-validated misclassification rate will be the average of these misclassification rates. By only needing to refit the model 5 times, instead of \\(n\\) times, we will save a lot of computation time. library(boot) set.seed(1) cv.glm(spam_trn, fit_caps, K = 5)$delta[1] ## [1] 0.2166961 cv.glm(spam_trn, fit_selected, K = 5)$delta[1] ## [1] 0.1587043 cv.glm(spam_trn, fit_additive, K = 5)$delta[1] ## [1] 0.08684467 cv.glm(spam_trn, fit_over, K = 5)$delta[1] ## [1] 0.153 Note that we’re suppressing warnings again here. (Now there would be a lot more, since were fitting a total of 20 models.) Based on these results, fit_caps and fit_selected are underfitting relative to fit_additive. Similarly, fit_over is overfitting relative to fit_additive. Thus, based on these results, we prefer the classifier created based on the logistic regression fit and stored in fit_additive. Going forward, to evaluate and report on the efficacy of this classifier, we’ll use the test dataset. We’re going to take the position that the test data set should never be used in training, which is why we used cross-validation within the training dataset to select a model. Even though cross-validation uses hold-out sets to generate metrics, at some point all of the data is used for training. To quickly summarize how well this classifier works, we’ll create a confusion matrix. Confusion Matrix It further breaks down the classification errors into false positives and false negatives. make_conf_mat = function(predicted, actual) { table(predicted = predicted, actual = actual) } Let’s explicitly store the predicted values of our classifier on the test dataset. spam_tst_pred = ifelse(predict(fit_additive, spam_tst) &gt; 0, &quot;spam&quot;, &quot;nonspam&quot;) spam_tst_pred = ifelse(predict(fit_additive, spam_tst, type = &quot;response&quot;) &gt; 0.5, &quot;spam&quot;, &quot;nonspam&quot;) The previous two lines of code produce the same output, that is the same predictions, since \\[ \\eta({\\bf x}) = 0 \\iff p({\\bf x}) = 0.5 \\] Now we’ll use these predictions to create a confusion matrix. (conf_mat_50 = make_conf_mat(predicted = spam_tst_pred, actual = spam_tst$type)) ## actual ## predicted nonspam spam ## nonspam 2057 157 ## spam 127 1260 \\[ \\text{Prev} = \\frac{\\text{P}}{\\text{Total Obs}}= \\frac{\\text{TP + FN}}{\\text{Total Obs}} \\] table(spam_tst$type) / nrow(spam_tst) ## ## nonspam spam ## 0.6064982 0.3935018 First, note that to be a reasonable classifier, it needs to outperform the obvious classifier of simply classifying all observations to the majority class. In this case, classifying everything as non-spam for a test misclassification rate of 0.3935018 Next, we can see that using the classifier created from fit_additive, only a total of \\(127 + 157 = 284\\) from the total of 3601 emails in the test set are misclassified. Overall, the accuracy in the test set it mean(spam_tst_pred == spam_tst$type) ## [1] 0.921133 In other words, the test misclassification is mean(spam_tst_pred != spam_tst$type) ## [1] 0.07886698 This seems like a decent classifier… However, are all errors created equal? In this case, absolutely not. The 127 non-spam emails that were marked as spam (false positives) are a problem. We can’t allow important information, say, a job offer, to miss our inbox and get sent to the spam folder. On the other hand, the 157 spam email that would make it to an inbox (false negatives) are easily dealt with, just delete them. Instead of simply evaluating a classifier based on its misclassification rate (or accuracy), we’ll define two additional metrics, sensitivity and specificity. Note that these are simply two of many more metrics that can be considered. The Wikipedia page for sensitivity and specificity details a large number of metrics that can be derived from a confusion matrix. Sensitivity is essentially the true positive rate. So when sensitivity is high, the number of false negatives is low. \\[ \\text{Sens} = \\text{True Positive Rate} = \\frac{\\text{TP}}{\\text{P}} = \\frac{\\text{TP}}{\\text{TP + FN}} \\] Here we have an R function to calculate the sensitivity based on the confusion matrix. Note that this function is good for illustrative purposes, but is easily broken. (Think about what happens if there are no “positives” predicted.) get_sens = function(conf_mat) { conf_mat[2, 2] / sum(conf_mat[, 2]) } Specificity is essentially the true negative rate. So when specificity is high, the number of false positives is low. \\[ \\text{Spec} = \\text{True Negative Rate} = \\frac{\\text{TN}}{\\text{N}} = \\frac{\\text{TN}}{\\text{TN + FP}} \\] get_spec = function(conf_mat) { conf_mat[1, 1] / sum(conf_mat[, 1]) } We calculate both based on the confusion matrix we had created for our classifier. get_sens(conf_mat_50) ## [1] 0.8892025 get_spec(conf_mat_50) ## [1] 0.9418498 Recall that we had created this classifier using a probability of \\(0.5\\) as a “cutoff” for how observations should be classified. Now we’ll modify this cutoff. We’ll see that by modifying the cutoff, \\(c\\), we can improve sensitivity or specificity at the expense of the overall accuracy (misclassification rate). \\[ \\hat{C}(\\bf x) = \\begin{cases} 1 &amp; \\hat{p}({\\bf x}) &gt; c \\\\ 0 &amp; \\hat{p}({\\bf x}) \\leq c \\end{cases} \\] Additionally, if we change the cutoff to improve sensitivity, we’ll decrease specificity, and vice versa. First let’s see what happens when we lower the cutoff from \\(0.5\\) to \\(0.1\\) to create a new classifier, and thus new predictions. spam_tst_pred_10 = ifelse(predict(fit_additive, spam_tst, type = &quot;response&quot;) &gt; 0.1, &quot;spam&quot;, &quot;nonspam&quot;) This is essentially decreasing the threshold for an email to be labeled as spam, so far more emails will be labeled as spam. We see that in the following confusion matrix. (conf_mat_10 = make_conf_mat(predicted = spam_tst_pred_10, actual = spam_tst$type)) ## actual ## predicted nonspam spam ## nonspam 1583 29 ## spam 601 1388 Unfortunately, while this does greatly reduce false negatives, false positives have almost quadrupled. We see this reflected in the sensitivity and specificity. get_sens(conf_mat_10) ## [1] 0.9795342 get_spec(conf_mat_10) ## [1] 0.7248168 This classifier, using \\(0.1\\) instead of \\(0.5\\) has a higher sensitivity, but a much lower specificity. Clearly, we should have moved the cutoff in the other direction. Let’s try \\(0.9\\). spam_tst_pred_90 = ifelse(predict(fit_additive, spam_tst, type = &quot;response&quot;) &gt; 0.9, &quot;spam&quot;, &quot;nonspam&quot;) This is essentially increasing the threshold for an email to be labeled as spam, so far fewer emails will be labeled as spam. Again, we see that in the following confusion matrix. (conf_mat_90 = make_conf_mat(predicted = spam_tst_pred_90, actual = spam_tst$type)) ## actual ## predicted nonspam spam ## nonspam 2136 537 ## spam 48 880 This is the result we’re looking for. We have far fewer false positives. While sensitivity is greatly reduced, specificity has gone up. get_sens(conf_mat_90) ## [1] 0.6210303 get_spec(conf_mat_90) ## [1] 0.978022 While this is far fewer false positives, is it acceptable though? Still probably not. Also, don’t forget, this would actually be a terrible spam detector today since this is based on data from a very different era of the internet, for a very specific set of people. Spam has changed a lot since the 90s! (Ironically, machine learning is probably partially to blame.) This chapter has provided a rather quick introduction to classification, and thus, machine learning. For a more complete coverage of machine learning, An Introduction to Statistical Learning is a highly recommended resource. Additionally, R for Statistical Learning has been written as a supplement which provides additional detail on how to perform these methods using R. The classification and logistic regression chapters might be useful. We should note that the code to perform classification using logistic regression is presented in a way that illustrates the concepts to the reader. In practice, you may prefer to use a more general machine learning pipeline such as caret in R. This will streamline processes for creating predictions and generating evaluation metrics. 17.5 R Markdown The R Markdown file for this chapter can be found here: logistic.Rmd The file was created using R version 4.4.1. "],["beyond.html", "Chapter 18 Beyond 18.1 What’s Next 18.2 RStudio 18.3 Tidy Data 18.4 Visualization 18.5 Web Applications 18.6 Experimental Design 18.7 Machine Learning 18.8 Time Series 18.9 Bayesianism 18.10 High Performance Computing 18.11 Further R Resources", " Chapter 18 Beyond “End? No, the journey doesn’t end here.” — J.R.R. Tolkien After reading this chapter you will be able to: Understand the roadmap to continued education about models and the R programming language. 18.1 What’s Next So you’ve completed STAT 420, where do you go from here? Now that you understand the basics of linear modeling, there is a wide world of applied statistics waiting to be explored. We’ll briefly detail some resources and discuss how they relate to what you have learned in STAT 420. 18.2 RStudio RStudio has recently released version 1.0! This is exciting for a number of reasons, especially the release of R Notebooks. R Notebooks combine the RMarkdown you have already learned with the ability to work interactively. 18.3 Tidy Data In this textbook, much of the data we have seen has been nice and tidy. It was rectangular where each row was an observation and each column was a variable. This is not always the case! Many packages have been developed to deal with data, and force it into a nice format, which is called tidy data, that we can then use for modeling. Often during analysis, this is where a large portion of your time will be spent. The R community has started to call this collection of packages the Tidyverse. It was once called the Hadleyverse, as Hadley Wickham has authored so many of the packages. Hadley is writing a book called R for Data Science which describes the use of many of these packages. (And also how to use some to make the modeling process better!) This book is a great starting point for diving deeper into the R community. The two main packages are dplyr and tidyr both of which are used internally in RStudio. 18.4 Visualization In this course, we have mostly used the base plotting methods in R. When working with tidy data, many users prefer to use the ggplot2 package, also developed by Hadley Wickham. RStudio provides a rather detailed “cheat sheet” for working with ggplot2. The community maintains a graph gallery of examples. Use of the manipulate package with RStudio gives the ability to quickly change a static graphic to become interactive. 18.5 Web Applications RStudio has made it incredibly easy to create data products through the use of Shiny, which allows for the creation of web applications with R. RStudio maintains an ever-growing tutorial and gallery of examples. 18.6 Experimental Design In the ANOVA chapter, we briefly discussed experimental design. This topic could easily be its own class, and is currently an area of revitalized interest with the rise of A/B testing. Two more classic statistical references include Statistics for Experimenters by Box, Hunter, and Hunter as well as Design and Analysis of Experiments by Douglas Montgomery. There are several R packages for design of experiments, listed in the CRAN Task View. 18.7 Machine Learning Using models for prediction is the key focus of machine learning. There are many methods, each with its own package, however R has a wonderful package called caret, Classification And REgression Training, which provides a unified interface to training these models. It also contains various utilities for data processing and visualization that are useful for predictive modeling. Applied Predictive Modeling by Max Kuhn, the author of the caret package, is a good general resource for predictive modeling, which obviously utilizes R. An Introduction to Statistical Learning by James, Witten, Hastie, and Tibshirani is a gentle introduction to machine learning from a statistical perspective which uses R and picks up right where this course stops. This is based on the often referenced The Elements of Statistical Learning by Hastie, Tibshirani, and Friedman. Both are freely available online. 18.7.1 Deep Learning While it probably isn’t the best tool for the job, R now has the ability to train deep neural networks via TensorFlow. 18.8 Time Series In this class we have only considered independent data. What if data is dependent? Time Series is the area of statistics which deals with this issue, and could easily span multiple courses. The primary textbook for STAT 429: Time Series Analysis at the University of Illinois that is free is: Time Series Analysis and Its Applications: With R Examples by Shumway and Stoffer Some tutorials: Little Book of R for Time Series Quick R: Time Series and Forecasting TSA: Start to Finish Examples When performing time series analysis in R you should be aware of the many packages that are useful for analysis. It should be hard to avoid the forecast and zoo packages. Often the most difficult part will be dealing with time and date data. Make sure you are utilizing one of the many packages that help with this. 18.9 Bayesianism In this class, we have worked within the frequentist view of statistics. There is an entire alternative universe of Bayesian statistics. Doing Bayesian Data Analysis: A Tutorial with R, JAGS, and Stan by John Kruschke is a great introduction to the topic. It introduces the world of probabilistic programming, in particular Stan, which can be used in both R and Python. 18.10 High Performance Computing Often R will be called a “slow” language, for two reasons. One, because many do not understand R. Two, because sometimes it really is. Luckily, it is easy to extend R via the Rcpp package to allow for faster code. Many modern R packages utilize Rcpp to achieve better performance. 18.11 Further R Resources Also, don’t forget that previously in this book we have outlined a large number of R resources. Now that you’ve gotten started with R many of these will be much more useful. If any of these topics interest you, and you would like more information, please don’t hesitate to start a discussion on the forums! :) "],["appendix.html", "Chapter 19 Appendix", " Chapter 19 Appendix "],["404.html", "Page not found", " Page not found The page you requested cannot be found (perhaps it was moved or renamed). You may want to try searching to find the page's new location, or use the table of contents to find the page you are looking for. "]]
diff --git a/selection.md b/selection.md
index accd342..5525d11 100644
--- a/selection.md
+++ b/selection.md
@@ -135,7 +135,7 @@ Each of the previous three metrics explicitly used $p$, the number of parameters
 We'll now briefly introduce **overfitting** and **cross-validation**.
 
 
-```r
+``` r
 make_poly_data = function(sample_size = 11) {
   x = seq(0, 10)
   y = 3 + x + 4 * x ^ 2 + rnorm(n = sample_size, mean = 0, sd = 20)
@@ -144,7 +144,7 @@ make_poly_data = function(sample_size = 11) {
 ```
 
 
-```r
+``` r
 set.seed(1234)
 poly_data = make_poly_data()
 ```
@@ -158,7 +158,7 @@ Y = 3 + x + 4 x ^ 2 + \epsilon.
 We'll now fit two models to this data, one which has the correct form, quadratic, and one that is large, which includes terms up to and including an eighth degree.
 
 
-```r
+``` r
 fit_quad = lm(y ~ poly(x, degree = 2), data = poly_data)
 fit_big  = lm(y ~ poly(x, degree = 8), data = poly_data)
 ```
@@ -166,7 +166,7 @@ fit_big  = lm(y ~ poly(x, degree = 8), data = poly_data)
 We then plot the data and the results of the two models.
 
 
-```r
+``` r
 plot(y ~ x, data = poly_data, ylim = c(-100, 400), cex = 2, pch = 20)
 xplot = seq(0, 10, by = 0.1)
 lines(xplot, predict(fit_quad, newdata = data.frame(x = xplot)),
@@ -184,7 +184,7 @@ We can see that the solid blue curve models this data rather nicely. The dashed
 We see that the larger model indeed has a lower $\text{RMSE}$.
 
 
-```r
+``` r
 sqrt(mean(resid(fit_quad) ^ 2))
 ```
 
@@ -192,7 +192,7 @@ sqrt(mean(resid(fit_quad) ^ 2))
 ## [1] 17.61812
 ```
 
-```r
+``` r
 sqrt(mean(resid(fit_big) ^ 2))
 ```
 
@@ -231,7 +231,7 @@ where $h_i$ are the leverages and $e_i$ are the usual residuals. This is great,
 Let's calculate LOOCV $\text{RMSE}$ for both models, then discuss *why* we want to do so. We first write a function which calculates the LOOCV $\text{RMSE}$ as defined using the shortcut formula for linear models.
 
 
-```r
+``` r
 calc_loocv_rmse = function(model) {
   sqrt(mean((resid(model) / (1 - hatvalues(model))) ^ 2))
 }
@@ -240,7 +240,7 @@ calc_loocv_rmse = function(model) {
 Then calculate the metric for both models.
 
 
-```r
+``` r
 calc_loocv_rmse(fit_quad)
 ```
 
@@ -248,7 +248,7 @@ calc_loocv_rmse(fit_quad)
 ## [1] 23.57189
 ```
 
-```r
+``` r
 calc_loocv_rmse(fit_big)
 ```
 
@@ -259,7 +259,7 @@ calc_loocv_rmse(fit_big)
 Now we see that the quadratic model has a much smaller LOOCV $\text{RMSE}$, so we would prefer this quadratic model. This is because the large model has *severely* over-fit the data. By leaving a single data point out and fitting the large model, the resulting fit is much different than the fit using all of the data. For example, let's leave out the third data point and fit both models, then plot the result.
 
 
-```r
+``` r
 fit_quad_removed = lm(y ~ poly(x, degree = 2), data = poly_data[-3, ])
 fit_big_removed  = lm(y ~ poly(x, degree = 8), data = poly_data[-3, ])
 
@@ -284,7 +284,7 @@ This is the purpose of cross-validation. By assessing how the model fits points
 We've now seen a number of model quality criteria, but now we need to address which models to consider. Model selection involves both a quality criterion, plus a search procedure. 
 
 
-```r
+``` r
 library(faraway)
 hipcenter_mod = lm(hipcenter ~ ., data = seatpos)
 coef(hipcenter_mod)
@@ -312,7 +312,7 @@ If we had 10 or more predictors, we would already be considering over 1000 model
 Backward selection procedures start with all possible predictors in the model, then consider how deleting a single predictor will affect a chosen metric. Let's try this on the `seatpos` data. We will use the `step()` function in `R` which by default uses $\text{AIC}$ as its metric of choice.
 
 
-```r
+``` r
 hipcenter_mod_back_aic = step(hipcenter_mod, direction = "backward")
 ```
 
@@ -401,7 +401,7 @@ In this example, at the first step, the current model is `hipcenter ~ Age + Weig
 which we quickly verify.
 
 
-```r
+``` r
 extractAIC(hipcenter_mod) # returns both p and AIC
 ```
 
@@ -409,7 +409,7 @@ extractAIC(hipcenter_mod) # returns both p and AIC
 ## [1]   9.000 283.624
 ```
 
-```r
+``` r
 n = length(resid(hipcenter_mod))
 (p = length(coef(hipcenter_mod)))
 ```
@@ -418,7 +418,7 @@ n = length(resid(hipcenter_mod))
 ## [1] 9
 ```
 
-```r
+``` r
 n * log(mean(resid(hipcenter_mod) ^ 2)) + 2 * p
 ```
 
@@ -435,7 +435,7 @@ Notice, in the second step, we start with the model `hipcenter ~ Age + Weight +
 We continue the process until we reach the model `hipcenter ~ Age + HtShoes + Leg`. At this step, the row for `<none>` tops the list, as removing any additional variable will not improve the $\text{AIC}$ This is the model which is stored in `hipcenter_mod_back_aic`.
 
 
-```r
+``` r
 coef(hipcenter_mod_back_aic)
 ```
 
@@ -447,7 +447,7 @@ coef(hipcenter_mod_back_aic)
 We could also search through the possible models in a backwards fashion using $\text{BIC}$. To do so, we again use the `step()` function, but now specify `k = log(n)`, where `n` stores the number of observations in the data.
 
 
-```r
+``` r
 n = length(resid(hipcenter_mod))
 hipcenter_mod_back_bic = step(hipcenter_mod, direction = "backward", k = log(n))
 ```
@@ -544,7 +544,7 @@ The procedure is exactly the same, except at each step we look to improve the $\
 The variable `hipcenter_mod_back_bic` stores the model chosen by this procedure.
 
 
-```r
+``` r
 coef(hipcenter_mod_back_bic)
 ```
 
@@ -558,7 +558,7 @@ We note that this model is *smaller*, has fewer predictors, than the model chose
 We can use information from the `summary()` function to compare their Adjusted $R^2$ values. Note that either selected model performs better than the original full model.
 
 
-```r
+``` r
 summary(hipcenter_mod)$adj.r.squared
 ```
 
@@ -566,7 +566,7 @@ summary(hipcenter_mod)$adj.r.squared
 ## [1] 0.6000855
 ```
 
-```r
+``` r
 summary(hipcenter_mod_back_aic)$adj.r.squared
 ```
 
@@ -574,7 +574,7 @@ summary(hipcenter_mod_back_aic)$adj.r.squared
 ## [1] 0.6531427
 ```
 
-```r
+``` r
 summary(hipcenter_mod_back_bic)$adj.r.squared
 ```
 
@@ -585,7 +585,7 @@ summary(hipcenter_mod_back_bic)$adj.r.squared
 We can also calculate the LOOCV $\text{RMSE}$ for both selected models, as well as the full model.
 
 
-```r
+``` r
 calc_loocv_rmse(hipcenter_mod)
 ```
 
@@ -593,7 +593,7 @@ calc_loocv_rmse(hipcenter_mod)
 ## [1] 44.44564
 ```
 
-```r
+``` r
 calc_loocv_rmse(hipcenter_mod_back_aic)
 ```
 
@@ -601,7 +601,7 @@ calc_loocv_rmse(hipcenter_mod_back_aic)
 ## [1] 37.58473
 ```
 
-```r
+``` r
 calc_loocv_rmse(hipcenter_mod_back_bic)
 ```
 
@@ -616,7 +616,7 @@ We see that we would prefer the model chosen via $\text{BIC}$ if using LOOCV $\t
 Forward selection is the exact opposite of backwards selection. Here we tell `R` to start with a model using no predictors, that is `hipcenter ~ 1`, then at each step `R` will attempt to add a predictor until it finds a good model or reaches `hipcenter ~ Age + Weight + HtShoes + Ht + Seated + Arm + Thigh + Leg`.
 
 
-```r
+``` r
 hipcenter_mod_start = lm(hipcenter ~ 1, data = seatpos)
 hipcenter_mod_forw_aic = step(
   hipcenter_mod_start, 
@@ -679,7 +679,7 @@ hipcenter_mod_forw_aic = step(
 Again, by default `R` uses $\text{AIC}$ as its quality metric when using the `step()` function. Also note that now the rows begin with a `+` which indicates addition of predictors to the current model from any step.
 
 
-```r
+``` r
 hipcenter_mod_forw_bic = step(
   hipcenter_mod_start, 
   scope = hipcenter ~ Age + Weight + HtShoes + Ht + Seated + Arm + Thigh + Leg, 
@@ -718,7 +718,7 @@ hipcenter_mod_forw_bic = step(
 We can make the same modification as last time to instead use $\text{BIC}$ with forward selection.
 
 
-```r
+``` r
 summary(hipcenter_mod)$adj.r.squared
 ```
 
@@ -726,7 +726,7 @@ summary(hipcenter_mod)$adj.r.squared
 ## [1] 0.6000855
 ```
 
-```r
+``` r
 summary(hipcenter_mod_forw_aic)$adj.r.squared
 ```
 
@@ -734,7 +734,7 @@ summary(hipcenter_mod_forw_aic)$adj.r.squared
 ## [1] 0.6533055
 ```
 
-```r
+``` r
 summary(hipcenter_mod_forw_bic)$adj.r.squared
 ```
 
@@ -745,7 +745,7 @@ summary(hipcenter_mod_forw_bic)$adj.r.squared
 We can compare the two selected models' Adjusted $R^2$ as well as their LOOCV $\text{RMSE}$ The results are very similar to those using backwards selection, although the models are not exactly the same.
 
 
-```r
+``` r
 calc_loocv_rmse(hipcenter_mod)
 ```
 
@@ -753,7 +753,7 @@ calc_loocv_rmse(hipcenter_mod)
 ## [1] 44.44564
 ```
 
-```r
+``` r
 calc_loocv_rmse(hipcenter_mod_forw_aic)
 ```
 
@@ -761,7 +761,7 @@ calc_loocv_rmse(hipcenter_mod_forw_aic)
 ## [1] 37.62516
 ```
 
-```r
+``` r
 calc_loocv_rmse(hipcenter_mod_forw_bic)
 ```
 
@@ -776,7 +776,7 @@ Stepwise search checks going both backwards and forwards at every step. It consi
 Here we perform stepwise search using $\text{AIC}$ as our metric. We start with the model `hipcenter ~ 1` and search up to `hipcenter ~ Age + Weight + HtShoes + Ht + Seated + Arm + Thigh + Leg`. Notice that at many of the steps, some row begin with `-`, while others begin with `+`.
 
 
-```r
+``` r
 hipcenter_mod_both_aic = step(
   hipcenter_mod_start, 
   scope = hipcenter ~ Age + Weight + HtShoes + Ht + Seated + Arm + Thigh + Leg, 
@@ -844,7 +844,7 @@ hipcenter_mod_both_aic = step(
 We could again instead use $\text{BIC}$ as our metric.
 
 
-```r
+``` r
 hipcenter_mod_both_bic = step(
   hipcenter_mod_start, 
   scope = hipcenter ~ Age + Weight + HtShoes + Ht + Seated + Arm + Thigh + Leg, 
@@ -884,7 +884,7 @@ hipcenter_mod_both_bic = step(
 Adjusted $R^2$ and LOOCV $\text{RMSE}$ comparisons are similar to backwards and forwards, which is not at all surprising, as some of the models selected are the same as before.
 
 
-```r
+``` r
 summary(hipcenter_mod)$adj.r.squared
 ```
 
@@ -892,7 +892,7 @@ summary(hipcenter_mod)$adj.r.squared
 ## [1] 0.6000855
 ```
 
-```r
+``` r
 summary(hipcenter_mod_both_aic)$adj.r.squared
 ```
 
@@ -900,7 +900,7 @@ summary(hipcenter_mod_both_aic)$adj.r.squared
 ## [1] 0.6533055
 ```
 
-```r
+``` r
 summary(hipcenter_mod_both_bic)$adj.r.squared
 ```
 
@@ -909,7 +909,7 @@ summary(hipcenter_mod_both_bic)$adj.r.squared
 ```
 
 
-```r
+``` r
 calc_loocv_rmse(hipcenter_mod)
 ```
 
@@ -917,7 +917,7 @@ calc_loocv_rmse(hipcenter_mod)
 ## [1] 44.44564
 ```
 
-```r
+``` r
 calc_loocv_rmse(hipcenter_mod_both_aic)
 ```
 
@@ -925,7 +925,7 @@ calc_loocv_rmse(hipcenter_mod_both_aic)
 ## [1] 37.62516
 ```
 
-```r
+``` r
 calc_loocv_rmse(hipcenter_mod_both_bic)
 ```
 
@@ -940,7 +940,7 @@ Backward, forward, and stepwise search are all useful, but do have an obvious is
 However, with a reasonably sized dataset, it isn't too difficult to check all possible models. To do so, we will use the `regsubsets()` function in the `R` package `leaps`.
 
 
-```r
+``` r
 library(leaps)
 all_hipcenter_mod = summary(regsubsets(hipcenter ~ ., data = seatpos))
 ```
@@ -950,7 +950,7 @@ A few points about this line of code. First, note that we immediately use `summa
 We'll now look at the information stored in `all_hipcenter_mod`.
 
 
-```r
+``` r
 all_hipcenter_mod$which
 ```
 
@@ -969,7 +969,7 @@ all_hipcenter_mod$which
 Using `$which` gives us the best model, according to $\text{RSS}$, for a model of each possible size, in this case ranging from one to eight predictors. For example the best model with four predictors ($p = 5$) would use `Age`, `HtShoes`, `Thigh`, and `Leg`.
 
 
-```r
+``` r
 all_hipcenter_mod$rss
 ```
 
@@ -984,7 +984,7 @@ Now that we have the $\text{RSS}$ for each of these models, it is rather easy to
 Conveniently, Adjusted $R^2$ is automatically calculated.
 
 
-```r
+``` r
 all_hipcenter_mod$adjr2
 ```
 
@@ -996,7 +996,7 @@ all_hipcenter_mod$adjr2
 To find which model has the highest Adjusted $R^2$ we can use the `which.max()` function.
 
 
-```r
+``` r
 (best_r2_ind = which.max(all_hipcenter_mod$adjr2))
 ```
 
@@ -1007,7 +1007,7 @@ To find which model has the highest Adjusted $R^2$ we can use the `which.max()`
 We can then extract the predictors of that model.
 
 
-```r
+``` r
 all_hipcenter_mod$which[best_r2_ind, ]
 ```
 
@@ -1021,7 +1021,7 @@ all_hipcenter_mod$which[best_r2_ind, ]
 We'll now calculate $\text{AIC}$ and $\text{BIC}$ for each of the models with the best $\text{RSS}$. To do so, we will need both $n$ and the $p$ for the largest possible model.
 
 
-```r
+``` r
 p = length(coef(hipcenter_mod))
 n = length(resid(hipcenter_mod))
 ```
@@ -1035,14 +1035,14 @@ We'll use the form of $\text{AIC}$ which leaves out the constant term that is eq
 Since we have the $\text{RSS}$ of each model stored, this is easy to calculate.
 
 
-```r
+``` r
 hipcenter_mod_aic = n * log(all_hipcenter_mod$rss / n) + 2 * (2:p)
 ```
 
 We can then extract the predictors of the model with the best $\text{AIC}$.
 
 
-```r
+``` r
 best_aic_ind = which.min(hipcenter_mod_aic)
 all_hipcenter_mod$which[best_aic_ind,]
 ```
@@ -1057,14 +1057,14 @@ all_hipcenter_mod$which[best_aic_ind,]
 Let's fit this model so we can compare to our previously chosen models using $\text{AIC}$ and search procedures.
 
 
-```r
+``` r
 hipcenter_mod_best_aic = lm(hipcenter ~ Age + Ht + Leg, data = seatpos)
 ```
 
 The `extractAIC()` function will calculate the $\text{AIC}$ defined above for a fitted model.
 
 
-```r
+``` r
 extractAIC(hipcenter_mod_best_aic)
 ```
 
@@ -1072,7 +1072,7 @@ extractAIC(hipcenter_mod_best_aic)
 ## [1]   4.0000 274.2418
 ```
 
-```r
+``` r
 extractAIC(hipcenter_mod_back_aic)
 ```
 
@@ -1080,7 +1080,7 @@ extractAIC(hipcenter_mod_back_aic)
 ## [1]   4.0000 274.2597
 ```
 
-```r
+``` r
 extractAIC(hipcenter_mod_forw_aic)
 ```
 
@@ -1088,7 +1088,7 @@ extractAIC(hipcenter_mod_forw_aic)
 ## [1]   4.0000 274.2418
 ```
 
-```r
+``` r
 extractAIC(hipcenter_mod_both_aic)
 ```
 
@@ -1099,7 +1099,7 @@ extractAIC(hipcenter_mod_both_aic)
 We see that two of the models chosen by search procedures have the best possible $\text{AIC}$, as they are the same model. This is however never guaranteed. We see that the model chosen using backwards selection does not achieve the smallest possible $\text{AIC}$.
 
 
-```r
+``` r
 plot(hipcenter_mod_aic ~ I(2:p), ylab = "AIC", xlab = "p, number of parameters", 
      pch = 20, col = "dodgerblue", type = "b", cex = 2,
      main = "AIC vs Model Complexity")
@@ -1116,12 +1116,12 @@ We could easily repeat this process for $\text{BIC}$.
 \]
 
 
-```r
+``` r
 hipcenter_mod_bic = n * log(all_hipcenter_mod$rss / n) + log(n) * (2:p)
 ```
 
 
-```r
+``` r
 which.min(hipcenter_mod_bic)
 ```
 
@@ -1129,7 +1129,7 @@ which.min(hipcenter_mod_bic)
 ## [1] 1
 ```
 
-```r
+``` r
 all_hipcenter_mod$which[1,]
 ```
 
@@ -1141,12 +1141,12 @@ all_hipcenter_mod$which[1,]
 ```
 
 
-```r
+``` r
 hipcenter_mod_best_bic = lm(hipcenter ~ Ht, data = seatpos)
 ```
 
 
-```r
+``` r
 extractAIC(hipcenter_mod_best_bic, k = log(n))
 ```
 
@@ -1154,7 +1154,7 @@ extractAIC(hipcenter_mod_best_bic, k = log(n))
 ## [1]   2.0000 278.3418
 ```
 
-```r
+``` r
 extractAIC(hipcenter_mod_back_bic, k = log(n))
 ```
 
@@ -1162,7 +1162,7 @@ extractAIC(hipcenter_mod_back_bic, k = log(n))
 ## [1]   2.0000 278.7306
 ```
 
-```r
+``` r
 extractAIC(hipcenter_mod_forw_bic, k = log(n))
 ```
 
@@ -1170,7 +1170,7 @@ extractAIC(hipcenter_mod_forw_bic, k = log(n))
 ## [1]   2.0000 278.3418
 ```
 
-```r
+``` r
 extractAIC(hipcenter_mod_both_bic, k = log(n))
 ```
 
@@ -1183,7 +1183,7 @@ extractAIC(hipcenter_mod_both_bic, k = log(n))
 So far we have only allowed first-order terms in our models. Let's return to the `autompg` dataset to explore higher-order terms.
 
 
-```r
+``` r
 autompg = read.table(
   "http://archive.ics.uci.edu/ml/machine-learning-databases/auto-mpg/auto-mpg.data",
   quote = "\"",
@@ -1205,7 +1205,7 @@ autompg = subset(autompg, select = c("mpg", "cyl", "disp", "hp",
 ```
 
 
-```r
+``` r
 str(autompg)
 ```
 
@@ -1224,7 +1224,7 @@ str(autompg)
 Recall that we have two factor variables, `cyl` and `domestic`. The `cyl` variable has three levels, while the `domestic` variable has only two. Thus the `cyl` variable will be coded using two dummy variables, while the `domestic` variable will only need one. We will pay attention to this later.
 
 
-```r
+``` r
 pairs(autompg, col = "dodgerblue")
 ```
 
@@ -1237,7 +1237,7 @@ We'll use the `pairs()` plot to determine which variables may benefit from a qua
 So now, we'll fit this rather large model. We'll use a log-transformed response. Notice that `log(mpg) ~ . ^ 2` will automatically consider all first-order terms, as well as all two-way interactions. We use `I(var_name ^ 2)` to add quadratic terms for some variables. This generally works better than using `poly()` when performing variable selection.
 
 
-```r
+``` r
 autompg_big_mod = lm(
   log(mpg) ~ . ^ 2 + I(disp ^ 2) + I(hp ^ 2) + I(wt ^ 2) + I(acc ^ 2), 
   data = autompg)
@@ -1246,7 +1246,7 @@ autompg_big_mod = lm(
 We think it is rather unlikely that we truly need all of these terms. There are quite a few!
 
 
-```r
+``` r
 length(coef(autompg_big_mod))
 ```
 
@@ -1257,7 +1257,7 @@ length(coef(autompg_big_mod))
 We'll try backwards search with both $\text{AIC}$ and $\text{BIC}$ to attempt to find a smaller, more reasonable model.
 
 
-```r
+``` r
 autompg_mod_back_aic = step(autompg_big_mod, direction = "backward", trace = 0)
 ```
 
@@ -1268,7 +1268,7 @@ You should also notice that `R` respects hierarchy when attempting to remove var
 We also use $\text{BIC}$.
 
 
-```r
+``` r
 n = length(resid(autompg_big_mod))
 autompg_mod_back_bic = step(autompg_big_mod, direction = "backward", 
                             k = log(n), trace = 0)
@@ -1277,7 +1277,7 @@ autompg_mod_back_bic = step(autompg_big_mod, direction = "backward",
 Looking at the coefficients of the two chosen models, we see they are still rather large.
 
 
-```r
+``` r
 coef(autompg_mod_back_aic)
 ```
 
@@ -1292,7 +1292,7 @@ coef(autompg_mod_back_aic)
 ##  -1.838985e-04   2.345625e-03  -2.372468e-02  -7.332725e-03
 ```
 
-```r
+``` r
 coef(autompg_mod_back_bic)
 ```
 
@@ -1308,7 +1308,7 @@ coef(autompg_mod_back_bic)
 However, they are much smaller than the original full model. Also notice that the resulting models respect hierarchy.
 
 
-```r
+``` r
 length(coef(autompg_big_mod))
 ```
 
@@ -1316,7 +1316,7 @@ length(coef(autompg_big_mod))
 ## [1] 40
 ```
 
-```r
+``` r
 length(coef(autompg_mod_back_aic))
 ```
 
@@ -1324,7 +1324,7 @@ length(coef(autompg_mod_back_aic))
 ## [1] 19
 ```
 
-```r
+``` r
 length(coef(autompg_mod_back_bic))
 ```
 
@@ -1335,7 +1335,7 @@ length(coef(autompg_mod_back_bic))
 Calculating the LOOCV $\text{RMSE}$ for each, we see that the model chosen using $\text{BIC}$ performs the best. That means that it is both the best model for prediction, since it achieves the best LOOCV $\text{RMSE}$, but also the best model for explanation, as it is also the smallest.
 
 
-```r
+``` r
 calc_loocv_rmse(autompg_big_mod)
 ```
 
@@ -1343,7 +1343,7 @@ calc_loocv_rmse(autompg_big_mod)
 ## [1] 0.1112024
 ```
 
-```r
+``` r
 calc_loocv_rmse(autompg_mod_back_aic)
 ```
 
@@ -1351,7 +1351,7 @@ calc_loocv_rmse(autompg_mod_back_aic)
 ## [1] 0.1032888
 ```
 
-```r
+``` r
 calc_loocv_rmse(autompg_mod_back_bic)
 ```
 
@@ -1366,7 +1366,7 @@ Throughout this chapter, we have attempted to find reasonably "small" models, wh
 We'll further discuss the model `autompg_mod_back_bic` to better explain the difference between using models for *explaining* and *predicting*. This is the model fit to the `autompg` data that was chosen using Backwards Search and $\text{BIC}$, which obtained the lowest LOOCV $\text{RMSE}$ of the models we considered.
 
 
-```r
+``` r
 autompg_mod_back_bic
 ```
 
@@ -1404,7 +1404,7 @@ A word of caution when using a model to *explain* a relationship. There are two
 Just because two variables are correlated does not necessarily mean that one causes the other. For example, considering modeling `mpg` as only a function of `hp`.
 
 
-```r
+``` r
 plot(mpg ~ hp, data = autompg, col = "dodgerblue", pch = 20, cex = 1.5)
 ```
 
@@ -1436,4 +1436,4 @@ The `R` Markdown file for this chapter can be found here:
 
 - [`selection.Rmd`](selection.Rmd){target="_blank"}
 
-The file was created using `R` version `4.3.2`.
+The file was created using `R` version `4.4.1`.
diff --git a/simple-linear-regression.html b/simple-linear-regression.html
index fd43e91..c54ad4c 100644
--- a/simple-linear-regression.html
+++ b/simple-linear-regression.html
@@ -6,7 +6,7 @@
   <meta http-equiv="X-UA-Compatible" content="IE=edge" />
   <title>Chapter 7 Simple Linear Regression | Applied Statistics with R</title>
   <meta name="description" content="Chapter 7 Simple Linear Regression | Applied Statistics with R" />
-  <meta name="generator" content="bookdown 0.37 and GitBook 2.6.7" />
+  <meta name="generator" content="bookdown 0.40 and GitBook 2.6.7" />
 
   <meta property="og:title" content="Chapter 7 Simple Linear Regression | Applied Statistics with R" />
   <meta property="og:type" content="book" />
@@ -55,7 +55,7 @@
 
 <style type="text/css">
 pre > code.sourceCode { white-space: pre; position: relative; }
-pre > code.sourceCode > span { display: inline-block; line-height: 1.25; }
+pre > code.sourceCode > span { line-height: 1.25; }
 pre > code.sourceCode > span:empty { height: 1.2em; }
 .sourceCode { overflow: visible; }
 code.sourceCode > span { color: inherit; text-decoration: inherit; }
@@ -482,29 +482,29 @@ <h1><span class="header-section-number">Chapter 7</span> Simple Linear Regressio
 <h2><span class="header-section-number">7.1</span> Modeling<a href="simple-linear-regression.html#modeling" class="anchor-section" aria-label="Anchor link to header"></a></h2>
 <p>Let’s consider a simple example of how the speed of a car affects its stopping distance, that is, how far it travels before it comes to a stop. To examine this relationship, we will use the <code>cars</code> dataset, which is a default <code>R</code> dataset. Thus, we don’t need to load a package first; it is immediately available.</p>
 <p>To get a first look at the data you can use the <code>View()</code> function inside RStudio.</p>
-<div class="sourceCode" id="cb422"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb422-1"><a href="simple-linear-regression.html#cb422-1" aria-hidden="true" tabindex="-1"></a><span class="fu">View</span>(cars)</span></code></pre></div>
+<div class="sourceCode" id="cb422"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb422-1"><a href="simple-linear-regression.html#cb422-1" tabindex="-1"></a><span class="fu">View</span>(cars)</span></code></pre></div>
 <p>We could also take a look at the variable names, the dimension of the data frame, and some sample observations with <code>str()</code>.</p>
-<div class="sourceCode" id="cb423"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb423-1"><a href="simple-linear-regression.html#cb423-1" aria-hidden="true" tabindex="-1"></a><span class="fu">str</span>(cars)</span></code></pre></div>
+<div class="sourceCode" id="cb423"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb423-1"><a href="simple-linear-regression.html#cb423-1" tabindex="-1"></a><span class="fu">str</span>(cars)</span></code></pre></div>
 <pre><code>## &#39;data.frame&#39;:    50 obs. of  2 variables:
 ##  $ speed: num  4 4 7 7 8 9 10 10 10 11 ...
 ##  $ dist : num  2 10 4 22 16 10 18 26 34 17 ...</code></pre>
 <p>As we have seen before with data frames, there are a number of additional functions to access some of this information directly.</p>
-<div class="sourceCode" id="cb425"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb425-1"><a href="simple-linear-regression.html#cb425-1" aria-hidden="true" tabindex="-1"></a><span class="fu">dim</span>(cars)</span></code></pre></div>
+<div class="sourceCode" id="cb425"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb425-1"><a href="simple-linear-regression.html#cb425-1" tabindex="-1"></a><span class="fu">dim</span>(cars)</span></code></pre></div>
 <pre><code>## [1] 50  2</code></pre>
-<div class="sourceCode" id="cb427"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb427-1"><a href="simple-linear-regression.html#cb427-1" aria-hidden="true" tabindex="-1"></a><span class="fu">nrow</span>(cars)</span></code></pre></div>
+<div class="sourceCode" id="cb427"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb427-1"><a href="simple-linear-regression.html#cb427-1" tabindex="-1"></a><span class="fu">nrow</span>(cars)</span></code></pre></div>
 <pre><code>## [1] 50</code></pre>
-<div class="sourceCode" id="cb429"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb429-1"><a href="simple-linear-regression.html#cb429-1" aria-hidden="true" tabindex="-1"></a><span class="fu">ncol</span>(cars)</span></code></pre></div>
+<div class="sourceCode" id="cb429"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb429-1"><a href="simple-linear-regression.html#cb429-1" tabindex="-1"></a><span class="fu">ncol</span>(cars)</span></code></pre></div>
 <pre><code>## [1] 2</code></pre>
 <p>Other than the two variable names and the number of observations, this data is still just a bunch of numbers, so we should probably obtain some context.</p>
-<div class="sourceCode" id="cb431"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb431-1"><a href="simple-linear-regression.html#cb431-1" aria-hidden="true" tabindex="-1"></a>?cars</span></code></pre></div>
+<div class="sourceCode" id="cb431"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb431-1"><a href="simple-linear-regression.html#cb431-1" tabindex="-1"></a>?cars</span></code></pre></div>
 <p>Reading the documentation we learn that this is data gathered during the 1920s about the speed of cars and the resulting distance it takes for the car to come to a stop. The interesting task here is to determine how far a car travels before stopping, when traveling at a certain speed. So, we will first plot the stopping distance against the speed.</p>
-<div class="sourceCode" id="cb432"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb432-1"><a href="simple-linear-regression.html#cb432-1" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(dist <span class="sc">~</span> speed, <span class="at">data =</span> cars,</span>
-<span id="cb432-2"><a href="simple-linear-regression.html#cb432-2" aria-hidden="true" tabindex="-1"></a>     <span class="at">xlab =</span> <span class="st">&quot;Speed (in Miles Per Hour)&quot;</span>,</span>
-<span id="cb432-3"><a href="simple-linear-regression.html#cb432-3" aria-hidden="true" tabindex="-1"></a>     <span class="at">ylab =</span> <span class="st">&quot;Stopping Distance (in Feet)&quot;</span>,</span>
-<span id="cb432-4"><a href="simple-linear-regression.html#cb432-4" aria-hidden="true" tabindex="-1"></a>     <span class="at">main =</span> <span class="st">&quot;Stopping Distance vs Speed&quot;</span>,</span>
-<span id="cb432-5"><a href="simple-linear-regression.html#cb432-5" aria-hidden="true" tabindex="-1"></a>     <span class="at">pch  =</span> <span class="dv">20</span>,</span>
-<span id="cb432-6"><a href="simple-linear-regression.html#cb432-6" aria-hidden="true" tabindex="-1"></a>     <span class="at">cex  =</span> <span class="dv">2</span>,</span>
-<span id="cb432-7"><a href="simple-linear-regression.html#cb432-7" aria-hidden="true" tabindex="-1"></a>     <span class="at">col  =</span> <span class="st">&quot;grey&quot;</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb432"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb432-1"><a href="simple-linear-regression.html#cb432-1" tabindex="-1"></a><span class="fu">plot</span>(dist <span class="sc">~</span> speed, <span class="at">data =</span> cars,</span>
+<span id="cb432-2"><a href="simple-linear-regression.html#cb432-2" tabindex="-1"></a>     <span class="at">xlab =</span> <span class="st">&quot;Speed (in Miles Per Hour)&quot;</span>,</span>
+<span id="cb432-3"><a href="simple-linear-regression.html#cb432-3" tabindex="-1"></a>     <span class="at">ylab =</span> <span class="st">&quot;Stopping Distance (in Feet)&quot;</span>,</span>
+<span id="cb432-4"><a href="simple-linear-regression.html#cb432-4" tabindex="-1"></a>     <span class="at">main =</span> <span class="st">&quot;Stopping Distance vs Speed&quot;</span>,</span>
+<span id="cb432-5"><a href="simple-linear-regression.html#cb432-5" tabindex="-1"></a>     <span class="at">pch  =</span> <span class="dv">20</span>,</span>
+<span id="cb432-6"><a href="simple-linear-regression.html#cb432-6" tabindex="-1"></a>     <span class="at">cex  =</span> <span class="dv">2</span>,</span>
+<span id="cb432-7"><a href="simple-linear-regression.html#cb432-7" tabindex="-1"></a>     <span class="at">col  =</span> <span class="st">&quot;grey&quot;</span>)</span></code></pre></div>
 <p><img src="slr_files/figure-html/unnamed-chunk-6-1.png" width="672" style="display: block; margin: auto;" /></p>
 <p>Let’s now define some terminology. We have pairs of data, <span class="math inline">\((x_i, y_i)\)</span>, for <span class="math inline">\(i = 1, 2, \ldots n\)</span>, where <span class="math inline">\(n\)</span> is the sample size of the dataset.</p>
 <p>We use <span class="math inline">\(i\)</span> as an index, simply for notation. We use <span class="math inline">\(x_i\)</span> as the <strong>predictor</strong> (explanatory) variable. The predictor variable is used to help <em>predict</em> or explain the <strong>response</strong> (target, outcome) variable, <span class="math inline">\(y_i\)</span>.</p>
@@ -560,9 +560,9 @@ <h3><span class="header-section-number">7.1.1</span> Simple Linear Regression Mo
 \text{Var}[Y_i \mid X_i = x_i] = \sigma^2.
 \]</span></p>
 <p>This is visually displayed in the image below. We see that for any value <span class="math inline">\(x\)</span>, the expected value of <span class="math inline">\(Y\)</span> is <span class="math inline">\(\beta_0 + \beta_1 x\)</span>. At each value of <span class="math inline">\(x\)</span>, <span class="math inline">\(Y\)</span> has the same variance <span class="math inline">\(\sigma^2\)</span>.</p>
-<div class="figure">
-<img src="images/model.jpg" alt="" />
-<p class="caption">Simple Linear Regression Model <a href="http://statwiki.ucdavis.edu/Textbook_Maps/General_Statistics/Map%3A_Introductory_Statistics_(Shafer_and_Zhang)/10%3A_Correlation_and_Regression/10.3_Modelling_Linear_Relationships_with_Randomness_Present" target="_blank">Introductory Statistics (Shafer and Zhang), UC Davis Stat Wiki</a></p>
+<div class="float">
+<img src="images/model.jpg" alt="Simple Linear Regression Model Introductory Statistics (Shafer and Zhang), UC Davis Stat Wiki" />
+<div class="figcaption">Simple Linear Regression Model <a href="https://stats.libretexts.org/Bookshelves/Introductory_Statistics/Introductory_Statistics_(Shafer_and_Zhang)/10%3A_Correlation_and_Regression/10.03%3A_Modelling_Linear_Relationships_with_Randomness_Present" target="_blank">Introductory Statistics (Shafer and Zhang), UC Davis Stat Wiki</a></div>
 </div>
 <p>Often, we directly talk about the assumptions that this model makes. They can be cleverly shortened to <strong>LINE</strong>.</p>
 <ul>
@@ -644,18 +644,18 @@ <h2><span class="header-section-number">7.2</span> Least Squares Approach<a href
 \]</span></p>
 <p>Traditionally we would now calculate <span class="math inline">\(\hat{\beta}_0\)</span> and <span class="math inline">\(\hat{\beta}_1\)</span> by hand for the <code>cars</code> dataset. However because we are living in the 21st century and are intelligent (or lazy or efficient, depending on your perspective), we will utilize <code>R</code> to do the number crunching for us.</p>
 <p>To keep some notation consistent with the above mathematics, we will store the response variable as <code>y</code> and the predictor variable as <code>x</code>.</p>
-<div class="sourceCode" id="cb433"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb433-1"><a href="simple-linear-regression.html#cb433-1" aria-hidden="true" tabindex="-1"></a>x <span class="ot">=</span> cars<span class="sc">$</span>speed</span>
-<span id="cb433-2"><a href="simple-linear-regression.html#cb433-2" aria-hidden="true" tabindex="-1"></a>y <span class="ot">=</span> cars<span class="sc">$</span>dist</span></code></pre></div>
+<div class="sourceCode" id="cb433"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb433-1"><a href="simple-linear-regression.html#cb433-1" tabindex="-1"></a>x <span class="ot">=</span> cars<span class="sc">$</span>speed</span>
+<span id="cb433-2"><a href="simple-linear-regression.html#cb433-2" tabindex="-1"></a>y <span class="ot">=</span> cars<span class="sc">$</span>dist</span></code></pre></div>
 <p>We then calculate the three sums of squares defined above.</p>
-<div class="sourceCode" id="cb434"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb434-1"><a href="simple-linear-regression.html#cb434-1" aria-hidden="true" tabindex="-1"></a>Sxy <span class="ot">=</span> <span class="fu">sum</span>((x <span class="sc">-</span> <span class="fu">mean</span>(x)) <span class="sc">*</span> (y <span class="sc">-</span> <span class="fu">mean</span>(y)))</span>
-<span id="cb434-2"><a href="simple-linear-regression.html#cb434-2" aria-hidden="true" tabindex="-1"></a>Sxx <span class="ot">=</span> <span class="fu">sum</span>((x <span class="sc">-</span> <span class="fu">mean</span>(x)) <span class="sc">^</span> <span class="dv">2</span>)</span>
-<span id="cb434-3"><a href="simple-linear-regression.html#cb434-3" aria-hidden="true" tabindex="-1"></a>Syy <span class="ot">=</span> <span class="fu">sum</span>((y <span class="sc">-</span> <span class="fu">mean</span>(y)) <span class="sc">^</span> <span class="dv">2</span>)</span>
-<span id="cb434-4"><a href="simple-linear-regression.html#cb434-4" aria-hidden="true" tabindex="-1"></a><span class="fu">c</span>(Sxy, Sxx, Syy)</span></code></pre></div>
+<div class="sourceCode" id="cb434"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb434-1"><a href="simple-linear-regression.html#cb434-1" tabindex="-1"></a>Sxy <span class="ot">=</span> <span class="fu">sum</span>((x <span class="sc">-</span> <span class="fu">mean</span>(x)) <span class="sc">*</span> (y <span class="sc">-</span> <span class="fu">mean</span>(y)))</span>
+<span id="cb434-2"><a href="simple-linear-regression.html#cb434-2" tabindex="-1"></a>Sxx <span class="ot">=</span> <span class="fu">sum</span>((x <span class="sc">-</span> <span class="fu">mean</span>(x)) <span class="sc">^</span> <span class="dv">2</span>)</span>
+<span id="cb434-3"><a href="simple-linear-regression.html#cb434-3" tabindex="-1"></a>Syy <span class="ot">=</span> <span class="fu">sum</span>((y <span class="sc">-</span> <span class="fu">mean</span>(y)) <span class="sc">^</span> <span class="dv">2</span>)</span>
+<span id="cb434-4"><a href="simple-linear-regression.html#cb434-4" tabindex="-1"></a><span class="fu">c</span>(Sxy, Sxx, Syy)</span></code></pre></div>
 <pre><code>## [1]  5387.40  1370.00 32538.98</code></pre>
 <p>Then finally calculate <span class="math inline">\(\hat{\beta}_0\)</span> and <span class="math inline">\(\hat{\beta}_1\)</span>.</p>
-<div class="sourceCode" id="cb436"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb436-1"><a href="simple-linear-regression.html#cb436-1" aria-hidden="true" tabindex="-1"></a>beta_1_hat <span class="ot">=</span> Sxy <span class="sc">/</span> Sxx</span>
-<span id="cb436-2"><a href="simple-linear-regression.html#cb436-2" aria-hidden="true" tabindex="-1"></a>beta_0_hat <span class="ot">=</span> <span class="fu">mean</span>(y) <span class="sc">-</span> beta_1_hat <span class="sc">*</span> <span class="fu">mean</span>(x)</span>
-<span id="cb436-3"><a href="simple-linear-regression.html#cb436-3" aria-hidden="true" tabindex="-1"></a><span class="fu">c</span>(beta_0_hat, beta_1_hat)</span></code></pre></div>
+<div class="sourceCode" id="cb436"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb436-1"><a href="simple-linear-regression.html#cb436-1" tabindex="-1"></a>beta_1_hat <span class="ot">=</span> Sxy <span class="sc">/</span> Sxx</span>
+<span id="cb436-2"><a href="simple-linear-regression.html#cb436-2" tabindex="-1"></a>beta_0_hat <span class="ot">=</span> <span class="fu">mean</span>(y) <span class="sc">-</span> beta_1_hat <span class="sc">*</span> <span class="fu">mean</span>(x)</span>
+<span id="cb436-3"><a href="simple-linear-regression.html#cb436-3" tabindex="-1"></a><span class="fu">c</span>(beta_0_hat, beta_1_hat)</span></code></pre></div>
 <pre><code>## [1] -17.579095   3.932409</code></pre>
 <p>What do these values tell us about our dataset?</p>
 <p>The slope <em>parameter</em> <span class="math inline">\(\beta_1\)</span> tells us that for an increase in speed of one mile per hour, the <strong>mean</strong> stopping distance increases by <span class="math inline">\(\beta_1\)</span>. It is important to specify that we are talking about the mean. Recall that <span class="math inline">\(\beta_0 + \beta_1 x\)</span> is the mean of <span class="math inline">\(Y\)</span>, in this case stopping distance, for a particular value of <span class="math inline">\(x\)</span>. (In this case speed.) So <span class="math inline">\(\beta_1\)</span> tells us how the mean of <span class="math inline">\(Y\)</span> is affected by a change in <span class="math inline">\(x\)</span>.</p>
@@ -672,36 +672,36 @@ <h3><span class="header-section-number">7.2.1</span> Making Predictions<a href="
 \hat{y} = -17.58 + 3.93 x.
 \]</span></p>
 <p>We can now use this line to make predictions. First, let’s see the possible <span class="math inline">\(x\)</span> values in the <code>cars</code> dataset. Since some <span class="math inline">\(x\)</span> values may appear more than once, we use the <code>unique()</code> to return each unique value only once.</p>
-<div class="sourceCode" id="cb438"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb438-1"><a href="simple-linear-regression.html#cb438-1" aria-hidden="true" tabindex="-1"></a><span class="fu">unique</span>(cars<span class="sc">$</span>speed)</span></code></pre></div>
+<div class="sourceCode" id="cb438"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb438-1"><a href="simple-linear-regression.html#cb438-1" tabindex="-1"></a><span class="fu">unique</span>(cars<span class="sc">$</span>speed)</span></code></pre></div>
 <pre><code>##  [1]  4  7  8  9 10 11 12 13 14 15 16 17 18 19 20 22 23 24 25</code></pre>
 <p>Let’s make a prediction for the stopping distance of a car traveling at 8 miles per hour.</p>
 <p><span class="math display">\[
 \hat{y} = -17.58 + 3.93 \times 8 % = 13.88
 \]</span></p>
-<div class="sourceCode" id="cb440"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb440-1"><a href="simple-linear-regression.html#cb440-1" aria-hidden="true" tabindex="-1"></a>beta_0_hat <span class="sc">+</span> beta_1_hat <span class="sc">*</span> <span class="dv">8</span></span></code></pre></div>
+<div class="sourceCode" id="cb440"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb440-1"><a href="simple-linear-regression.html#cb440-1" tabindex="-1"></a>beta_0_hat <span class="sc">+</span> beta_1_hat <span class="sc">*</span> <span class="dv">8</span></span></code></pre></div>
 <pre><code>## [1] 13.88018</code></pre>
 <p>This tells us that the estimated mean stopping distance of a car traveling at 8 miles per hour is <span class="math inline">\(13.88\)</span>.</p>
 <p>Now let’s make a prediction for the stopping distance of a car traveling at 21 miles per hour. This is considered <strong>interpolation</strong> as 21 is not an observed value of <span class="math inline">\(x\)</span>. (But is in the data range.) We can use the special <code>%in%</code> operator to quickly verify this in <code>R</code>.</p>
-<div class="sourceCode" id="cb442"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb442-1"><a href="simple-linear-regression.html#cb442-1" aria-hidden="true" tabindex="-1"></a><span class="dv">8</span> <span class="sc">%in%</span> <span class="fu">unique</span>(cars<span class="sc">$</span>speed)</span></code></pre></div>
+<div class="sourceCode" id="cb442"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb442-1"><a href="simple-linear-regression.html#cb442-1" tabindex="-1"></a><span class="dv">8</span> <span class="sc">%in%</span> <span class="fu">unique</span>(cars<span class="sc">$</span>speed)</span></code></pre></div>
 <pre><code>## [1] TRUE</code></pre>
-<div class="sourceCode" id="cb444"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb444-1"><a href="simple-linear-regression.html#cb444-1" aria-hidden="true" tabindex="-1"></a><span class="dv">21</span> <span class="sc">%in%</span> <span class="fu">unique</span>(cars<span class="sc">$</span>speed)</span></code></pre></div>
+<div class="sourceCode" id="cb444"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb444-1"><a href="simple-linear-regression.html#cb444-1" tabindex="-1"></a><span class="dv">21</span> <span class="sc">%in%</span> <span class="fu">unique</span>(cars<span class="sc">$</span>speed)</span></code></pre></div>
 <pre><code>## [1] FALSE</code></pre>
-<div class="sourceCode" id="cb446"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb446-1"><a href="simple-linear-regression.html#cb446-1" aria-hidden="true" tabindex="-1"></a><span class="fu">min</span>(cars<span class="sc">$</span>speed) <span class="sc">&lt;</span> <span class="dv">21</span> <span class="sc">&amp;</span> <span class="dv">21</span> <span class="sc">&lt;</span> <span class="fu">max</span>(cars<span class="sc">$</span>speed)</span></code></pre></div>
+<div class="sourceCode" id="cb446"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb446-1"><a href="simple-linear-regression.html#cb446-1" tabindex="-1"></a><span class="fu">min</span>(cars<span class="sc">$</span>speed) <span class="sc">&lt;</span> <span class="dv">21</span> <span class="sc">&amp;</span> <span class="dv">21</span> <span class="sc">&lt;</span> <span class="fu">max</span>(cars<span class="sc">$</span>speed)</span></code></pre></div>
 <pre><code>## [1] TRUE</code></pre>
 <p><span class="math display">\[
 \hat{y} = -17.58 + 3.93 \times 21 % = 65
 \]</span></p>
-<div class="sourceCode" id="cb448"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb448-1"><a href="simple-linear-regression.html#cb448-1" aria-hidden="true" tabindex="-1"></a>beta_0_hat <span class="sc">+</span> beta_1_hat <span class="sc">*</span> <span class="dv">21</span></span></code></pre></div>
+<div class="sourceCode" id="cb448"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb448-1"><a href="simple-linear-regression.html#cb448-1" tabindex="-1"></a>beta_0_hat <span class="sc">+</span> beta_1_hat <span class="sc">*</span> <span class="dv">21</span></span></code></pre></div>
 <pre><code>## [1] 65.00149</code></pre>
 <p>Lastly, we can make a prediction for the stopping distance of a car traveling at 50 miles per hour. This is considered <a href="https://xkcd.com/605/" target="_blank"><strong>extrapolation</strong></a> as 50 is not an observed value of <span class="math inline">\(x\)</span> and is outside data range. We should be less confident in predictions of this type.</p>
-<div class="sourceCode" id="cb450"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb450-1"><a href="simple-linear-regression.html#cb450-1" aria-hidden="true" tabindex="-1"></a><span class="fu">range</span>(cars<span class="sc">$</span>speed)</span></code></pre></div>
+<div class="sourceCode" id="cb450"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb450-1"><a href="simple-linear-regression.html#cb450-1" tabindex="-1"></a><span class="fu">range</span>(cars<span class="sc">$</span>speed)</span></code></pre></div>
 <pre><code>## [1]  4 25</code></pre>
-<div class="sourceCode" id="cb452"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb452-1"><a href="simple-linear-regression.html#cb452-1" aria-hidden="true" tabindex="-1"></a><span class="fu">range</span>(cars<span class="sc">$</span>speed)[<span class="dv">1</span>] <span class="sc">&lt;</span> <span class="dv">50</span> <span class="sc">&amp;</span> <span class="dv">50</span> <span class="sc">&lt;</span> <span class="fu">range</span>(cars<span class="sc">$</span>speed)[<span class="dv">2</span>] </span></code></pre></div>
+<div class="sourceCode" id="cb452"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb452-1"><a href="simple-linear-regression.html#cb452-1" tabindex="-1"></a><span class="fu">range</span>(cars<span class="sc">$</span>speed)[<span class="dv">1</span>] <span class="sc">&lt;</span> <span class="dv">50</span> <span class="sc">&amp;</span> <span class="dv">50</span> <span class="sc">&lt;</span> <span class="fu">range</span>(cars<span class="sc">$</span>speed)[<span class="dv">2</span>] </span></code></pre></div>
 <pre><code>## [1] FALSE</code></pre>
 <p><span class="math display">\[
 \hat{y} = -17.58 + 3.93 \times 50 % = 179.04
 \]</span></p>
-<div class="sourceCode" id="cb454"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb454-1"><a href="simple-linear-regression.html#cb454-1" aria-hidden="true" tabindex="-1"></a>beta_0_hat <span class="sc">+</span> beta_1_hat <span class="sc">*</span> <span class="dv">50</span></span></code></pre></div>
+<div class="sourceCode" id="cb454"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb454-1"><a href="simple-linear-regression.html#cb454-1" tabindex="-1"></a>beta_0_hat <span class="sc">+</span> beta_1_hat <span class="sc">*</span> <span class="dv">50</span></span></code></pre></div>
 <pre><code>## [1] 179.0413</code></pre>
 <p>Cars travel 50 miles per hour rather easily today, but not in the 1920s!</p>
 <p>This is also an issue we saw when interpreting <span class="math inline">\(\hat{\beta}_0 = -17.58\)</span>, which is equivalent to making a prediction at <span class="math inline">\(x = 0\)</span>. We should not be confident in the estimated linear relationship outside of the range of data we have observed.</p>
@@ -717,19 +717,19 @@ <h3><span class="header-section-number">7.2.2</span> Residuals<a href="simple-li
 e_i = y_i - \hat{y}_i
 \]</span></p>
 <p>Let’s calculate the residual for the prediction we made for a car traveling 8 miles per hour. First, we need to obtain the observed value of <span class="math inline">\(y\)</span> for this <span class="math inline">\(x\)</span> value.</p>
-<div class="sourceCode" id="cb456"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb456-1"><a href="simple-linear-regression.html#cb456-1" aria-hidden="true" tabindex="-1"></a><span class="fu">which</span>(cars<span class="sc">$</span>speed <span class="sc">==</span> <span class="dv">8</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb456"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb456-1"><a href="simple-linear-regression.html#cb456-1" tabindex="-1"></a><span class="fu">which</span>(cars<span class="sc">$</span>speed <span class="sc">==</span> <span class="dv">8</span>)</span></code></pre></div>
 <pre><code>## [1] 5</code></pre>
-<div class="sourceCode" id="cb458"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb458-1"><a href="simple-linear-regression.html#cb458-1" aria-hidden="true" tabindex="-1"></a>cars[<span class="dv">5</span>, ]</span></code></pre></div>
+<div class="sourceCode" id="cb458"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb458-1"><a href="simple-linear-regression.html#cb458-1" tabindex="-1"></a>cars[<span class="dv">5</span>, ]</span></code></pre></div>
 <pre><code>##   speed dist
 ## 5     8   16</code></pre>
-<div class="sourceCode" id="cb460"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb460-1"><a href="simple-linear-regression.html#cb460-1" aria-hidden="true" tabindex="-1"></a>cars[<span class="fu">which</span>(cars<span class="sc">$</span>speed <span class="sc">==</span> <span class="dv">8</span>), ]</span></code></pre></div>
+<div class="sourceCode" id="cb460"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb460-1"><a href="simple-linear-regression.html#cb460-1" tabindex="-1"></a>cars[<span class="fu">which</span>(cars<span class="sc">$</span>speed <span class="sc">==</span> <span class="dv">8</span>), ]</span></code></pre></div>
 <pre><code>##   speed dist
 ## 5     8   16</code></pre>
 <p>We can then calculate the residual.</p>
 <p><span class="math display">\[
 e = 16 - 13.88 = 2.12
 \]</span></p>
-<div class="sourceCode" id="cb462"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb462-1"><a href="simple-linear-regression.html#cb462-1" aria-hidden="true" tabindex="-1"></a><span class="dv">16</span> <span class="sc">-</span> (beta_0_hat <span class="sc">+</span> beta_1_hat <span class="sc">*</span> <span class="dv">8</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb462"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb462-1"><a href="simple-linear-regression.html#cb462-1" tabindex="-1"></a><span class="dv">16</span> <span class="sc">-</span> (beta_0_hat <span class="sc">+</span> beta_1_hat <span class="sc">*</span> <span class="dv">8</span>)</span></code></pre></div>
 <pre><code>## [1] 2.119825</code></pre>
 <p>The positive residual value indicates that the observed stopping distance is actually 2.12 feet more than what was predicted.</p>
 </div>
@@ -759,15 +759,15 @@ <h3><span class="header-section-number">7.2.3</span> Variance Estimation<a href=
 \]</span></p>
 <p>Here, <span class="math inline">\(s^2\)</span> is the estimate of <span class="math inline">\(\sigma^2\)</span> when we have a single random variable <span class="math inline">\(X\)</span>. In this case <span class="math inline">\(\bar{x}\)</span> is an estimate of <span class="math inline">\(\mu\)</span> which is assumed to be the same for each <span class="math inline">\(x\)</span>.</p>
 <p>Now, in the regression case, with <span class="math inline">\(s_e^2\)</span> each <span class="math inline">\(y\)</span> has a different mean because of the relationship with <span class="math inline">\(x\)</span>. Thus, for each <span class="math inline">\(y_i\)</span>, we use a different estimate of the mean, that is <span class="math inline">\(\hat{y}_i\)</span>.</p>
-<div class="sourceCode" id="cb464"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb464-1"><a href="simple-linear-regression.html#cb464-1" aria-hidden="true" tabindex="-1"></a>y_hat <span class="ot">=</span> beta_0_hat <span class="sc">+</span> beta_1_hat <span class="sc">*</span> x</span>
-<span id="cb464-2"><a href="simple-linear-regression.html#cb464-2" aria-hidden="true" tabindex="-1"></a>e     <span class="ot">=</span> y <span class="sc">-</span> y_hat</span>
-<span id="cb464-3"><a href="simple-linear-regression.html#cb464-3" aria-hidden="true" tabindex="-1"></a>n     <span class="ot">=</span> <span class="fu">length</span>(e)</span>
-<span id="cb464-4"><a href="simple-linear-regression.html#cb464-4" aria-hidden="true" tabindex="-1"></a>s2_e  <span class="ot">=</span> <span class="fu">sum</span>(e<span class="sc">^</span><span class="dv">2</span>) <span class="sc">/</span> (n <span class="sc">-</span> <span class="dv">2</span>)</span>
-<span id="cb464-5"><a href="simple-linear-regression.html#cb464-5" aria-hidden="true" tabindex="-1"></a>s2_e</span></code></pre></div>
+<div class="sourceCode" id="cb464"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb464-1"><a href="simple-linear-regression.html#cb464-1" tabindex="-1"></a>y_hat <span class="ot">=</span> beta_0_hat <span class="sc">+</span> beta_1_hat <span class="sc">*</span> x</span>
+<span id="cb464-2"><a href="simple-linear-regression.html#cb464-2" tabindex="-1"></a>e     <span class="ot">=</span> y <span class="sc">-</span> y_hat</span>
+<span id="cb464-3"><a href="simple-linear-regression.html#cb464-3" tabindex="-1"></a>n     <span class="ot">=</span> <span class="fu">length</span>(e)</span>
+<span id="cb464-4"><a href="simple-linear-regression.html#cb464-4" tabindex="-1"></a>s2_e  <span class="ot">=</span> <span class="fu">sum</span>(e<span class="sc">^</span><span class="dv">2</span>) <span class="sc">/</span> (n <span class="sc">-</span> <span class="dv">2</span>)</span>
+<span id="cb464-5"><a href="simple-linear-regression.html#cb464-5" tabindex="-1"></a>s2_e</span></code></pre></div>
 <pre><code>## [1] 236.5317</code></pre>
 <p>Just as with the univariate measure of variance, this value of 236.53 doesn’t have a practical interpretation in terms of stopping distance. Taking the square root, however, computes the standard deviation of the residuals, also known as <em>residual standard error</em>.</p>
-<div class="sourceCode" id="cb466"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb466-1"><a href="simple-linear-regression.html#cb466-1" aria-hidden="true" tabindex="-1"></a>s_e <span class="ot">=</span> <span class="fu">sqrt</span>(s2_e)</span>
-<span id="cb466-2"><a href="simple-linear-regression.html#cb466-2" aria-hidden="true" tabindex="-1"></a>s_e</span></code></pre></div>
+<div class="sourceCode" id="cb466"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb466-1"><a href="simple-linear-regression.html#cb466-1" tabindex="-1"></a>s_e <span class="ot">=</span> <span class="fu">sqrt</span>(s2_e)</span>
+<span id="cb466-2"><a href="simple-linear-regression.html#cb466-2" tabindex="-1"></a>s_e</span></code></pre></div>
 <pre><code>## [1] 15.37959</code></pre>
 <p>This tells us that our estimates of mean stopping distance are “typically” off by 15.38 feet.</p>
 </div>
@@ -808,20 +808,20 @@ <h4>Sum of Squares Error<a href="simple-linear-regression.html#sum-of-squares-er
 \text{SSE} = \text{RSS} = \sum_{i=1}^{n}(y_i - \hat{y}_i)^2
 \]</span></p>
 <p>The quantity “Sum of Squares Error,” <span class="math inline">\(\text{SSE}\)</span>, represents the <strong>unexplained variation</strong> of the observed <span class="math inline">\(y\)</span> values. You will often see <span class="math inline">\(\text{SSE}\)</span> written as <span class="math inline">\(\text{RSS}\)</span>, or “Residual Sum of Squares.”</p>
-<div class="sourceCode" id="cb468"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb468-1"><a href="simple-linear-regression.html#cb468-1" aria-hidden="true" tabindex="-1"></a>SST   <span class="ot">=</span> <span class="fu">sum</span>((y <span class="sc">-</span> <span class="fu">mean</span>(y)) <span class="sc">^</span> <span class="dv">2</span>)</span>
-<span id="cb468-2"><a href="simple-linear-regression.html#cb468-2" aria-hidden="true" tabindex="-1"></a>SSReg <span class="ot">=</span> <span class="fu">sum</span>((y_hat <span class="sc">-</span> <span class="fu">mean</span>(y)) <span class="sc">^</span> <span class="dv">2</span>)</span>
-<span id="cb468-3"><a href="simple-linear-regression.html#cb468-3" aria-hidden="true" tabindex="-1"></a>SSE   <span class="ot">=</span> <span class="fu">sum</span>((y <span class="sc">-</span> y_hat) <span class="sc">^</span> <span class="dv">2</span>)</span>
-<span id="cb468-4"><a href="simple-linear-regression.html#cb468-4" aria-hidden="true" tabindex="-1"></a><span class="fu">c</span>(<span class="at">SST =</span> SST, <span class="at">SSReg =</span> SSReg, <span class="at">SSE =</span> SSE)</span></code></pre></div>
+<div class="sourceCode" id="cb468"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb468-1"><a href="simple-linear-regression.html#cb468-1" tabindex="-1"></a>SST   <span class="ot">=</span> <span class="fu">sum</span>((y <span class="sc">-</span> <span class="fu">mean</span>(y)) <span class="sc">^</span> <span class="dv">2</span>)</span>
+<span id="cb468-2"><a href="simple-linear-regression.html#cb468-2" tabindex="-1"></a>SSReg <span class="ot">=</span> <span class="fu">sum</span>((y_hat <span class="sc">-</span> <span class="fu">mean</span>(y)) <span class="sc">^</span> <span class="dv">2</span>)</span>
+<span id="cb468-3"><a href="simple-linear-regression.html#cb468-3" tabindex="-1"></a>SSE   <span class="ot">=</span> <span class="fu">sum</span>((y <span class="sc">-</span> y_hat) <span class="sc">^</span> <span class="dv">2</span>)</span>
+<span id="cb468-4"><a href="simple-linear-regression.html#cb468-4" tabindex="-1"></a><span class="fu">c</span>(<span class="at">SST =</span> SST, <span class="at">SSReg =</span> SSReg, <span class="at">SSE =</span> SSE)</span></code></pre></div>
 <pre><code>##      SST    SSReg      SSE 
 ## 32538.98 21185.46 11353.52</code></pre>
 <p>Note that,</p>
 <p><span class="math display">\[
 s_e^2 = \frac{\text{SSE}}{n - 2}.
 \]</span></p>
-<div class="sourceCode" id="cb470"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb470-1"><a href="simple-linear-regression.html#cb470-1" aria-hidden="true" tabindex="-1"></a>SSE <span class="sc">/</span> (n <span class="sc">-</span> <span class="dv">2</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb470"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb470-1"><a href="simple-linear-regression.html#cb470-1" tabindex="-1"></a>SSE <span class="sc">/</span> (n <span class="sc">-</span> <span class="dv">2</span>)</span></code></pre></div>
 <pre><code>## [1] 236.5317</code></pre>
 <p>We can use <code>R</code> to verify that this matches our previous calculation of <span class="math inline">\(s_e^2\)</span>.</p>
-<div class="sourceCode" id="cb472"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb472-1"><a href="simple-linear-regression.html#cb472-1" aria-hidden="true" tabindex="-1"></a>s2_e <span class="sc">==</span> SSE <span class="sc">/</span> (n <span class="sc">-</span> <span class="dv">2</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb472"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb472-1"><a href="simple-linear-regression.html#cb472-1" tabindex="-1"></a>s2_e <span class="sc">==</span> SSE <span class="sc">/</span> (n <span class="sc">-</span> <span class="dv">2</span>)</span></code></pre></div>
 <pre><code>## [1] TRUE</code></pre>
 <p>These three measures also do not have an important practical interpretation individually. But together, they’re about to reveal a new statistic to help measure the strength of an SLR model.</p>
 </div>
@@ -837,8 +837,8 @@ <h3><span class="header-section-number">7.3.1</span> Coefficient of Determinatio
 \end{aligned}
 \]</span></p>
 <p>The coefficient of determination is interpreted as the proportion of observed variation in <span class="math inline">\(y\)</span> that can be explained by the simple linear regression model.</p>
-<div class="sourceCode" id="cb474"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb474-1"><a href="simple-linear-regression.html#cb474-1" aria-hidden="true" tabindex="-1"></a>R2 <span class="ot">=</span> SSReg <span class="sc">/</span> SST</span>
-<span id="cb474-2"><a href="simple-linear-regression.html#cb474-2" aria-hidden="true" tabindex="-1"></a>R2</span></code></pre></div>
+<div class="sourceCode" id="cb474"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb474-1"><a href="simple-linear-regression.html#cb474-1" tabindex="-1"></a>R2 <span class="ot">=</span> SSReg <span class="sc">/</span> SST</span>
+<span id="cb474-2"><a href="simple-linear-regression.html#cb474-2" tabindex="-1"></a>R2</span></code></pre></div>
 <pre><code>## [1] 0.6510794</code></pre>
 <p>For the <code>cars</code> example, we calculate <span class="math inline">\(R^2 = 0.65\)</span>. We then say that <span class="math inline">\(65\%\)</span> of the observed variability in stopping distance is explained by the linear relationship with speed.</p>
 <p>The following plots visually demonstrate the three “sums of squares” for a simulated dataset which has <span class="math inline">\(R^2 = 0.92\)</span> which is a somewhat high value. Notice in the final plot, that the orange arrows account for a larger proportion of the total arrow.</p>
@@ -852,10 +852,10 @@ <h2><span class="header-section-number">7.4</span> The <code>lm</code> Function<
 <p>So far we have done regression by deriving the least squares estimates, then writing simple <code>R</code> commands to perform the necessary calculations. Since this is such a common task, this is functionality that is built directly into <code>R</code> via the <code>lm()</code> command.</p>
 <p>The <code>lm()</code> command is used to fit <strong>linear models</strong> which actually account for a broader class of models than simple linear regression, but we will use SLR as our first demonstration of <code>lm()</code>. The <code>lm()</code> function will be one of our most commonly used tools, so you may want to take a look at the documentation by using <code>?lm</code>. You’ll notice there is a lot of information there, but we will start with just the very basics. This is documentation you will want to return to often.</p>
 <p>We’ll continue using the <code>cars</code> data, and essentially use the <code>lm()</code> function to check the work we had previously done.</p>
-<div class="sourceCode" id="cb476"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb476-1"><a href="simple-linear-regression.html#cb476-1" aria-hidden="true" tabindex="-1"></a>stop_dist_model <span class="ot">=</span> <span class="fu">lm</span>(dist <span class="sc">~</span> speed, <span class="at">data =</span> cars)</span></code></pre></div>
+<div class="sourceCode" id="cb476"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb476-1"><a href="simple-linear-regression.html#cb476-1" tabindex="-1"></a>stop_dist_model <span class="ot">=</span> <span class="fu">lm</span>(dist <span class="sc">~</span> speed, <span class="at">data =</span> cars)</span></code></pre></div>
 <p>This line of code fits our very first linear model. The syntax should look somewhat familiar. We use the <code>dist ~ speed</code> syntax to tell <code>R</code> we would like to model the response variable <code>dist</code> as a linear function of the predictor variable <code>speed</code>. In general, you should think of the syntax as <code>response ~ predictor</code>. The <code>data = cars</code> argument then tells <code>R</code> that the <code>dist</code> and <code>speed</code> variables are from the dataset <code>cars</code>. We then store this result in a variable <code>stop_dist_model</code>.</p>
 <p>The variable <code>stop_dist_model</code> now contains a wealth of information, and we will now see how to extract and use that information. The first thing we will do is simply output whatever is stored immediately in the variable <code>stop_dist_model</code>.</p>
-<div class="sourceCode" id="cb477"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb477-1"><a href="simple-linear-regression.html#cb477-1" aria-hidden="true" tabindex="-1"></a>stop_dist_model</span></code></pre></div>
+<div class="sourceCode" id="cb477"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb477-1"><a href="simple-linear-regression.html#cb477-1" tabindex="-1"></a>stop_dist_model</span></code></pre></div>
 <pre><code>## 
 ## Call:
 ## lm(formula = dist ~ speed, data = cars)
@@ -864,26 +864,26 @@ <h2><span class="header-section-number">7.4</span> The <code>lm</code> Function<
 ## (Intercept)        speed  
 ##     -17.579        3.932</code></pre>
 <p>We see that it first tells us the formula we input into <code>R</code>, that is <code>lm(formula = dist ~ speed, data = cars)</code>. We also see the coefficients of the model. We can check that these are what we had calculated previously. (Minus some rounding that <code>R</code> is doing when displaying the results. They are stored with full precision.)</p>
-<div class="sourceCode" id="cb479"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb479-1"><a href="simple-linear-regression.html#cb479-1" aria-hidden="true" tabindex="-1"></a><span class="fu">c</span>(beta_0_hat, beta_1_hat)</span></code></pre></div>
+<div class="sourceCode" id="cb479"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb479-1"><a href="simple-linear-regression.html#cb479-1" tabindex="-1"></a><span class="fu">c</span>(beta_0_hat, beta_1_hat)</span></code></pre></div>
 <pre><code>## [1] -17.579095   3.932409</code></pre>
 <p>Next, it would be nice to add the fitted line to the scatterplot. To do so we will use the <code>abline()</code> function.</p>
-<div class="sourceCode" id="cb481"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb481-1"><a href="simple-linear-regression.html#cb481-1" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(dist <span class="sc">~</span> speed, <span class="at">data =</span> cars,</span>
-<span id="cb481-2"><a href="simple-linear-regression.html#cb481-2" aria-hidden="true" tabindex="-1"></a>     <span class="at">xlab =</span> <span class="st">&quot;Speed (in Miles Per Hour)&quot;</span>,</span>
-<span id="cb481-3"><a href="simple-linear-regression.html#cb481-3" aria-hidden="true" tabindex="-1"></a>     <span class="at">ylab =</span> <span class="st">&quot;Stopping Distance (in Feet)&quot;</span>,</span>
-<span id="cb481-4"><a href="simple-linear-regression.html#cb481-4" aria-hidden="true" tabindex="-1"></a>     <span class="at">main =</span> <span class="st">&quot;Stopping Distance vs Speed&quot;</span>,</span>
-<span id="cb481-5"><a href="simple-linear-regression.html#cb481-5" aria-hidden="true" tabindex="-1"></a>     <span class="at">pch  =</span> <span class="dv">20</span>,</span>
-<span id="cb481-6"><a href="simple-linear-regression.html#cb481-6" aria-hidden="true" tabindex="-1"></a>     <span class="at">cex  =</span> <span class="dv">2</span>,</span>
-<span id="cb481-7"><a href="simple-linear-regression.html#cb481-7" aria-hidden="true" tabindex="-1"></a>     <span class="at">col  =</span> <span class="st">&quot;grey&quot;</span>)</span>
-<span id="cb481-8"><a href="simple-linear-regression.html#cb481-8" aria-hidden="true" tabindex="-1"></a><span class="fu">abline</span>(stop_dist_model, <span class="at">lwd =</span> <span class="dv">3</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb481"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb481-1"><a href="simple-linear-regression.html#cb481-1" tabindex="-1"></a><span class="fu">plot</span>(dist <span class="sc">~</span> speed, <span class="at">data =</span> cars,</span>
+<span id="cb481-2"><a href="simple-linear-regression.html#cb481-2" tabindex="-1"></a>     <span class="at">xlab =</span> <span class="st">&quot;Speed (in Miles Per Hour)&quot;</span>,</span>
+<span id="cb481-3"><a href="simple-linear-regression.html#cb481-3" tabindex="-1"></a>     <span class="at">ylab =</span> <span class="st">&quot;Stopping Distance (in Feet)&quot;</span>,</span>
+<span id="cb481-4"><a href="simple-linear-regression.html#cb481-4" tabindex="-1"></a>     <span class="at">main =</span> <span class="st">&quot;Stopping Distance vs Speed&quot;</span>,</span>
+<span id="cb481-5"><a href="simple-linear-regression.html#cb481-5" tabindex="-1"></a>     <span class="at">pch  =</span> <span class="dv">20</span>,</span>
+<span id="cb481-6"><a href="simple-linear-regression.html#cb481-6" tabindex="-1"></a>     <span class="at">cex  =</span> <span class="dv">2</span>,</span>
+<span id="cb481-7"><a href="simple-linear-regression.html#cb481-7" tabindex="-1"></a>     <span class="at">col  =</span> <span class="st">&quot;grey&quot;</span>)</span>
+<span id="cb481-8"><a href="simple-linear-regression.html#cb481-8" tabindex="-1"></a><span class="fu">abline</span>(stop_dist_model, <span class="at">lwd =</span> <span class="dv">3</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>)</span></code></pre></div>
 <p><img src="slr_files/figure-html/unnamed-chunk-33-1.png" width="672" style="display: block; margin: auto;" /></p>
 <p>The <code>abline()</code> function is used to add lines of the form <span class="math inline">\(a + bx\)</span> to a plot. (Hence <strong><code>ab</code></strong><code>line</code>.) When we give it <code>stop_dist_model</code> as an argument, it automatically extracts the regression coefficient estimates (<span class="math inline">\(\hat{\beta}_0\)</span> and <span class="math inline">\(\hat{\beta}_1\)</span>) and uses them as the intercept and slope of the line. Here we also use <code>lwd</code> to modify the width of the line, as well as <code>col</code> to modify the color of the line.</p>
 <p>The “thing” that is returned by the <code>lm()</code> function is actually an object of class <code>lm</code> which is a list. The exact details of this are unimportant unless you are seriously interested in the inner-workings of <code>R</code>, but know that we can determine the names of the elements of the list using the <code>names()</code> command.</p>
-<div class="sourceCode" id="cb482"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb482-1"><a href="simple-linear-regression.html#cb482-1" aria-hidden="true" tabindex="-1"></a><span class="fu">names</span>(stop_dist_model)</span></code></pre></div>
+<div class="sourceCode" id="cb482"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb482-1"><a href="simple-linear-regression.html#cb482-1" tabindex="-1"></a><span class="fu">names</span>(stop_dist_model)</span></code></pre></div>
 <pre><code>##  [1] &quot;coefficients&quot;  &quot;residuals&quot;     &quot;effects&quot;       &quot;rank&quot;         
 ##  [5] &quot;fitted.values&quot; &quot;assign&quot;        &quot;qr&quot;            &quot;df.residual&quot;  
 ##  [9] &quot;xlevels&quot;       &quot;call&quot;          &quot;terms&quot;         &quot;model&quot;</code></pre>
 <p>We can then use this information to, for example, access the residuals using the <code>$</code> operator.</p>
-<div class="sourceCode" id="cb484"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb484-1"><a href="simple-linear-regression.html#cb484-1" aria-hidden="true" tabindex="-1"></a>stop_dist_model<span class="sc">$</span>residuals</span></code></pre></div>
+<div class="sourceCode" id="cb484"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb484-1"><a href="simple-linear-regression.html#cb484-1" tabindex="-1"></a>stop_dist_model<span class="sc">$</span>residuals</span></code></pre></div>
 <pre><code>##          1          2          3          4          5          6          7 
 ##   3.849460  11.849460  -5.947766  12.052234   2.119825  -7.812584  -3.744993 
 ##          8          9         10         11         12         13         14 
@@ -901,10 +901,10 @@ <h2><span class="header-section-number">7.4</span> The <code>lm</code> Function<
 ##         50 
 ##   4.268876</code></pre>
 <p>Another way to access stored information in <code>stop_dist_model</code> are the <code>coef()</code>, <code>resid()</code>, and <code>fitted()</code> functions. These return the coefficients, residuals, and fitted values, respectively.</p>
-<div class="sourceCode" id="cb486"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb486-1"><a href="simple-linear-regression.html#cb486-1" aria-hidden="true" tabindex="-1"></a><span class="fu">coef</span>(stop_dist_model)</span></code></pre></div>
+<div class="sourceCode" id="cb486"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb486-1"><a href="simple-linear-regression.html#cb486-1" tabindex="-1"></a><span class="fu">coef</span>(stop_dist_model)</span></code></pre></div>
 <pre><code>## (Intercept)       speed 
 ##  -17.579095    3.932409</code></pre>
-<div class="sourceCode" id="cb488"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb488-1"><a href="simple-linear-regression.html#cb488-1" aria-hidden="true" tabindex="-1"></a><span class="fu">resid</span>(stop_dist_model)</span></code></pre></div>
+<div class="sourceCode" id="cb488"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb488-1"><a href="simple-linear-regression.html#cb488-1" tabindex="-1"></a><span class="fu">resid</span>(stop_dist_model)</span></code></pre></div>
 <pre><code>##          1          2          3          4          5          6          7 
 ##   3.849460  11.849460  -5.947766  12.052234   2.119825  -7.812584  -3.744993 
 ##          8          9         10         11         12         13         14 
@@ -921,7 +921,7 @@ <h2><span class="header-section-number">7.4</span> The <code>lm</code> Function<
 ##   2.930920  -2.933898 -18.866307  -6.798715  15.201285  16.201285  43.201285 
 ##         50 
 ##   4.268876</code></pre>
-<div class="sourceCode" id="cb490"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb490-1"><a href="simple-linear-regression.html#cb490-1" aria-hidden="true" tabindex="-1"></a><span class="fu">fitted</span>(stop_dist_model)</span></code></pre></div>
+<div class="sourceCode" id="cb490"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb490-1"><a href="simple-linear-regression.html#cb490-1" tabindex="-1"></a><span class="fu">fitted</span>(stop_dist_model)</span></code></pre></div>
 <pre><code>##         1         2         3         4         5         6         7         8 
 ## -1.849460 -1.849460  9.947766  9.947766 13.880175 17.812584 21.744993 21.744993 
 ##         9        10        11        12        13        14        15        16 
@@ -937,7 +937,7 @@ <h2><span class="header-section-number">7.4</span> The <code>lm</code> Function<
 ##        49        50 
 ## 76.798715 80.731124</code></pre>
 <p>An <code>R</code> function that is useful in many situations is <code>summary()</code>. We see that when it is called on our model, it returns a good deal of information. By the end of the course, you will know what every value here is used for. For now, you should immediately notice the coefficient estimates, and you may recognize the <span class="math inline">\(R^2\)</span> value we saw earlier.</p>
-<div class="sourceCode" id="cb492"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb492-1"><a href="simple-linear-regression.html#cb492-1" aria-hidden="true" tabindex="-1"></a><span class="fu">summary</span>(stop_dist_model)</span></code></pre></div>
+<div class="sourceCode" id="cb492"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb492-1"><a href="simple-linear-regression.html#cb492-1" tabindex="-1"></a><span class="fu">summary</span>(stop_dist_model)</span></code></pre></div>
 <pre><code>## 
 ## Call:
 ## lm(formula = dist ~ speed, data = cars)
@@ -957,15 +957,15 @@ <h2><span class="header-section-number">7.4</span> The <code>lm</code> Function<
 ## Multiple R-squared:  0.6511, Adjusted R-squared:  0.6438 
 ## F-statistic: 89.57 on 1 and 48 DF,  p-value: 1.49e-12</code></pre>
 <p>The <code>summary()</code> command also returns a list, and we can again use <code>names()</code> to learn what about the elements of this list.</p>
-<div class="sourceCode" id="cb494"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb494-1"><a href="simple-linear-regression.html#cb494-1" aria-hidden="true" tabindex="-1"></a><span class="fu">names</span>(<span class="fu">summary</span>(stop_dist_model))</span></code></pre></div>
+<div class="sourceCode" id="cb494"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb494-1"><a href="simple-linear-regression.html#cb494-1" tabindex="-1"></a><span class="fu">names</span>(<span class="fu">summary</span>(stop_dist_model))</span></code></pre></div>
 <pre><code>##  [1] &quot;call&quot;          &quot;terms&quot;         &quot;residuals&quot;     &quot;coefficients&quot; 
 ##  [5] &quot;aliased&quot;       &quot;sigma&quot;         &quot;df&quot;            &quot;r.squared&quot;    
 ##  [9] &quot;adj.r.squared&quot; &quot;fstatistic&quot;    &quot;cov.unscaled&quot;</code></pre>
 <p>So, for example, if we wanted to directly access the value of <span class="math inline">\(R^2\)</span>, instead of copy and pasting it out of the printed statement from <code>summary()</code>, we could do so.</p>
-<div class="sourceCode" id="cb496"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb496-1"><a href="simple-linear-regression.html#cb496-1" aria-hidden="true" tabindex="-1"></a><span class="fu">summary</span>(stop_dist_model)<span class="sc">$</span>r.squared</span></code></pre></div>
+<div class="sourceCode" id="cb496"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb496-1"><a href="simple-linear-regression.html#cb496-1" tabindex="-1"></a><span class="fu">summary</span>(stop_dist_model)<span class="sc">$</span>r.squared</span></code></pre></div>
 <pre><code>## [1] 0.6510794</code></pre>
 <p>Another value we may want to access is <span class="math inline">\(s_e\)</span>, which <code>R</code> calls <code>sigma</code>.</p>
-<div class="sourceCode" id="cb498"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb498-1"><a href="simple-linear-regression.html#cb498-1" aria-hidden="true" tabindex="-1"></a><span class="fu">summary</span>(stop_dist_model)<span class="sc">$</span>sigma</span></code></pre></div>
+<div class="sourceCode" id="cb498"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb498-1"><a href="simple-linear-regression.html#cb498-1" tabindex="-1"></a><span class="fu">summary</span>(stop_dist_model)<span class="sc">$</span>sigma</span></code></pre></div>
 <pre><code>## [1] 15.37959</code></pre>
 <p>Note that this is the same result seen earlier as <code>s_e</code>. You may also notice that this value was displayed above as a result of the <code>summary()</code> command, which <code>R</code> labeled the “Residual Standard Error.”</p>
 <p><span class="math display">\[
@@ -973,12 +973,12 @@ <h2><span class="header-section-number">7.4</span> The <code>lm</code> Function<
 \]</span></p>
 <p>Often it is useful to talk about <span class="math inline">\(s_e\)</span> (or RSE) instead of <span class="math inline">\(s_e^2\)</span> because of their units. The unit of <span class="math inline">\(s_e\)</span> in the <code>cars</code> example is feet, while the unit of <span class="math inline">\(s_e^2\)</span> is feet-squared.</p>
 <p>Another useful function, which we will use almost as often as <code>lm()</code> is the <code>predict()</code> function.</p>
-<div class="sourceCode" id="cb500"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb500-1"><a href="simple-linear-regression.html#cb500-1" aria-hidden="true" tabindex="-1"></a><span class="fu">predict</span>(stop_dist_model, <span class="at">newdata =</span> <span class="fu">data.frame</span>(<span class="at">speed =</span> <span class="dv">8</span>))</span></code></pre></div>
+<div class="sourceCode" id="cb500"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb500-1"><a href="simple-linear-regression.html#cb500-1" tabindex="-1"></a><span class="fu">predict</span>(stop_dist_model, <span class="at">newdata =</span> <span class="fu">data.frame</span>(<span class="at">speed =</span> <span class="dv">8</span>))</span></code></pre></div>
 <pre><code>##        1 
 ## 13.88018</code></pre>
 <p>The above code reads “predict the stopping distance of a car traveling 8 miles per hour using the <code>stop_dist_model</code>.” Importantly, the second argument to <code>predict()</code> is a data frame that we make in place. We do this so that we can specify that <code>8</code> is a value of <code>speed</code>, so that predict knows how to use it with the model stored in <code>stop_dist_model</code>. We see that this result is what we had calculated “by hand” previously.</p>
 <p>We could also predict multiple values at once.</p>
-<div class="sourceCode" id="cb502"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb502-1"><a href="simple-linear-regression.html#cb502-1" aria-hidden="true" tabindex="-1"></a><span class="fu">predict</span>(stop_dist_model, <span class="at">newdata =</span> <span class="fu">data.frame</span>(<span class="at">speed =</span> <span class="fu">c</span>(<span class="dv">8</span>, <span class="dv">21</span>, <span class="dv">50</span>)))</span></code></pre></div>
+<div class="sourceCode" id="cb502"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb502-1"><a href="simple-linear-regression.html#cb502-1" tabindex="-1"></a><span class="fu">predict</span>(stop_dist_model, <span class="at">newdata =</span> <span class="fu">data.frame</span>(<span class="at">speed =</span> <span class="fu">c</span>(<span class="dv">8</span>, <span class="dv">21</span>, <span class="dv">50</span>)))</span></code></pre></div>
 <pre><code>##         1         2         3 
 ##  13.88018  65.00149 179.04134</code></pre>
 <p><span class="math display">\[
@@ -989,7 +989,7 @@ <h2><span class="header-section-number">7.4</span> The <code>lm</code> Function<
 \end{aligned}
 \]</span></p>
 <p>Or we could calculate the fitted value for each of the original data points. We can simply supply the original data frame, <code>cars</code>, since it contains a variables called <code>speed</code> which has the values we would like to predict at.</p>
-<div class="sourceCode" id="cb504"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb504-1"><a href="simple-linear-regression.html#cb504-1" aria-hidden="true" tabindex="-1"></a><span class="fu">predict</span>(stop_dist_model, <span class="at">newdata =</span> cars)</span></code></pre></div>
+<div class="sourceCode" id="cb504"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb504-1"><a href="simple-linear-regression.html#cb504-1" tabindex="-1"></a><span class="fu">predict</span>(stop_dist_model, <span class="at">newdata =</span> cars)</span></code></pre></div>
 <pre><code>##         1         2         3         4         5         6         7         8 
 ## -1.849460 -1.849460  9.947766  9.947766 13.880175 17.812584 21.744993 21.744993 
 ##         9        10        11        12        13        14        15        16 
@@ -1004,9 +1004,9 @@ <h2><span class="header-section-number">7.4</span> The <code>lm</code> Function<
 ## 61.069080 61.069080 61.069080 68.933898 72.866307 76.798715 76.798715 76.798715 
 ##        49        50 
 ## 76.798715 80.731124</code></pre>
-<div class="sourceCode" id="cb506"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb506-1"><a href="simple-linear-regression.html#cb506-1" aria-hidden="true" tabindex="-1"></a><span class="co"># predict(stop_dist_model, newdata = data.frame(speed = cars$speed))</span></span></code></pre></div>
+<div class="sourceCode" id="cb506"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb506-1"><a href="simple-linear-regression.html#cb506-1" tabindex="-1"></a><span class="co"># predict(stop_dist_model, newdata = data.frame(speed = cars$speed))</span></span></code></pre></div>
 <p>This is actually equivalent to simply calling <code>predict()</code> on <code>stop_dist_model</code> without a second argument.</p>
-<div class="sourceCode" id="cb507"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb507-1"><a href="simple-linear-regression.html#cb507-1" aria-hidden="true" tabindex="-1"></a><span class="fu">predict</span>(stop_dist_model)</span></code></pre></div>
+<div class="sourceCode" id="cb507"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb507-1"><a href="simple-linear-regression.html#cb507-1" tabindex="-1"></a><span class="fu">predict</span>(stop_dist_model)</span></code></pre></div>
 <pre><code>##         1         2         3         4         5         6         7         8 
 ## -1.849460 -1.849460  9.947766  9.947766 13.880175 17.812584 21.744993 21.744993 
 ##         9        10        11        12        13        14        15        16 
@@ -1022,7 +1022,7 @@ <h2><span class="header-section-number">7.4</span> The <code>lm</code> Function<
 ##        49        50 
 ## 76.798715 80.731124</code></pre>
 <p>Note that then in this case, this is the same as using <code>fitted()</code>.</p>
-<div class="sourceCode" id="cb509"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb509-1"><a href="simple-linear-regression.html#cb509-1" aria-hidden="true" tabindex="-1"></a><span class="fu">fitted</span>(stop_dist_model)</span></code></pre></div>
+<div class="sourceCode" id="cb509"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb509-1"><a href="simple-linear-regression.html#cb509-1" tabindex="-1"></a><span class="fu">fitted</span>(stop_dist_model)</span></code></pre></div>
 <pre><code>##         1         2         3         4         5         6         7         8 
 ## -1.849460 -1.849460  9.947766  9.947766 13.880175 17.812584 21.744993 21.744993 
 ##         9        10        11        12        13        14        15        16 
@@ -1124,41 +1124,41 @@ <h2><span class="header-section-number">7.6</span> Simulating SLR<a href="simple
 Y \mid X \sim N(\mu = 5 - 2 x, \sigma^ 2 = 9).
 \]</span></p>
 <p>We first set the true parameters of the model to be simulated.</p>
-<div class="sourceCode" id="cb511"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb511-1"><a href="simple-linear-regression.html#cb511-1" aria-hidden="true" tabindex="-1"></a>num_obs <span class="ot">=</span> <span class="dv">21</span></span>
-<span id="cb511-2"><a href="simple-linear-regression.html#cb511-2" aria-hidden="true" tabindex="-1"></a>beta_0  <span class="ot">=</span> <span class="dv">5</span></span>
-<span id="cb511-3"><a href="simple-linear-regression.html#cb511-3" aria-hidden="true" tabindex="-1"></a>beta_1  <span class="ot">=</span> <span class="sc">-</span><span class="dv">2</span></span>
-<span id="cb511-4"><a href="simple-linear-regression.html#cb511-4" aria-hidden="true" tabindex="-1"></a>sigma   <span class="ot">=</span> <span class="dv">3</span></span></code></pre></div>
+<div class="sourceCode" id="cb511"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb511-1"><a href="simple-linear-regression.html#cb511-1" tabindex="-1"></a>num_obs <span class="ot">=</span> <span class="dv">21</span></span>
+<span id="cb511-2"><a href="simple-linear-regression.html#cb511-2" tabindex="-1"></a>beta_0  <span class="ot">=</span> <span class="dv">5</span></span>
+<span id="cb511-3"><a href="simple-linear-regression.html#cb511-3" tabindex="-1"></a>beta_1  <span class="ot">=</span> <span class="sc">-</span><span class="dv">2</span></span>
+<span id="cb511-4"><a href="simple-linear-regression.html#cb511-4" tabindex="-1"></a>sigma   <span class="ot">=</span> <span class="dv">3</span></span></code></pre></div>
 <p>Next, we obtain simulated values of <span class="math inline">\(\epsilon_i\)</span> after setting a seed for reproducibility.</p>
-<div class="sourceCode" id="cb512"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb512-1"><a href="simple-linear-regression.html#cb512-1" aria-hidden="true" tabindex="-1"></a><span class="fu">set.seed</span>(<span class="dv">1</span>)</span>
-<span id="cb512-2"><a href="simple-linear-regression.html#cb512-2" aria-hidden="true" tabindex="-1"></a>epsilon <span class="ot">=</span> <span class="fu">rnorm</span>(<span class="at">n =</span> num_obs, <span class="at">mean =</span> <span class="dv">0</span>, <span class="at">sd =</span> sigma)</span></code></pre></div>
+<div class="sourceCode" id="cb512"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb512-1"><a href="simple-linear-regression.html#cb512-1" tabindex="-1"></a><span class="fu">set.seed</span>(<span class="dv">1</span>)</span>
+<span id="cb512-2"><a href="simple-linear-regression.html#cb512-2" tabindex="-1"></a>epsilon <span class="ot">=</span> <span class="fu">rnorm</span>(<span class="at">n =</span> num_obs, <span class="at">mean =</span> <span class="dv">0</span>, <span class="at">sd =</span> sigma)</span></code></pre></div>
 <p>Now, since the <span class="math inline">\(x_i\)</span> values in SLR are considered fixed and known, we simply specify 21 values. Another common practice is to generate them from a uniform distribution, and then use them for the remainder of the analysis.</p>
-<div class="sourceCode" id="cb513"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb513-1"><a href="simple-linear-regression.html#cb513-1" aria-hidden="true" tabindex="-1"></a>x_vals <span class="ot">=</span> <span class="fu">seq</span>(<span class="at">from =</span> <span class="dv">0</span>, <span class="at">to =</span> <span class="dv">10</span>, <span class="at">length.out =</span> num_obs)</span>
-<span id="cb513-2"><a href="simple-linear-regression.html#cb513-2" aria-hidden="true" tabindex="-1"></a><span class="co"># set.seed(1)</span></span>
-<span id="cb513-3"><a href="simple-linear-regression.html#cb513-3" aria-hidden="true" tabindex="-1"></a><span class="co"># x_vals = runif(num_obs, 0, 10)</span></span></code></pre></div>
+<div class="sourceCode" id="cb513"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb513-1"><a href="simple-linear-regression.html#cb513-1" tabindex="-1"></a>x_vals <span class="ot">=</span> <span class="fu">seq</span>(<span class="at">from =</span> <span class="dv">0</span>, <span class="at">to =</span> <span class="dv">10</span>, <span class="at">length.out =</span> num_obs)</span>
+<span id="cb513-2"><a href="simple-linear-regression.html#cb513-2" tabindex="-1"></a><span class="co"># set.seed(1)</span></span>
+<span id="cb513-3"><a href="simple-linear-regression.html#cb513-3" tabindex="-1"></a><span class="co"># x_vals = runif(num_obs, 0, 10)</span></span></code></pre></div>
 <p>We then generate the <span class="math inline">\(y\)</span> values according to the specified functional relationship.</p>
-<div class="sourceCode" id="cb514"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb514-1"><a href="simple-linear-regression.html#cb514-1" aria-hidden="true" tabindex="-1"></a>y_vals <span class="ot">=</span> beta_0 <span class="sc">+</span> beta_1 <span class="sc">*</span> x_vals <span class="sc">+</span> epsilon</span></code></pre></div>
+<div class="sourceCode" id="cb514"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb514-1"><a href="simple-linear-regression.html#cb514-1" tabindex="-1"></a>y_vals <span class="ot">=</span> beta_0 <span class="sc">+</span> beta_1 <span class="sc">*</span> x_vals <span class="sc">+</span> epsilon</span></code></pre></div>
 <p>The data, <span class="math inline">\((x_i, y_i)\)</span>, represent a possible sample from the true distribution. Now to check how well the method of least squares works, we use <code>lm()</code> to fit the model to our simulated data, then take a look at the estimated coefficients.</p>
-<div class="sourceCode" id="cb515"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb515-1"><a href="simple-linear-regression.html#cb515-1" aria-hidden="true" tabindex="-1"></a>sim_fit <span class="ot">=</span> <span class="fu">lm</span>(y_vals <span class="sc">~</span> x_vals)</span>
-<span id="cb515-2"><a href="simple-linear-regression.html#cb515-2" aria-hidden="true" tabindex="-1"></a><span class="fu">coef</span>(sim_fit)</span></code></pre></div>
+<div class="sourceCode" id="cb515"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb515-1"><a href="simple-linear-regression.html#cb515-1" tabindex="-1"></a>sim_fit <span class="ot">=</span> <span class="fu">lm</span>(y_vals <span class="sc">~</span> x_vals)</span>
+<span id="cb515-2"><a href="simple-linear-regression.html#cb515-2" tabindex="-1"></a><span class="fu">coef</span>(sim_fit)</span></code></pre></div>
 <pre><code>## (Intercept)      x_vals 
 ##    4.832639   -1.831401</code></pre>
 <p>And look at that, they aren’t too far from the true parameters we specified!</p>
-<div class="sourceCode" id="cb517"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb517-1"><a href="simple-linear-regression.html#cb517-1" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(y_vals <span class="sc">~</span> x_vals)</span>
-<span id="cb517-2"><a href="simple-linear-regression.html#cb517-2" aria-hidden="true" tabindex="-1"></a><span class="fu">abline</span>(sim_fit)</span></code></pre></div>
+<div class="sourceCode" id="cb517"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb517-1"><a href="simple-linear-regression.html#cb517-1" tabindex="-1"></a><span class="fu">plot</span>(y_vals <span class="sc">~</span> x_vals)</span>
+<span id="cb517-2"><a href="simple-linear-regression.html#cb517-2" tabindex="-1"></a><span class="fu">abline</span>(sim_fit)</span></code></pre></div>
 <p><img src="slr_files/figure-html/unnamed-chunk-51-1.png" width="672" style="display: block; margin: auto;" /></p>
 <p>We should say here, that we’re being sort of lazy, and not the good kind of lazy that could be considered efficient. Any time you simulate data, you should consider doing two things: writing a function, and storing the data in a data frame.</p>
 <p>The function below, <code>sim_slr()</code>, can be used for the same task as above, but is much more flexible. Notice that we provide <code>x</code> to the function, instead of generating <code>x</code> inside the function. In the SLR model, the <span class="math inline">\(x_i\)</span> are considered known values. That is, they are not random, so we do not assume a distribution for the <span class="math inline">\(x_i\)</span>. Because of this, we will repeatedly use the same <code>x</code> values across all simulations.</p>
-<div class="sourceCode" id="cb518"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb518-1"><a href="simple-linear-regression.html#cb518-1" aria-hidden="true" tabindex="-1"></a>sim_slr <span class="ot">=</span> <span class="cf">function</span>(x, <span class="at">beta_0 =</span> <span class="dv">10</span>, <span class="at">beta_1 =</span> <span class="dv">5</span>, <span class="at">sigma =</span> <span class="dv">1</span>) {</span>
-<span id="cb518-2"><a href="simple-linear-regression.html#cb518-2" aria-hidden="true" tabindex="-1"></a>  n <span class="ot">=</span> <span class="fu">length</span>(x)</span>
-<span id="cb518-3"><a href="simple-linear-regression.html#cb518-3" aria-hidden="true" tabindex="-1"></a>  epsilon <span class="ot">=</span> <span class="fu">rnorm</span>(n, <span class="at">mean =</span> <span class="dv">0</span>, <span class="at">sd =</span> sigma)</span>
-<span id="cb518-4"><a href="simple-linear-regression.html#cb518-4" aria-hidden="true" tabindex="-1"></a>  y <span class="ot">=</span> beta_0 <span class="sc">+</span> beta_1 <span class="sc">*</span> x <span class="sc">+</span> epsilon</span>
-<span id="cb518-5"><a href="simple-linear-regression.html#cb518-5" aria-hidden="true" tabindex="-1"></a>  <span class="fu">data.frame</span>(<span class="at">predictor =</span> x, <span class="at">response =</span> y)</span>
-<span id="cb518-6"><a href="simple-linear-regression.html#cb518-6" aria-hidden="true" tabindex="-1"></a>}</span></code></pre></div>
+<div class="sourceCode" id="cb518"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb518-1"><a href="simple-linear-regression.html#cb518-1" tabindex="-1"></a>sim_slr <span class="ot">=</span> <span class="cf">function</span>(x, <span class="at">beta_0 =</span> <span class="dv">10</span>, <span class="at">beta_1 =</span> <span class="dv">5</span>, <span class="at">sigma =</span> <span class="dv">1</span>) {</span>
+<span id="cb518-2"><a href="simple-linear-regression.html#cb518-2" tabindex="-1"></a>  n <span class="ot">=</span> <span class="fu">length</span>(x)</span>
+<span id="cb518-3"><a href="simple-linear-regression.html#cb518-3" tabindex="-1"></a>  epsilon <span class="ot">=</span> <span class="fu">rnorm</span>(n, <span class="at">mean =</span> <span class="dv">0</span>, <span class="at">sd =</span> sigma)</span>
+<span id="cb518-4"><a href="simple-linear-regression.html#cb518-4" tabindex="-1"></a>  y <span class="ot">=</span> beta_0 <span class="sc">+</span> beta_1 <span class="sc">*</span> x <span class="sc">+</span> epsilon</span>
+<span id="cb518-5"><a href="simple-linear-regression.html#cb518-5" tabindex="-1"></a>  <span class="fu">data.frame</span>(<span class="at">predictor =</span> x, <span class="at">response =</span> y)</span>
+<span id="cb518-6"><a href="simple-linear-regression.html#cb518-6" tabindex="-1"></a>}</span></code></pre></div>
 <p>Here, we use the function to repeat the analysis above.</p>
-<div class="sourceCode" id="cb519"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb519-1"><a href="simple-linear-regression.html#cb519-1" aria-hidden="true" tabindex="-1"></a><span class="fu">set.seed</span>(<span class="dv">1</span>)</span>
-<span id="cb519-2"><a href="simple-linear-regression.html#cb519-2" aria-hidden="true" tabindex="-1"></a>sim_data <span class="ot">=</span> <span class="fu">sim_slr</span>(<span class="at">x =</span> x_vals, <span class="at">beta_0 =</span> <span class="dv">5</span>, <span class="at">beta_1 =</span> <span class="sc">-</span><span class="dv">2</span>, <span class="at">sigma =</span> <span class="dv">3</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb519"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb519-1"><a href="simple-linear-regression.html#cb519-1" tabindex="-1"></a><span class="fu">set.seed</span>(<span class="dv">1</span>)</span>
+<span id="cb519-2"><a href="simple-linear-regression.html#cb519-2" tabindex="-1"></a>sim_data <span class="ot">=</span> <span class="fu">sim_slr</span>(<span class="at">x =</span> x_vals, <span class="at">beta_0 =</span> <span class="dv">5</span>, <span class="at">beta_1 =</span> <span class="sc">-</span><span class="dv">2</span>, <span class="at">sigma =</span> <span class="dv">3</span>)</span></code></pre></div>
 <p>This time, the simulated observations are stored in a data frame.</p>
-<div class="sourceCode" id="cb520"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb520-1"><a href="simple-linear-regression.html#cb520-1" aria-hidden="true" tabindex="-1"></a><span class="fu">head</span>(sim_data)</span></code></pre></div>
+<div class="sourceCode" id="cb520"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb520-1"><a href="simple-linear-regression.html#cb520-1" tabindex="-1"></a><span class="fu">head</span>(sim_data)</span></code></pre></div>
 <pre><code>##   predictor   response
 ## 1       0.0  3.1206386
 ## 2       0.5  4.5509300
@@ -1167,22 +1167,22 @@ <h2><span class="header-section-number">7.6</span> Simulating SLR<a href="simple
 ## 5       2.0  1.9885233
 ## 6       2.5 -2.4614052</code></pre>
 <p>Now when we fit the model with <code>lm()</code> we can use a <code>data</code> argument, a very good practice.</p>
-<div class="sourceCode" id="cb522"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb522-1"><a href="simple-linear-regression.html#cb522-1" aria-hidden="true" tabindex="-1"></a>sim_fit <span class="ot">=</span> <span class="fu">lm</span>(response <span class="sc">~</span> predictor, <span class="at">data =</span> sim_data)</span>
-<span id="cb522-2"><a href="simple-linear-regression.html#cb522-2" aria-hidden="true" tabindex="-1"></a><span class="fu">coef</span>(sim_fit)</span></code></pre></div>
+<div class="sourceCode" id="cb522"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb522-1"><a href="simple-linear-regression.html#cb522-1" tabindex="-1"></a>sim_fit <span class="ot">=</span> <span class="fu">lm</span>(response <span class="sc">~</span> predictor, <span class="at">data =</span> sim_data)</span>
+<span id="cb522-2"><a href="simple-linear-regression.html#cb522-2" tabindex="-1"></a><span class="fu">coef</span>(sim_fit)</span></code></pre></div>
 <pre><code>## (Intercept)   predictor 
 ##    4.832639   -1.831401</code></pre>
 <p>And this time, we’ll make the plot look a lot nicer.</p>
-<div class="sourceCode" id="cb524"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb524-1"><a href="simple-linear-regression.html#cb524-1" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(response <span class="sc">~</span> predictor, <span class="at">data =</span> sim_data,</span>
-<span id="cb524-2"><a href="simple-linear-regression.html#cb524-2" aria-hidden="true" tabindex="-1"></a>     <span class="at">xlab =</span> <span class="st">&quot;Simulated Predictor Variable&quot;</span>,</span>
-<span id="cb524-3"><a href="simple-linear-regression.html#cb524-3" aria-hidden="true" tabindex="-1"></a>     <span class="at">ylab =</span> <span class="st">&quot;Simulated Response Variable&quot;</span>,</span>
-<span id="cb524-4"><a href="simple-linear-regression.html#cb524-4" aria-hidden="true" tabindex="-1"></a>     <span class="at">main =</span> <span class="st">&quot;Simulated Regression Data&quot;</span>,</span>
-<span id="cb524-5"><a href="simple-linear-regression.html#cb524-5" aria-hidden="true" tabindex="-1"></a>     <span class="at">pch  =</span> <span class="dv">20</span>,</span>
-<span id="cb524-6"><a href="simple-linear-regression.html#cb524-6" aria-hidden="true" tabindex="-1"></a>     <span class="at">cex  =</span> <span class="dv">2</span>,</span>
-<span id="cb524-7"><a href="simple-linear-regression.html#cb524-7" aria-hidden="true" tabindex="-1"></a>     <span class="at">col  =</span> <span class="st">&quot;grey&quot;</span>)</span>
-<span id="cb524-8"><a href="simple-linear-regression.html#cb524-8" aria-hidden="true" tabindex="-1"></a><span class="fu">abline</span>(sim_fit, <span class="at">lwd =</span> <span class="dv">3</span>, <span class="at">lty =</span> <span class="dv">1</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>)</span>
-<span id="cb524-9"><a href="simple-linear-regression.html#cb524-9" aria-hidden="true" tabindex="-1"></a><span class="fu">abline</span>(beta_0, beta_1, <span class="at">lwd =</span> <span class="dv">3</span>, <span class="at">lty =</span> <span class="dv">2</span>, <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>)</span>
-<span id="cb524-10"><a href="simple-linear-regression.html#cb524-10" aria-hidden="true" tabindex="-1"></a><span class="fu">legend</span>(<span class="st">&quot;topright&quot;</span>, <span class="fu">c</span>(<span class="st">&quot;Estimate&quot;</span>, <span class="st">&quot;Truth&quot;</span>), <span class="at">lty =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>), <span class="at">lwd =</span> <span class="dv">2</span>,</span>
-<span id="cb524-11"><a href="simple-linear-regression.html#cb524-11" aria-hidden="true" tabindex="-1"></a>       <span class="at">col =</span> <span class="fu">c</span>(<span class="st">&quot;darkorange&quot;</span>, <span class="st">&quot;dodgerblue&quot;</span>))</span></code></pre></div>
+<div class="sourceCode" id="cb524"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb524-1"><a href="simple-linear-regression.html#cb524-1" tabindex="-1"></a><span class="fu">plot</span>(response <span class="sc">~</span> predictor, <span class="at">data =</span> sim_data,</span>
+<span id="cb524-2"><a href="simple-linear-regression.html#cb524-2" tabindex="-1"></a>     <span class="at">xlab =</span> <span class="st">&quot;Simulated Predictor Variable&quot;</span>,</span>
+<span id="cb524-3"><a href="simple-linear-regression.html#cb524-3" tabindex="-1"></a>     <span class="at">ylab =</span> <span class="st">&quot;Simulated Response Variable&quot;</span>,</span>
+<span id="cb524-4"><a href="simple-linear-regression.html#cb524-4" tabindex="-1"></a>     <span class="at">main =</span> <span class="st">&quot;Simulated Regression Data&quot;</span>,</span>
+<span id="cb524-5"><a href="simple-linear-regression.html#cb524-5" tabindex="-1"></a>     <span class="at">pch  =</span> <span class="dv">20</span>,</span>
+<span id="cb524-6"><a href="simple-linear-regression.html#cb524-6" tabindex="-1"></a>     <span class="at">cex  =</span> <span class="dv">2</span>,</span>
+<span id="cb524-7"><a href="simple-linear-regression.html#cb524-7" tabindex="-1"></a>     <span class="at">col  =</span> <span class="st">&quot;grey&quot;</span>)</span>
+<span id="cb524-8"><a href="simple-linear-regression.html#cb524-8" tabindex="-1"></a><span class="fu">abline</span>(sim_fit, <span class="at">lwd =</span> <span class="dv">3</span>, <span class="at">lty =</span> <span class="dv">1</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>)</span>
+<span id="cb524-9"><a href="simple-linear-regression.html#cb524-9" tabindex="-1"></a><span class="fu">abline</span>(beta_0, beta_1, <span class="at">lwd =</span> <span class="dv">3</span>, <span class="at">lty =</span> <span class="dv">2</span>, <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>)</span>
+<span id="cb524-10"><a href="simple-linear-regression.html#cb524-10" tabindex="-1"></a><span class="fu">legend</span>(<span class="st">&quot;topright&quot;</span>, <span class="fu">c</span>(<span class="st">&quot;Estimate&quot;</span>, <span class="st">&quot;Truth&quot;</span>), <span class="at">lty =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>), <span class="at">lwd =</span> <span class="dv">2</span>,</span>
+<span id="cb524-11"><a href="simple-linear-regression.html#cb524-11" tabindex="-1"></a>       <span class="at">col =</span> <span class="fu">c</span>(<span class="st">&quot;darkorange&quot;</span>, <span class="st">&quot;dodgerblue&quot;</span>))</span></code></pre></div>
 <p><img src="slr_files/figure-html/unnamed-chunk-56-1.png" width="672" style="display: block; margin: auto;" /></p>
 </div>
 <div id="history" class="section level2 hasAnchor" number="7.7">
@@ -1195,7 +1195,7 @@ <h2><span class="header-section-number">7.8</span> <code>R</code> Markdown<a hre
 <ul>
 <li><a href="slr.Rmd" target="_blank"><code>slr.Rmd</code></a></li>
 </ul>
-<p>The file was created using <code>R</code> version <code>4.3.2</code>.</p>
+<p>The file was created using <code>R</code> version <code>4.4.1</code>.</p>
 
 </div>
 </div>
diff --git a/slr-inf.md b/slr-inf.md
index 07786b6..cdfb4b3 100644
--- a/slr-inf.md
+++ b/slr-inf.md
@@ -44,7 +44,7 @@ which we also called $\text{RSE}$, for "Residual Standard Error."
 When applied to the `cars` data, we obtained the following results:
 
 
-```r
+``` r
 stop_dist_model = lm(dist ~ speed, data = cars)
 summary(stop_dist_model)
 ```
@@ -73,7 +73,7 @@ summary(stop_dist_model)
 Last chapter, we only discussed the `Estimate`, `Residual standard error`, and `Multiple R-squared` values. In this chapter, we will discuss all of the information under `Coefficients` as well as `F-statistic`.
 
 
-```r
+``` r
 plot(dist ~ speed, data = cars,
      xlab = "Speed (in Miles Per Hour)",
      ylab = "Stopping Distance (in Feet)",
@@ -245,7 +245,7 @@ and
 First we need to decide ahead of time what our $x$ values will be for this simulation, since the $x$ values in SLR are also considered known quantities. The choice of $x$ values is arbitrary. Here we also set a seed for randomization, and calculate $S_{xx}$ which we will need going forward.
 
 
-```r
+``` r
 set.seed(42)
 sample_size = 100 # this is n
 x = seq(-1, 1, length = sample_size)
@@ -255,7 +255,7 @@ Sxx = sum((x - mean(x)) ^ 2)
 We also fix our parameter values.
 
 
-```r
+``` r
 beta_0 = 3
 beta_1 = 6
 sigma  = 2
@@ -264,7 +264,7 @@ sigma  = 2
 With this information, we know the sampling distributions should be:
 
 
-```r
+``` r
 (var_beta_1_hat = sigma ^ 2 / Sxx)
 ```
 
@@ -272,7 +272,7 @@ With this information, we know the sampling distributions should be:
 ## [1] 0.1176238
 ```
 
-```r
+``` r
 (var_beta_0_hat = sigma ^ 2 * (1 / sample_size + mean(x) ^ 2 / Sxx))
 ```
 
@@ -311,7 +311,7 @@ and
 We now simulate data from this model 10,000 times. Note this may not be the most `R` way of doing the simulation. We perform the simulation in this manner in an attempt at clarity. For example, we could have used the `sim_slr()` function from the previous chapter. We also simply store variables in the global environment instead of creating a data frame for each new simulated dataset.
 
 
-```r
+``` r
 num_samples = 10000
 beta_0_hats = rep(0, num_samples)
 beta_1_hats = rep(0, num_samples)
@@ -332,7 +332,7 @@ Each time we simulated the data, we obtained values of the estimated coefficient
 We first verify the distribution of $\hat{\beta}_1$.
 
 
-```r
+``` r
 mean(beta_1_hats) # empirical mean
 ```
 
@@ -340,7 +340,7 @@ mean(beta_1_hats) # empirical mean
 ## [1] 6.001998
 ```
 
-```r
+``` r
 beta_1            # true mean
 ```
 
@@ -348,7 +348,7 @@ beta_1            # true mean
 ## [1] 6
 ```
 
-```r
+``` r
 var(beta_1_hats)  # empirical variance
 ```
 
@@ -356,7 +356,7 @@ var(beta_1_hats)  # empirical variance
 ## [1] 0.11899
 ```
 
-```r
+``` r
 var_beta_1_hat    # true variance
 ```
 
@@ -367,7 +367,7 @@ var_beta_1_hat    # true variance
 We see that the empirical and true means and variances are *very* similar. We also verify that the empirical distribution is normal. To do so, we plot a histogram of the `beta_1_hats`, and add the curve for the true distribution of $\hat{\beta}_1$. We use `prob = TRUE` to put the histogram on the same scale as the normal curve.
 
 
-```r
+``` r
 # note need to use prob = TRUE
 hist(beta_1_hats, prob = TRUE, breaks = 20, 
      xlab = expression(hat(beta)[1]), main = "", border = "dodgerblue")
@@ -382,7 +382,7 @@ curve(dnorm(x, mean = beta_1, sd = sqrt(var_beta_1_hat)),
 We then repeat the process for $\hat{\beta}_0$.
 
 
-```r
+``` r
 mean(beta_0_hats) # empirical mean
 ```
 
@@ -390,7 +390,7 @@ mean(beta_0_hats) # empirical mean
 ## [1] 3.001147
 ```
 
-```r
+``` r
 beta_0            # true mean
 ```
 
@@ -398,7 +398,7 @@ beta_0            # true mean
 ## [1] 3
 ```
 
-```r
+``` r
 var(beta_0_hats)  # empirical variance
 ```
 
@@ -406,7 +406,7 @@ var(beta_0_hats)  # empirical variance
 ## [1] 0.04017924
 ```
 
-```r
+``` r
 var_beta_0_hat    # true variance
 ```
 
@@ -415,7 +415,7 @@ var_beta_0_hat    # true variance
 ```
 
 
-```r
+``` r
 hist(beta_0_hats, prob = TRUE, breaks = 25, 
      xlab = expression(hat(beta)[0]), main = "", border = "dodgerblue")
 curve(dnorm(x, mean = beta_0, sd = sqrt(var_beta_0_hat)),
@@ -429,7 +429,7 @@ curve(dnorm(x, mean = beta_0, sd = sqrt(var_beta_0_hat)),
 In this simulation study, we have only simulated a finite number of samples. To truly verify the distributional results, we would need to observe an infinite number of samples. However, the following plot should make it clear that if we continued simulating, the empirical results would get closer and closer to what we should expect.
 
 
-```r
+``` r
 par(mar = c(5, 5, 1, 1)) # adjusted plot margins, otherwise the "hat" does not display
 plot(cumsum(beta_1_hats) / (1:length(beta_1_hats)), type = "l", ylim = c(5.95, 6.05),
      xlab = "Number of Simulations",
@@ -442,7 +442,7 @@ abline(h = 6, col = "darkorange", lwd = 2)
 
 \begin{center}\includegraphics{slr-inf_files/figure-latex/unnamed-chunk-12-1} \end{center}
 
-```r
+``` r
 par(mar = c(5, 5, 1, 1)) # adjusted plot margins, otherwise the "hat" does not display
 plot(cumsum(beta_0_hats) / (1:length(beta_0_hats)), type = "l", ylim = c(2.95, 3.05),
      xlab = "Number of Simulations",
@@ -554,7 +554,7 @@ where $Z \sim N(0,1)$.
 Recall that a $t$ distribution is similar to a standard normal, but with heavier tails. As the degrees of freedom increases, the $t$ distribution becomes more and more like a standard normal. Below we plot a standard normal distribution as well as two examples of a $t$ distribution with different degrees of freedom. Notice how the $t$ distribution with the larger degrees of freedom is more similar to the standard normal curve.
 
 
-```r
+``` r
 # define grid of x values
 x = seq(-4, 4, length = 100)
 
@@ -652,7 +652,7 @@ which again, under the null hypothesis, follows a $t$ distribution with $n - 2$
 We now return to the `cars` example from last chapter to illustrate these concepts. We first fit the model using `lm()` then use `summary()` to view the results in greater detail.
 
 
-```r
+``` r
 stop_dist_model = lm(dist ~ speed, data = cars)
 summary(stop_dist_model)
 ```
@@ -683,7 +683,7 @@ summary(stop_dist_model)
 We will now discuss the results displayed called `Coefficients`. First recall that we can extract this information directly.
 
 
-```r
+``` r
 names(summary(stop_dist_model))
 ```
 
@@ -693,7 +693,7 @@ names(summary(stop_dist_model))
 ##  [9] "adj.r.squared" "fstatistic"    "cov.unscaled"
 ```
 
-```r
+``` r
 summary(stop_dist_model)$coefficients
 ```
 
@@ -720,7 +720,7 @@ which are our estimates for the model parameters $\beta_0$ and $\beta_1$.
 Let's now focus on the second row of output, which is relevant to $\beta_1$.
 
 
-```r
+``` r
 summary(stop_dist_model)$coefficients[2,]
 ```
 
@@ -758,7 +758,7 @@ Note here, we are specifically testing whether or not $\beta_1 = 0$.
 The first row of output reports the same values, but for $\beta_0$.
 
 
-```r
+``` r
 summary(stop_dist_model)$coefficients[1,]
 ```
 
@@ -770,7 +770,7 @@ summary(stop_dist_model)$coefficients[1,]
 In summary, the following code stores the information of `summary(stop_dist_model)$coefficients` in a new variable `stop_dist_model_test_info`, then extracts each element into a new variable which describes the information it contains.
 
 
-```r
+``` r
 stop_dist_model_test_info = summary(stop_dist_model)$coefficients
 
 beta_0_hat      = stop_dist_model_test_info[1, 1] # Estimate
@@ -787,7 +787,7 @@ beta_1_hat_pval = stop_dist_model_test_info[2, 4] # Pr(>|t|)
 We can then verify some equivalent expressions: the $t$ test statistic for $\hat{\beta}_1$ and the two-sided p-value associated with that test statistic.
 
 
-```r
+``` r
 (beta_1_hat - 0) / beta_1_hat_se
 ```
 
@@ -795,7 +795,7 @@ We can then verify some equivalent expressions: the $t$ test statistic for $\hat
 ## [1] 9.46399
 ```
 
-```r
+``` r
 beta_1_hat_t
 ```
 
@@ -804,7 +804,7 @@ beta_1_hat_t
 ```
 
 
-```r
+``` r
 2 * pt(abs(beta_1_hat_t), df = length(resid(stop_dist_model)) - 2, lower.tail = FALSE)
 ```
 
@@ -812,7 +812,7 @@ beta_1_hat_t
 ## [1] 1.489836e-12
 ```
 
-```r
+``` r
 beta_1_hat_pval
 ```
 
@@ -870,7 +870,7 @@ In this plot of simulated data, we see a clear relationship between $x$ and $y$,
 Using `R` we can very easily obtain the confidence intervals for $\beta_0$ and $\beta_1$.
 
 
-```r
+``` r
 confint(stop_dist_model, level = 0.99)
 ```
 
@@ -891,7 +891,7 @@ You should be somewhat suspicious of the confidence interval for $\beta_0$, as i
 Note, we can extract specific values from this output a number of ways. This code is not run, and instead, you should check how it relates to the output of the code above.
 
 
-```r
+``` r
 confint(stop_dist_model, level = 0.99)[1,]
 confint(stop_dist_model, level = 0.99)[1, 1]
 confint(stop_dist_model, level = 0.99)[1, 2]
@@ -905,7 +905,7 @@ confint(stop_dist_model, parm = "speed", level = 0.99)
 We can also verify that calculations that `R` is performing for the $\beta_1$ interval.
 
 
-```r
+``` r
 # store estimate
 beta_1_hat = coef(stop_dist_model)[2]
 
@@ -973,7 +973,7 @@ We can then use this to find the confidence interval for the mean response,
 To find confidence intervals for the mean response using `R`, we use the `predict()` function. We give the function our fitted model as well as new data, stored as a data frame. (This is important, so that `R` knows the name of the predictor variable.) Here, we are finding the confidence interval for the mean stopping distance when a car is travelling 5 miles per hour and when a car is travelling 21 miles per hour.
 
 
-```r
+``` r
 new_speeds = data.frame(speed = c(5, 21))
 predict(stop_dist_model, newdata = new_speeds, 
         interval = c("confidence"), level = 0.99)
@@ -1016,7 +1016,7 @@ We can then find a **prediction interval** using,
 To calculate this for a set of points in `R` notice there is only a minor change in syntax from finding a confidence interval for the mean response.
 
 
-```r
+``` r
 predict(stop_dist_model, newdata = new_speeds, 
         interval = c("prediction"), level = 0.99)
 ```
@@ -1034,7 +1034,7 @@ Also notice that these two intervals are wider than the corresponding confidence
 Often we will like to plot both confidence intervals for the mean response and prediction intervals for all possible values of $x$. We calls these confidence and prediction bands.
 
 
-```r
+``` r
 speed_grid = seq(min(cars$speed), max(cars$speed), by = 0.01)
 dist_ci_band = predict(stop_dist_model, 
                        newdata = data.frame(speed = speed_grid), 
@@ -1127,7 +1127,7 @@ In particular, we will reject the null when the $F$ statistic is large, that is,
 To perform the $F$ test in `R` you can look at the last row of the output from `summary()` called `F-statistic` which gives the value of the test statistic, the relevant degrees of freedom, as well as the p-value of the test.
 
 
-```r
+``` r
 summary(stop_dist_model)
 ```
 
@@ -1155,7 +1155,7 @@ summary(stop_dist_model)
 Additionally, you can use the `anova()` function to display the information in an ANOVA table.
 
 
-```r
+``` r
 anova(stop_dist_model)
 ```
 
@@ -1175,7 +1175,7 @@ This also gives a p-value for the test. You should notice that the p-value from
 Note that there is another equivalent way to do this in `R`, which we will return to often to compare two models.
 
 
-```r
+``` r
 anova(lm(dist ~ 1, data = cars), lm(dist ~ speed, data = cars))
 ```
 
@@ -1203,4 +1203,4 @@ The `R` Markdown file for this chapter can be found here:
 
 - [`slr-inf.Rmd`](slr-inf.Rmd){target="_blank"}
 
-The file was created using `R` version `4.3.2`.
+The file was created using `R` version `4.4.1`.
diff --git a/slr.Rmd b/slr.Rmd
index 232fb43..63c17f3 100644
--- a/slr.Rmd
+++ b/slr.Rmd
@@ -174,7 +174,7 @@ $$
 
 This is visually displayed in the image below. We see that for any value $x$, the expected value of $Y$ is $\beta_0 + \beta_1 x$. At each value of $x$, $Y$ has the same variance $\sigma^2$.
 
-![Simple Linear Regression Model [Introductory Statistics (Shafer and Zhang), UC Davis Stat Wiki](http://statwiki.ucdavis.edu/Textbook_Maps/General_Statistics/Map%3A_Introductory_Statistics_(Shafer_and_Zhang)/10%3A_Correlation_and_Regression/10.3_Modelling_Linear_Relationships_with_Randomness_Present){target="_blank"}](images/model.jpg)
+![Simple Linear Regression Model [Introductory Statistics (Shafer and Zhang), UC Davis Stat Wiki](https://stats.libretexts.org/Bookshelves/Introductory_Statistics/Introductory_Statistics_(Shafer_and_Zhang)/10%3A_Correlation_and_Regression/10.03%3A_Modelling_Linear_Relationships_with_Randomness_Present){target="_blank"}](images/model.jpg)
 
 Often, we directly talk about the assumptions that this model makes. They can be cleverly shortened to **LINE**.  
 
diff --git a/slr.md b/slr.md
index 58d420a..2826d6e 100644
--- a/slr.md
+++ b/slr.md
@@ -22,14 +22,14 @@ Let's consider a simple example of how the speed of a car affects its stopping d
 To get a first look at the data you can use the `View()` function inside RStudio.
 
 
-```r
+``` r
 View(cars)
 ```
 
 We could also take a look at the variable names, the dimension of the data frame, and some sample observations with `str()`.
 
 
-```r
+``` r
 str(cars)
 ```
 
@@ -42,7 +42,7 @@ str(cars)
 As we have seen before with data frames, there are a number of additional functions to access some of this information directly.
 
 
-```r
+``` r
 dim(cars)
 ```
 
@@ -50,7 +50,7 @@ dim(cars)
 ## [1] 50  2
 ```
 
-```r
+``` r
 nrow(cars)
 ```
 
@@ -58,7 +58,7 @@ nrow(cars)
 ## [1] 50
 ```
 
-```r
+``` r
 ncol(cars)
 ```
 
@@ -69,14 +69,14 @@ ncol(cars)
 Other than the two variable names and the number of observations, this data is still just a bunch of numbers, so we should probably obtain some context.
 
 
-```r
+``` r
 ?cars
 ```
 
 Reading the documentation we learn that this is data gathered during the 1920s about the speed of cars and the resulting distance it takes for the car to come to a stop. The interesting task here is to determine how far a car travels before stopping, when traveling at a certain speed. So, we will first plot the stopping distance against the speed.
 
 
-```r
+``` r
 plot(dist ~ speed, data = cars,
      xlab = "Speed (in Miles Per Hour)",
      ylab = "Stopping Distance (in Feet)",
@@ -177,7 +177,7 @@ $$
 
 This is visually displayed in the image below. We see that for any value $x$, the expected value of $Y$ is $\beta_0 + \beta_1 x$. At each value of $x$, $Y$ has the same variance $\sigma^2$.
 
-![Simple Linear Regression Model [Introductory Statistics (Shafer and Zhang), UC Davis Stat Wiki](http://statwiki.ucdavis.edu/Textbook_Maps/General_Statistics/Map%3A_Introductory_Statistics_(Shafer_and_Zhang)/10%3A_Correlation_and_Regression/10.3_Modelling_Linear_Relationships_with_Randomness_Present){target="_blank"}](images/model.jpg)
+![Simple Linear Regression Model [Introductory Statistics (Shafer and Zhang), UC Davis Stat Wiki](https://stats.libretexts.org/Bookshelves/Introductory_Statistics/Introductory_Statistics_(Shafer_and_Zhang)/10%3A_Correlation_and_Regression/10.03%3A_Modelling_Linear_Relationships_with_Randomness_Present){target="_blank"}](images/model.jpg)
 
 Often, we directly talk about the assumptions that this model makes. They can be cleverly shortened to **LINE**.  
 
@@ -286,7 +286,7 @@ Traditionally we would now calculate $\hat{\beta}_0$ and $\hat{\beta}_1$ by hand
 To keep some notation consistent with the above mathematics, we will store the response variable as `y` and the predictor variable as `x`.
 
 
-```r
+``` r
 x = cars$speed
 y = cars$dist
 ```
@@ -294,7 +294,7 @@ y = cars$dist
 We then calculate the three sums of squares defined above.
 
 
-```r
+``` r
 Sxy = sum((x - mean(x)) * (y - mean(y)))
 Sxx = sum((x - mean(x)) ^ 2)
 Syy = sum((y - mean(y)) ^ 2)
@@ -308,7 +308,7 @@ c(Sxy, Sxx, Syy)
 Then finally calculate $\hat{\beta}_0$ and $\hat{\beta}_1$.
 
 
-```r
+``` r
 beta_1_hat = Sxy / Sxx
 beta_0_hat = mean(y) - beta_1_hat * mean(x)
 c(beta_0_hat, beta_1_hat)
@@ -343,7 +343,7 @@ $$
 We can now use this line to make predictions. First, let's see the possible $x$ values in the `cars` dataset. Since some $x$ values may appear more than once, we use the `unique()` to return each unique value only once.
 
 
-```r
+``` r
 unique(cars$speed)
 ```
 
@@ -358,7 +358,7 @@ $$
 $$
 
 
-```r
+``` r
 beta_0_hat + beta_1_hat * 8
 ```
 
@@ -371,7 +371,7 @@ This tells us that the estimated mean stopping distance of a car traveling at 8
 Now let's make a prediction for the stopping distance of a car traveling at 21 miles per hour. This is considered **interpolation** as 21 is not an observed value of $x$. (But is in the data range.) We can use the special `%in%` operator to quickly verify this in `R`.
 
 
-```r
+``` r
 8 %in% unique(cars$speed)
 ```
 
@@ -379,7 +379,7 @@ Now let's make a prediction for the stopping distance of a car traveling at 21 m
 ## [1] TRUE
 ```
 
-```r
+``` r
 21 %in% unique(cars$speed)
 ```
 
@@ -388,7 +388,7 @@ Now let's make a prediction for the stopping distance of a car traveling at 21 m
 ```
 
 
-```r
+``` r
 min(cars$speed) < 21 & 21 < max(cars$speed)
 ```
 
@@ -401,7 +401,7 @@ $$
 $$
 
 
-```r
+``` r
 beta_0_hat + beta_1_hat * 21
 ```
 
@@ -412,7 +412,7 @@ beta_0_hat + beta_1_hat * 21
 Lastly, we can make a prediction for the stopping distance of a car traveling at 50 miles per hour. This is considered [**extrapolation**](https://xkcd.com/605/){target="_blank"} as 50 is not an observed value of $x$ and is outside data range. We should be less confident in predictions of this type.
 
 
-```r
+``` r
 range(cars$speed)
 ```
 
@@ -420,7 +420,7 @@ range(cars$speed)
 ## [1]  4 25
 ```
 
-```r
+``` r
 range(cars$speed)[1] < 50 & 50 < range(cars$speed)[2] 
 ```
 
@@ -433,7 +433,7 @@ $$
 $$
 
 
-```r
+``` r
 beta_0_hat + beta_1_hat * 50
 ```
 
@@ -462,7 +462,7 @@ $$
 Let's calculate the residual for the prediction we made for a car traveling 8 miles per hour. First, we need to obtain the observed value of $y$ for this $x$ value.
 
 
-```r
+``` r
 which(cars$speed == 8)
 ```
 
@@ -470,7 +470,7 @@ which(cars$speed == 8)
 ## [1] 5
 ```
 
-```r
+``` r
 cars[5, ]
 ```
 
@@ -479,7 +479,7 @@ cars[5, ]
 ## 5     8   16
 ```
 
-```r
+``` r
 cars[which(cars$speed == 8), ]
 ```
 
@@ -495,7 +495,7 @@ e = 16 - 13.88 = 2.12
 $$
 
 
-```r
+``` r
 16 - (beta_0_hat + beta_1_hat * 8)
 ```
 
@@ -544,7 +544,7 @@ Here, $s^2$ is the estimate of $\sigma^2$ when we have a single random variable
 Now, in the regression case, with $s_e^2$ each $y$ has a different mean because of the relationship with $x$. Thus, for each $y_i$, we use a different estimate of the mean, that is $\hat{y}_i$.
 
 
-```r
+``` r
 y_hat = beta_0_hat + beta_1_hat * x
 e     = y - y_hat
 n     = length(e)
@@ -559,7 +559,7 @@ s2_e
 Just as with the univariate measure of variance, this value of 236.53 doesn't have a practical interpretation in terms of stopping distance. Taking the square root, however, computes the standard deviation of the residuals, also known as *residual standard error*.
 
 
-```r
+``` r
 s_e = sqrt(s2_e)
 s_e
 ```
@@ -620,7 +620,7 @@ $$
 The quantity "Sum of Squares Error," $\text{SSE}$, represents the **unexplained variation** of the observed $y$ values. You will often see $\text{SSE}$ written as $\text{RSS}$, or "Residual Sum of Squares."
 
 
-```r
+``` r
 SST   = sum((y - mean(y)) ^ 2)
 SSReg = sum((y_hat - mean(y)) ^ 2)
 SSE   = sum((y - y_hat) ^ 2)
@@ -639,7 +639,7 @@ s_e^2 = \frac{\text{SSE}}{n - 2}.
 $$
 
 
-```r
+``` r
 SSE / (n - 2)
 ```
 
@@ -650,7 +650,7 @@ SSE / (n - 2)
 We can use `R` to verify that this matches our previous calculation of $s_e^2$.
 
 
-```r
+``` r
 s2_e == SSE / (n - 2)
 ```
 
@@ -676,7 +676,7 @@ $$
 The coefficient of determination is interpreted as the proportion of observed variation in $y$ that can be explained by the simple linear regression model.
 
 
-```r
+``` r
 R2 = SSReg / SST
 R2
 ```
@@ -713,7 +713,7 @@ The `lm()` command is used to fit **linear models** which actually account for a
 We'll continue using the `cars` data, and essentially use the `lm()` function to check the work we had previously done.
 
 
-```r
+``` r
 stop_dist_model = lm(dist ~ speed, data = cars)
 ```
 
@@ -722,7 +722,7 @@ This line of code fits our very first linear model. The syntax should look somew
 The variable `stop_dist_model` now contains a wealth of information, and we will now see how to extract and use that information. The first thing we will do is simply output whatever is stored immediately in the variable `stop_dist_model`.
 
 
-```r
+``` r
 stop_dist_model
 ```
 
@@ -739,7 +739,7 @@ stop_dist_model
 We see that it first tells us the formula we input into `R`, that is `lm(formula = dist ~ speed, data = cars)`. We also see the coefficients of the model. We can check that these are what we had calculated previously. (Minus some rounding that `R` is doing when displaying the results. They are stored with full precision.)
 
 
-```r
+``` r
 c(beta_0_hat, beta_1_hat)
 ```
 
@@ -750,7 +750,7 @@ c(beta_0_hat, beta_1_hat)
 Next, it would be nice to add the fitted line to the scatterplot. To do so we will use the `abline()` function.
 
 
-```r
+``` r
 plot(dist ~ speed, data = cars,
      xlab = "Speed (in Miles Per Hour)",
      ylab = "Stopping Distance (in Feet)",
@@ -770,7 +770,7 @@ The `abline()` function is used to add lines of the form $a + bx$ to a plot. (He
 The "thing" that is returned by the `lm()` function is actually an object of class `lm` which is a list. The exact details of this are unimportant unless you are seriously interested in the inner-workings of `R`, but know that we can determine the names of the elements of the list using the `names()` command.
 
 
-```r
+``` r
 names(stop_dist_model)
 ```
 
@@ -783,7 +783,7 @@ names(stop_dist_model)
 We can then use this information to, for example, access the residuals using the `$` operator.
 
 
-```r
+``` r
 stop_dist_model$residuals
 ```
 
@@ -809,7 +809,7 @@ stop_dist_model$residuals
 Another way to access stored information in `stop_dist_model` are the `coef()`, `resid()`, and `fitted()` functions. These return the coefficients, residuals, and fitted values, respectively.
 
 
-```r
+``` r
 coef(stop_dist_model)
 ```
 
@@ -818,7 +818,7 @@ coef(stop_dist_model)
 ##  -17.579095    3.932409
 ```
 
-```r
+``` r
 resid(stop_dist_model)
 ```
 
@@ -841,7 +841,7 @@ resid(stop_dist_model)
 ##   4.268876
 ```
 
-```r
+``` r
 fitted(stop_dist_model)
 ```
 
@@ -865,7 +865,7 @@ fitted(stop_dist_model)
 An `R` function that is useful in many situations is `summary()`. We see that when it is called on our model, it returns a good deal of information. By the end of the course, you will know what every value here is used for. For now, you should immediately notice the coefficient estimates, and you may recognize the $R^2$ value we saw earlier.
 
 
-```r
+``` r
 summary(stop_dist_model)
 ```
 
@@ -893,7 +893,7 @@ summary(stop_dist_model)
 The `summary()` command also returns a list, and we can again use `names()` to learn what about the elements of this list.
 
 
-```r
+``` r
 names(summary(stop_dist_model))
 ```
 
@@ -906,7 +906,7 @@ names(summary(stop_dist_model))
 So, for example, if we wanted to directly access the value of $R^2$, instead of copy and pasting it out of the printed statement from `summary()`, we could do so.
 
 
-```r
+``` r
 summary(stop_dist_model)$r.squared
 ```
 
@@ -917,7 +917,7 @@ summary(stop_dist_model)$r.squared
 Another value we may want to access is $s_e$, which `R` calls `sigma`.
 
 
-```r
+``` r
 summary(stop_dist_model)$sigma
 ```
 
@@ -936,7 +936,7 @@ Often it is useful to talk about $s_e$ (or RSE) instead of $s_e^2$ because of th
 Another useful function, which we will use almost as often as `lm()` is the `predict()` function.
 
 
-```r
+``` r
 predict(stop_dist_model, newdata = data.frame(speed = 8))
 ```
 
@@ -950,7 +950,7 @@ The above code reads "predict the stopping distance of a car traveling 8 miles p
 We could also predict multiple values at once.
 
 
-```r
+``` r
 predict(stop_dist_model, newdata = data.frame(speed = c(8, 21, 50)))
 ```
 
@@ -970,7 +970,7 @@ $$
 Or we could calculate the fitted value for each of the original data points. We can simply supply the original data frame, `cars`, since it contains a variables called `speed` which has the values we would like to predict at.
 
 
-```r
+``` r
 predict(stop_dist_model, newdata = cars)
 ```
 
@@ -991,14 +991,14 @@ predict(stop_dist_model, newdata = cars)
 ## 76.798715 80.731124
 ```
 
-```r
+``` r
 # predict(stop_dist_model, newdata = data.frame(speed = cars$speed))
 ```
 
 This is actually equivalent to simply calling `predict()` on `stop_dist_model` without a second argument.
 
 
-```r
+``` r
 predict(stop_dist_model)
 ```
 
@@ -1022,7 +1022,7 @@ predict(stop_dist_model)
 Note that then in this case, this is the same as using `fitted()`.
 
 
-```r
+``` r
 fitted(stop_dist_model)
 ```
 
@@ -1168,7 +1168,7 @@ $$
 We first set the true parameters of the model to be simulated.
 
 
-```r
+``` r
 num_obs = 21
 beta_0  = 5
 beta_1  = -2
@@ -1178,7 +1178,7 @@ sigma   = 3
 Next, we obtain simulated values of $\epsilon_i$ after setting a seed for reproducibility.
 
 
-```r
+``` r
 set.seed(1)
 epsilon = rnorm(n = num_obs, mean = 0, sd = sigma)
 ```
@@ -1186,7 +1186,7 @@ epsilon = rnorm(n = num_obs, mean = 0, sd = sigma)
 Now, since the $x_i$ values in SLR are considered fixed and known, we simply specify 21 values. Another common practice is to generate them from a uniform distribution, and then use them for the remainder of the analysis.
 
 
-```r
+``` r
 x_vals = seq(from = 0, to = 10, length.out = num_obs)
 # set.seed(1)
 # x_vals = runif(num_obs, 0, 10)
@@ -1195,14 +1195,14 @@ x_vals = seq(from = 0, to = 10, length.out = num_obs)
 We then generate the $y$ values according to the specified functional relationship.
 
 
-```r
+``` r
 y_vals = beta_0 + beta_1 * x_vals + epsilon
 ```
 
 The data, $(x_i, y_i)$, represent a possible sample from the true distribution. Now to check how well the method of least squares works, we use `lm()` to fit the model to our simulated data, then take a look at the estimated coefficients.
 
 
-```r
+``` r
 sim_fit = lm(y_vals ~ x_vals)
 coef(sim_fit)
 ```
@@ -1215,7 +1215,7 @@ coef(sim_fit)
 And look at that, they aren't too far from the true parameters we specified!
 
 
-```r
+``` r
 plot(y_vals ~ x_vals)
 abline(sim_fit)
 ```
@@ -1229,7 +1229,7 @@ We should say here, that we're being sort of lazy, and not the good kind of lazy
 The function below, `sim_slr()`, can be used for the same task as above, but is much more flexible. Notice that we provide `x` to the function, instead of generating `x` inside the function. In the SLR model, the $x_i$ are considered known values. That is, they are not random, so we do not assume a distribution for the $x_i$. Because of this, we will repeatedly use the same `x` values across all simulations.
 
 
-```r
+``` r
 sim_slr = function(x, beta_0 = 10, beta_1 = 5, sigma = 1) {
   n = length(x)
   epsilon = rnorm(n, mean = 0, sd = sigma)
@@ -1241,7 +1241,7 @@ sim_slr = function(x, beta_0 = 10, beta_1 = 5, sigma = 1) {
 Here, we use the function to repeat the analysis above.
 
 
-```r
+``` r
 set.seed(1)
 sim_data = sim_slr(x = x_vals, beta_0 = 5, beta_1 = -2, sigma = 3)
 ```
@@ -1249,7 +1249,7 @@ sim_data = sim_slr(x = x_vals, beta_0 = 5, beta_1 = -2, sigma = 3)
 This time, the simulated observations are stored in a data frame.
 
 
-```r
+``` r
 head(sim_data)
 ```
 
@@ -1266,7 +1266,7 @@ head(sim_data)
 Now when we fit the model with `lm()` we can use a `data` argument, a very good practice.
 
 
-```r
+``` r
 sim_fit = lm(response ~ predictor, data = sim_data)
 coef(sim_fit)
 ```
@@ -1279,7 +1279,7 @@ coef(sim_fit)
 And this time, we'll make the plot look a lot nicer.
 
 
-```r
+``` r
 plot(response ~ predictor, data = sim_data,
      xlab = "Simulated Predictor Variable",
      ylab = "Simulated Response Variable",
@@ -1309,4 +1309,4 @@ The `R` Markdown file for this chapter can be found here:
 
 - [`slr.Rmd`](slr.Rmd){target="_blank"}
 
-The file was created using `R` version `4.3.2`.
+The file was created using `R` version `4.4.1`.
diff --git a/summarizing-data.html b/summarizing-data.html
index 50915db..f26f966 100644
--- a/summarizing-data.html
+++ b/summarizing-data.html
@@ -6,7 +6,7 @@
   <meta http-equiv="X-UA-Compatible" content="IE=edge" />
   <title>Chapter 4 Summarizing Data | Applied Statistics with R</title>
   <meta name="description" content="Chapter 4 Summarizing Data | Applied Statistics with R" />
-  <meta name="generator" content="bookdown 0.37 and GitBook 2.6.7" />
+  <meta name="generator" content="bookdown 0.40 and GitBook 2.6.7" />
 
   <meta property="og:title" content="Chapter 4 Summarizing Data | Applied Statistics with R" />
   <meta property="og:type" content="book" />
@@ -55,7 +55,7 @@
 
 <style type="text/css">
 pre > code.sourceCode { white-space: pre; position: relative; }
-pre > code.sourceCode > span { display: inline-block; line-height: 1.25; }
+pre > code.sourceCode > span { line-height: 1.25; }
 pre > code.sourceCode > span:empty { height: 1.2em; }
 .sourceCode { overflow: visible; }
 code.sourceCode > span { color: inherit; text-decoration: inherit; }
@@ -469,7 +469,7 @@ <h1><span class="header-section-number">Chapter 4</span> Summarizing Data<a href
 <div id="summary-statistics" class="section level2 hasAnchor" number="4.1">
 <h2><span class="header-section-number">4.1</span> Summary Statistics<a href="summarizing-data.html#summary-statistics" class="anchor-section" aria-label="Anchor link to header"></a></h2>
 <p><code>R</code> has built-in functions for a large number of summary statistics. For numeric variables, we can summarize data with the center and spread. We’ll again look at the <code>mpg</code> dataset from the <code>ggplot2</code> package.</p>
-<div class="sourceCode" id="cb338"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb338-1"><a href="summarizing-data.html#cb338-1" aria-hidden="true" tabindex="-1"></a><span class="fu">library</span>(ggplot2)</span></code></pre></div>
+<div class="sourceCode" id="cb338"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb338-1"><a href="summarizing-data.html#cb338-1" tabindex="-1"></a><span class="fu">library</span>(ggplot2)</span></code></pre></div>
 <div id="central-tendency" class="section level3 unnumbered hasAnchor">
 <h3>Central Tendency<a href="summarizing-data.html#central-tendency" class="anchor-section" aria-label="Anchor link to header"></a></h3>
 <table>
@@ -541,11 +541,11 @@ <h3>Spread<a href="summarizing-data.html#spread" class="anchor-section" aria-lab
 <div id="categorical" class="section level3 unnumbered hasAnchor">
 <h3>Categorical<a href="summarizing-data.html#categorical" class="anchor-section" aria-label="Anchor link to header"></a></h3>
 <p>For categorical variables, counts and percentages can be used for summary.</p>
-<div class="sourceCode" id="cb339"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb339-1"><a href="summarizing-data.html#cb339-1" aria-hidden="true" tabindex="-1"></a><span class="fu">table</span>(mpg<span class="sc">$</span>drv)</span></code></pre></div>
+<div class="sourceCode" id="cb339"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb339-1"><a href="summarizing-data.html#cb339-1" tabindex="-1"></a><span class="fu">table</span>(mpg<span class="sc">$</span>drv)</span></code></pre></div>
 <pre><code>## 
 ##   4   f   r 
 ## 103 106  25</code></pre>
-<div class="sourceCode" id="cb341"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb341-1"><a href="summarizing-data.html#cb341-1" aria-hidden="true" tabindex="-1"></a><span class="fu">table</span>(mpg<span class="sc">$</span>drv) <span class="sc">/</span> <span class="fu">nrow</span>(mpg)</span></code></pre></div>
+<div class="sourceCode" id="cb341"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb341-1"><a href="summarizing-data.html#cb341-1" tabindex="-1"></a><span class="fu">table</span>(mpg<span class="sc">$</span>drv) <span class="sc">/</span> <span class="fu">nrow</span>(mpg)</span></code></pre></div>
 <pre><code>## 
 ##         4         f         r 
 ## 0.4401709 0.4529915 0.1068376</code></pre>
@@ -564,66 +564,66 @@ <h2><span class="header-section-number">4.2</span> Plotting<a href="summarizing-
 <div id="histograms" class="section level3 hasAnchor" number="4.2.1">
 <h3><span class="header-section-number">4.2.1</span> Histograms<a href="summarizing-data.html#histograms" class="anchor-section" aria-label="Anchor link to header"></a></h3>
 <p>When visualizing a single numerical variable, a <strong>histogram</strong> will be our go-to tool, which can be created in <code>R</code> using the <code>hist()</code> function.</p>
-<div class="sourceCode" id="cb343"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb343-1"><a href="summarizing-data.html#cb343-1" aria-hidden="true" tabindex="-1"></a><span class="fu">hist</span>(mpg<span class="sc">$</span>cty)</span></code></pre></div>
+<div class="sourceCode" id="cb343"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb343-1"><a href="summarizing-data.html#cb343-1" tabindex="-1"></a><span class="fu">hist</span>(mpg<span class="sc">$</span>cty)</span></code></pre></div>
 <p><img src="data-summary_files/figure-html/unnamed-chunk-3-1.png" width="672" /></p>
 <p>The histogram function has a number of parameters which can be changed to make our plot look much nicer. Use the <code>?</code> operator to read the documentation for the <code>hist()</code> to see a full list of these parameters.</p>
-<div class="sourceCode" id="cb344"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb344-1"><a href="summarizing-data.html#cb344-1" aria-hidden="true" tabindex="-1"></a><span class="fu">hist</span>(mpg<span class="sc">$</span>cty,</span>
-<span id="cb344-2"><a href="summarizing-data.html#cb344-2" aria-hidden="true" tabindex="-1"></a>     <span class="at">xlab   =</span> <span class="st">&quot;Miles Per Gallon (City)&quot;</span>,</span>
-<span id="cb344-3"><a href="summarizing-data.html#cb344-3" aria-hidden="true" tabindex="-1"></a>     <span class="at">main   =</span> <span class="st">&quot;Histogram of MPG (City)&quot;</span>,</span>
-<span id="cb344-4"><a href="summarizing-data.html#cb344-4" aria-hidden="true" tabindex="-1"></a>     <span class="at">breaks =</span> <span class="dv">12</span>,</span>
-<span id="cb344-5"><a href="summarizing-data.html#cb344-5" aria-hidden="true" tabindex="-1"></a>     <span class="at">col    =</span> <span class="st">&quot;dodgerblue&quot;</span>,</span>
-<span id="cb344-6"><a href="summarizing-data.html#cb344-6" aria-hidden="true" tabindex="-1"></a>     <span class="at">border =</span> <span class="st">&quot;darkorange&quot;</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb344"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb344-1"><a href="summarizing-data.html#cb344-1" tabindex="-1"></a><span class="fu">hist</span>(mpg<span class="sc">$</span>cty,</span>
+<span id="cb344-2"><a href="summarizing-data.html#cb344-2" tabindex="-1"></a>     <span class="at">xlab   =</span> <span class="st">&quot;Miles Per Gallon (City)&quot;</span>,</span>
+<span id="cb344-3"><a href="summarizing-data.html#cb344-3" tabindex="-1"></a>     <span class="at">main   =</span> <span class="st">&quot;Histogram of MPG (City)&quot;</span>,</span>
+<span id="cb344-4"><a href="summarizing-data.html#cb344-4" tabindex="-1"></a>     <span class="at">breaks =</span> <span class="dv">12</span>,</span>
+<span id="cb344-5"><a href="summarizing-data.html#cb344-5" tabindex="-1"></a>     <span class="at">col    =</span> <span class="st">&quot;dodgerblue&quot;</span>,</span>
+<span id="cb344-6"><a href="summarizing-data.html#cb344-6" tabindex="-1"></a>     <span class="at">border =</span> <span class="st">&quot;darkorange&quot;</span>)</span></code></pre></div>
 <p><img src="data-summary_files/figure-html/unnamed-chunk-4-1.png" width="672" /></p>
 <p>Importantly, you should always be sure to label your axes and give the plot a title. The argument <code>breaks</code> is specific to <code>hist()</code>. Entering an integer will give a suggestion to <code>R</code> for how many bars to use for the histogram. By default <code>R</code> will attempt to intelligently guess a good number of <code>breaks</code>, but as we can see here, it is sometimes useful to modify this yourself.</p>
 </div>
 <div id="barplots" class="section level3 hasAnchor" number="4.2.2">
 <h3><span class="header-section-number">4.2.2</span> Barplots<a href="summarizing-data.html#barplots" class="anchor-section" aria-label="Anchor link to header"></a></h3>
 <p>Somewhat similar to a histogram, a barplot can provide a visual summary of a categorical variable, or a numeric variable with a finite number of values, like a ranking from 1 to 10.</p>
-<div class="sourceCode" id="cb345"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb345-1"><a href="summarizing-data.html#cb345-1" aria-hidden="true" tabindex="-1"></a><span class="fu">barplot</span>(<span class="fu">table</span>(mpg<span class="sc">$</span>drv))</span></code></pre></div>
+<div class="sourceCode" id="cb345"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb345-1"><a href="summarizing-data.html#cb345-1" tabindex="-1"></a><span class="fu">barplot</span>(<span class="fu">table</span>(mpg<span class="sc">$</span>drv))</span></code></pre></div>
 <p><img src="data-summary_files/figure-html/unnamed-chunk-5-1.png" width="672" /></p>
-<div class="sourceCode" id="cb346"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb346-1"><a href="summarizing-data.html#cb346-1" aria-hidden="true" tabindex="-1"></a><span class="fu">barplot</span>(<span class="fu">table</span>(mpg<span class="sc">$</span>drv),</span>
-<span id="cb346-2"><a href="summarizing-data.html#cb346-2" aria-hidden="true" tabindex="-1"></a>     <span class="at">xlab   =</span> <span class="st">&quot;Drivetrain (f = FWD, r = RWD, 4 = 4WD)&quot;</span>,</span>
-<span id="cb346-3"><a href="summarizing-data.html#cb346-3" aria-hidden="true" tabindex="-1"></a>     <span class="at">ylab   =</span> <span class="st">&quot;Frequency&quot;</span>,</span>
-<span id="cb346-4"><a href="summarizing-data.html#cb346-4" aria-hidden="true" tabindex="-1"></a>     <span class="at">main   =</span> <span class="st">&quot;Drivetrains&quot;</span>,</span>
-<span id="cb346-5"><a href="summarizing-data.html#cb346-5" aria-hidden="true" tabindex="-1"></a>     <span class="at">col    =</span> <span class="st">&quot;dodgerblue&quot;</span>,</span>
-<span id="cb346-6"><a href="summarizing-data.html#cb346-6" aria-hidden="true" tabindex="-1"></a>     <span class="at">border =</span> <span class="st">&quot;darkorange&quot;</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb346"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb346-1"><a href="summarizing-data.html#cb346-1" tabindex="-1"></a><span class="fu">barplot</span>(<span class="fu">table</span>(mpg<span class="sc">$</span>drv),</span>
+<span id="cb346-2"><a href="summarizing-data.html#cb346-2" tabindex="-1"></a>     <span class="at">xlab   =</span> <span class="st">&quot;Drivetrain (f = FWD, r = RWD, 4 = 4WD)&quot;</span>,</span>
+<span id="cb346-3"><a href="summarizing-data.html#cb346-3" tabindex="-1"></a>     <span class="at">ylab   =</span> <span class="st">&quot;Frequency&quot;</span>,</span>
+<span id="cb346-4"><a href="summarizing-data.html#cb346-4" tabindex="-1"></a>     <span class="at">main   =</span> <span class="st">&quot;Drivetrains&quot;</span>,</span>
+<span id="cb346-5"><a href="summarizing-data.html#cb346-5" tabindex="-1"></a>     <span class="at">col    =</span> <span class="st">&quot;dodgerblue&quot;</span>,</span>
+<span id="cb346-6"><a href="summarizing-data.html#cb346-6" tabindex="-1"></a>     <span class="at">border =</span> <span class="st">&quot;darkorange&quot;</span>)</span></code></pre></div>
 <p><img src="data-summary_files/figure-html/unnamed-chunk-6-1.png" width="672" /></p>
 </div>
 <div id="boxplots" class="section level3 hasAnchor" number="4.2.3">
 <h3><span class="header-section-number">4.2.3</span> Boxplots<a href="summarizing-data.html#boxplots" class="anchor-section" aria-label="Anchor link to header"></a></h3>
 <p>To visualize the relationship between a numerical and categorical variable, we will use a <strong>boxplot</strong>. In the <code>mpg</code> dataset, the <code>drv</code> variable takes a small, finite number of values. A car can only be front wheel drive, 4 wheel drive, or rear wheel drive.</p>
-<div class="sourceCode" id="cb347"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb347-1"><a href="summarizing-data.html#cb347-1" aria-hidden="true" tabindex="-1"></a><span class="fu">unique</span>(mpg<span class="sc">$</span>drv)</span></code></pre></div>
+<div class="sourceCode" id="cb347"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb347-1"><a href="summarizing-data.html#cb347-1" tabindex="-1"></a><span class="fu">unique</span>(mpg<span class="sc">$</span>drv)</span></code></pre></div>
 <pre><code>## [1] &quot;f&quot; &quot;4&quot; &quot;r&quot;</code></pre>
 <p>First note that we can use a single boxplot as an alternative to a histogram for visualizing a single numerical variable. To do so in <code>R</code>, we use the <code>boxplot()</code> function.</p>
-<div class="sourceCode" id="cb349"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb349-1"><a href="summarizing-data.html#cb349-1" aria-hidden="true" tabindex="-1"></a><span class="fu">boxplot</span>(mpg<span class="sc">$</span>hwy)</span></code></pre></div>
+<div class="sourceCode" id="cb349"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb349-1"><a href="summarizing-data.html#cb349-1" tabindex="-1"></a><span class="fu">boxplot</span>(mpg<span class="sc">$</span>hwy)</span></code></pre></div>
 <p><img src="data-summary_files/figure-html/unnamed-chunk-8-1.png" width="672" /></p>
 <p>However, more often we will use boxplots to compare a numerical variable for different values of a categorical variable.</p>
-<div class="sourceCode" id="cb350"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb350-1"><a href="summarizing-data.html#cb350-1" aria-hidden="true" tabindex="-1"></a><span class="fu">boxplot</span>(hwy <span class="sc">~</span> drv, <span class="at">data =</span> mpg)</span></code></pre></div>
+<div class="sourceCode" id="cb350"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb350-1"><a href="summarizing-data.html#cb350-1" tabindex="-1"></a><span class="fu">boxplot</span>(hwy <span class="sc">~</span> drv, <span class="at">data =</span> mpg)</span></code></pre></div>
 <p><img src="data-summary_files/figure-html/unnamed-chunk-9-1.png" width="672" /></p>
 <p>Here we used the <code>boxplot()</code> command to create side-by-side boxplots. However, since we are now dealing with two variables, the syntax has changed. The <code>R</code> syntax <code>hwy ~ drv, data = mpg</code> reads “Plot the <code>hwy</code> variable against the <code>drv</code> variable using the dataset <code>mpg</code>.” We see the use of a <code>~</code> (which specifies a formula) and also a <code>data =</code> argument. This will be a syntax that is common to many functions we will use in this course.</p>
-<div class="sourceCode" id="cb351"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb351-1"><a href="summarizing-data.html#cb351-1" aria-hidden="true" tabindex="-1"></a><span class="fu">boxplot</span>(hwy <span class="sc">~</span> drv, <span class="at">data =</span> mpg,</span>
-<span id="cb351-2"><a href="summarizing-data.html#cb351-2" aria-hidden="true" tabindex="-1"></a>     <span class="at">xlab   =</span> <span class="st">&quot;Drivetrain (f = FWD, r = RWD, 4 = 4WD)&quot;</span>,</span>
-<span id="cb351-3"><a href="summarizing-data.html#cb351-3" aria-hidden="true" tabindex="-1"></a>     <span class="at">ylab   =</span> <span class="st">&quot;Miles Per Gallon (Highway)&quot;</span>,</span>
-<span id="cb351-4"><a href="summarizing-data.html#cb351-4" aria-hidden="true" tabindex="-1"></a>     <span class="at">main   =</span> <span class="st">&quot;MPG (Highway) vs Drivetrain&quot;</span>,</span>
-<span id="cb351-5"><a href="summarizing-data.html#cb351-5" aria-hidden="true" tabindex="-1"></a>     <span class="at">pch    =</span> <span class="dv">20</span>,</span>
-<span id="cb351-6"><a href="summarizing-data.html#cb351-6" aria-hidden="true" tabindex="-1"></a>     <span class="at">cex    =</span> <span class="dv">2</span>,</span>
-<span id="cb351-7"><a href="summarizing-data.html#cb351-7" aria-hidden="true" tabindex="-1"></a>     <span class="at">col    =</span> <span class="st">&quot;darkorange&quot;</span>,</span>
-<span id="cb351-8"><a href="summarizing-data.html#cb351-8" aria-hidden="true" tabindex="-1"></a>     <span class="at">border =</span> <span class="st">&quot;dodgerblue&quot;</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb351"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb351-1"><a href="summarizing-data.html#cb351-1" tabindex="-1"></a><span class="fu">boxplot</span>(hwy <span class="sc">~</span> drv, <span class="at">data =</span> mpg,</span>
+<span id="cb351-2"><a href="summarizing-data.html#cb351-2" tabindex="-1"></a>     <span class="at">xlab   =</span> <span class="st">&quot;Drivetrain (f = FWD, r = RWD, 4 = 4WD)&quot;</span>,</span>
+<span id="cb351-3"><a href="summarizing-data.html#cb351-3" tabindex="-1"></a>     <span class="at">ylab   =</span> <span class="st">&quot;Miles Per Gallon (Highway)&quot;</span>,</span>
+<span id="cb351-4"><a href="summarizing-data.html#cb351-4" tabindex="-1"></a>     <span class="at">main   =</span> <span class="st">&quot;MPG (Highway) vs Drivetrain&quot;</span>,</span>
+<span id="cb351-5"><a href="summarizing-data.html#cb351-5" tabindex="-1"></a>     <span class="at">pch    =</span> <span class="dv">20</span>,</span>
+<span id="cb351-6"><a href="summarizing-data.html#cb351-6" tabindex="-1"></a>     <span class="at">cex    =</span> <span class="dv">2</span>,</span>
+<span id="cb351-7"><a href="summarizing-data.html#cb351-7" tabindex="-1"></a>     <span class="at">col    =</span> <span class="st">&quot;darkorange&quot;</span>,</span>
+<span id="cb351-8"><a href="summarizing-data.html#cb351-8" tabindex="-1"></a>     <span class="at">border =</span> <span class="st">&quot;dodgerblue&quot;</span>)</span></code></pre></div>
 <p><img src="data-summary_files/figure-html/unnamed-chunk-10-1.png" width="672" /></p>
 <p>Again, <code>boxplot()</code> has a number of additional arguments which have the ability to make our plot more visually appealing.</p>
 </div>
 <div id="scatterplots" class="section level3 hasAnchor" number="4.2.4">
 <h3><span class="header-section-number">4.2.4</span> Scatterplots<a href="summarizing-data.html#scatterplots" class="anchor-section" aria-label="Anchor link to header"></a></h3>
 <p>Lastly, to visualize the relationship between two numeric variables we will use a <strong>scatterplot</strong>. This can be done with the <code>plot()</code> function and the <code>~</code> syntax we just used with a boxplot. (The function <code>plot()</code> can also be used more generally; see the documentation for details.)</p>
-<div class="sourceCode" id="cb352"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb352-1"><a href="summarizing-data.html#cb352-1" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(hwy <span class="sc">~</span> displ, <span class="at">data =</span> mpg)</span></code></pre></div>
+<div class="sourceCode" id="cb352"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb352-1"><a href="summarizing-data.html#cb352-1" tabindex="-1"></a><span class="fu">plot</span>(hwy <span class="sc">~</span> displ, <span class="at">data =</span> mpg)</span></code></pre></div>
 <p><img src="data-summary_files/figure-html/unnamed-chunk-11-1.png" width="672" /></p>
-<div class="sourceCode" id="cb353"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb353-1"><a href="summarizing-data.html#cb353-1" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(hwy <span class="sc">~</span> displ, <span class="at">data =</span> mpg,</span>
-<span id="cb353-2"><a href="summarizing-data.html#cb353-2" aria-hidden="true" tabindex="-1"></a>     <span class="at">xlab =</span> <span class="st">&quot;Engine Displacement (in Liters)&quot;</span>,</span>
-<span id="cb353-3"><a href="summarizing-data.html#cb353-3" aria-hidden="true" tabindex="-1"></a>     <span class="at">ylab =</span> <span class="st">&quot;Miles Per Gallon (Highway)&quot;</span>,</span>
-<span id="cb353-4"><a href="summarizing-data.html#cb353-4" aria-hidden="true" tabindex="-1"></a>     <span class="at">main =</span> <span class="st">&quot;MPG (Highway) vs Engine Displacement&quot;</span>,</span>
-<span id="cb353-5"><a href="summarizing-data.html#cb353-5" aria-hidden="true" tabindex="-1"></a>     <span class="at">pch  =</span> <span class="dv">20</span>,</span>
-<span id="cb353-6"><a href="summarizing-data.html#cb353-6" aria-hidden="true" tabindex="-1"></a>     <span class="at">cex  =</span> <span class="dv">2</span>,</span>
-<span id="cb353-7"><a href="summarizing-data.html#cb353-7" aria-hidden="true" tabindex="-1"></a>     <span class="at">col  =</span> <span class="st">&quot;dodgerblue&quot;</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb353"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb353-1"><a href="summarizing-data.html#cb353-1" tabindex="-1"></a><span class="fu">plot</span>(hwy <span class="sc">~</span> displ, <span class="at">data =</span> mpg,</span>
+<span id="cb353-2"><a href="summarizing-data.html#cb353-2" tabindex="-1"></a>     <span class="at">xlab =</span> <span class="st">&quot;Engine Displacement (in Liters)&quot;</span>,</span>
+<span id="cb353-3"><a href="summarizing-data.html#cb353-3" tabindex="-1"></a>     <span class="at">ylab =</span> <span class="st">&quot;Miles Per Gallon (Highway)&quot;</span>,</span>
+<span id="cb353-4"><a href="summarizing-data.html#cb353-4" tabindex="-1"></a>     <span class="at">main =</span> <span class="st">&quot;MPG (Highway) vs Engine Displacement&quot;</span>,</span>
+<span id="cb353-5"><a href="summarizing-data.html#cb353-5" tabindex="-1"></a>     <span class="at">pch  =</span> <span class="dv">20</span>,</span>
+<span id="cb353-6"><a href="summarizing-data.html#cb353-6" tabindex="-1"></a>     <span class="at">cex  =</span> <span class="dv">2</span>,</span>
+<span id="cb353-7"><a href="summarizing-data.html#cb353-7" tabindex="-1"></a>     <span class="at">col  =</span> <span class="st">&quot;dodgerblue&quot;</span>)</span></code></pre></div>
 <p><img src="data-summary_files/figure-html/unnamed-chunk-12-1.png" width="672" /></p>
 
 </div>
diff --git a/transformations.html b/transformations.html
index 50df3eb..29c22a4 100644
--- a/transformations.html
+++ b/transformations.html
@@ -6,7 +6,7 @@
   <meta http-equiv="X-UA-Compatible" content="IE=edge" />
   <title>Chapter 14 Transformations | Applied Statistics with R</title>
   <meta name="description" content="Chapter 14 Transformations | Applied Statistics with R" />
-  <meta name="generator" content="bookdown 0.37 and GitBook 2.6.7" />
+  <meta name="generator" content="bookdown 0.40 and GitBook 2.6.7" />
 
   <meta property="og:title" content="Chapter 14 Transformations | Applied Statistics with R" />
   <meta property="og:type" content="book" />
@@ -55,7 +55,7 @@
 
 <style type="text/css">
 pre > code.sourceCode { white-space: pre; position: relative; }
-pre > code.sourceCode > span { display: inline-block; line-height: 1.25; }
+pre > code.sourceCode > span { line-height: 1.25; }
 pre > code.sourceCode > span:empty { height: 1.2em; }
 .sourceCode { overflow: visible; }
 code.sourceCode > span { color: inherit; text-decoration: inherit; }
@@ -481,13 +481,13 @@ <h1><span class="header-section-number">Chapter 14</span> Transformations<a href
 <div id="response-transformation" class="section level2 hasAnchor" number="14.1">
 <h2><span class="header-section-number">14.1</span> Response Transformation<a href="transformations.html#response-transformation" class="anchor-section" aria-label="Anchor link to header"></a></h2>
 <p>Let’s look at some (<em>fictional</em>) salary data from the (<em>fictional</em>) company <em>Initech</em>. We will try to model <code>salary</code> as a function of <code>years</code> of experience. The data can be found in <a href="data/initech.csv"><code>initech.csv</code></a>.</p>
-<div class="sourceCode" id="cb1025"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1025-1"><a href="transformations.html#cb1025-1" aria-hidden="true" tabindex="-1"></a>initech <span class="ot">=</span> <span class="fu">read.csv</span>(<span class="st">&quot;data/initech.csv&quot;</span>)</span></code></pre></div>
-<div class="sourceCode" id="cb1026"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1026-1"><a href="transformations.html#cb1026-1" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(salary <span class="sc">~</span> years, <span class="at">data =</span> initech, <span class="at">col =</span> <span class="st">&quot;grey&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">cex =</span> <span class="fl">1.5</span>,</span>
-<span id="cb1026-2"><a href="transformations.html#cb1026-2" aria-hidden="true" tabindex="-1"></a>     <span class="at">main =</span> <span class="st">&quot;Salaries at Initech, By Seniority&quot;</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1025"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1025-1"><a href="transformations.html#cb1025-1" tabindex="-1"></a>initech <span class="ot">=</span> <span class="fu">read.csv</span>(<span class="st">&quot;data/initech.csv&quot;</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1026"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1026-1"><a href="transformations.html#cb1026-1" tabindex="-1"></a><span class="fu">plot</span>(salary <span class="sc">~</span> years, <span class="at">data =</span> initech, <span class="at">col =</span> <span class="st">&quot;grey&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">cex =</span> <span class="fl">1.5</span>,</span>
+<span id="cb1026-2"><a href="transformations.html#cb1026-2" tabindex="-1"></a>     <span class="at">main =</span> <span class="st">&quot;Salaries at Initech, By Seniority&quot;</span>)</span></code></pre></div>
 <p><img src="transformations_files/figure-html/unnamed-chunk-4-1.png" width="672" style="display: block; margin: auto;" /></p>
 <p>We first fit a simple linear model.</p>
-<div class="sourceCode" id="cb1027"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1027-1"><a href="transformations.html#cb1027-1" aria-hidden="true" tabindex="-1"></a>initech_fit <span class="ot">=</span> <span class="fu">lm</span>(salary <span class="sc">~</span> years, <span class="at">data =</span> initech)</span>
-<span id="cb1027-2"><a href="transformations.html#cb1027-2" aria-hidden="true" tabindex="-1"></a><span class="fu">summary</span>(initech_fit)</span></code></pre></div>
+<div class="sourceCode" id="cb1027"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1027-1"><a href="transformations.html#cb1027-1" tabindex="-1"></a>initech_fit <span class="ot">=</span> <span class="fu">lm</span>(salary <span class="sc">~</span> years, <span class="at">data =</span> initech)</span>
+<span id="cb1027-2"><a href="transformations.html#cb1027-2" tabindex="-1"></a><span class="fu">summary</span>(initech_fit)</span></code></pre></div>
 <pre><code>## 
 ## Call:
 ## lm(formula = salary ~ years, data = initech)
@@ -507,19 +507,19 @@ <h2><span class="header-section-number">14.1</span> Response Transformation<a hr
 ## Multiple R-squared:  0.8341, Adjusted R-squared:  0.8324 
 ## F-statistic: 492.8 on 1 and 98 DF,  p-value: &lt; 2.2e-16</code></pre>
 <p>This model appears significant, but does it meet the model assumptions?</p>
-<div class="sourceCode" id="cb1029"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1029-1"><a href="transformations.html#cb1029-1" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(salary <span class="sc">~</span> years, <span class="at">data =</span> initech, <span class="at">col =</span> <span class="st">&quot;grey&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">cex =</span> <span class="fl">1.5</span>,</span>
-<span id="cb1029-2"><a href="transformations.html#cb1029-2" aria-hidden="true" tabindex="-1"></a>     <span class="at">main =</span> <span class="st">&quot;Salaries at Initech, By Seniority&quot;</span>)</span>
-<span id="cb1029-3"><a href="transformations.html#cb1029-3" aria-hidden="true" tabindex="-1"></a><span class="fu">abline</span>(initech_fit, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1029"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1029-1"><a href="transformations.html#cb1029-1" tabindex="-1"></a><span class="fu">plot</span>(salary <span class="sc">~</span> years, <span class="at">data =</span> initech, <span class="at">col =</span> <span class="st">&quot;grey&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">cex =</span> <span class="fl">1.5</span>,</span>
+<span id="cb1029-2"><a href="transformations.html#cb1029-2" tabindex="-1"></a>     <span class="at">main =</span> <span class="st">&quot;Salaries at Initech, By Seniority&quot;</span>)</span>
+<span id="cb1029-3"><a href="transformations.html#cb1029-3" tabindex="-1"></a><span class="fu">abline</span>(initech_fit, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
 <p><img src="transformations_files/figure-html/unnamed-chunk-6-1.png" width="672" style="display: block; margin: auto;" /></p>
 <p>Adding the fitted line to the plot, we see that the linear relationship appears correct.</p>
-<div class="sourceCode" id="cb1030"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1030-1"><a href="transformations.html#cb1030-1" aria-hidden="true" tabindex="-1"></a><span class="fu">par</span>(<span class="at">mfrow =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>))</span>
-<span id="cb1030-2"><a href="transformations.html#cb1030-2" aria-hidden="true" tabindex="-1"></a></span>
-<span id="cb1030-3"><a href="transformations.html#cb1030-3" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(<span class="fu">fitted</span>(initech_fit), <span class="fu">resid</span>(initech_fit), <span class="at">col =</span> <span class="st">&quot;grey&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>,</span>
-<span id="cb1030-4"><a href="transformations.html#cb1030-4" aria-hidden="true" tabindex="-1"></a>     <span class="at">xlab =</span> <span class="st">&quot;Fitted&quot;</span>, <span class="at">ylab =</span> <span class="st">&quot;Residuals&quot;</span>, <span class="at">main =</span> <span class="st">&quot;Fitted versus Residuals&quot;</span>)</span>
-<span id="cb1030-5"><a href="transformations.html#cb1030-5" aria-hidden="true" tabindex="-1"></a><span class="fu">abline</span>(<span class="at">h =</span> <span class="dv">0</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span>
-<span id="cb1030-6"><a href="transformations.html#cb1030-6" aria-hidden="true" tabindex="-1"></a></span>
-<span id="cb1030-7"><a href="transformations.html#cb1030-7" aria-hidden="true" tabindex="-1"></a><span class="fu">qqnorm</span>(<span class="fu">resid</span>(initech_fit), <span class="at">main =</span> <span class="st">&quot;Normal Q-Q Plot&quot;</span>, <span class="at">col =</span> <span class="st">&quot;darkgrey&quot;</span>)</span>
-<span id="cb1030-8"><a href="transformations.html#cb1030-8" aria-hidden="true" tabindex="-1"></a><span class="fu">qqline</span>(<span class="fu">resid</span>(initech_fit), <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1030"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1030-1"><a href="transformations.html#cb1030-1" tabindex="-1"></a><span class="fu">par</span>(<span class="at">mfrow =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>))</span>
+<span id="cb1030-2"><a href="transformations.html#cb1030-2" tabindex="-1"></a></span>
+<span id="cb1030-3"><a href="transformations.html#cb1030-3" tabindex="-1"></a><span class="fu">plot</span>(<span class="fu">fitted</span>(initech_fit), <span class="fu">resid</span>(initech_fit), <span class="at">col =</span> <span class="st">&quot;grey&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>,</span>
+<span id="cb1030-4"><a href="transformations.html#cb1030-4" tabindex="-1"></a>     <span class="at">xlab =</span> <span class="st">&quot;Fitted&quot;</span>, <span class="at">ylab =</span> <span class="st">&quot;Residuals&quot;</span>, <span class="at">main =</span> <span class="st">&quot;Fitted versus Residuals&quot;</span>)</span>
+<span id="cb1030-5"><a href="transformations.html#cb1030-5" tabindex="-1"></a><span class="fu">abline</span>(<span class="at">h =</span> <span class="dv">0</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span>
+<span id="cb1030-6"><a href="transformations.html#cb1030-6" tabindex="-1"></a></span>
+<span id="cb1030-7"><a href="transformations.html#cb1030-7" tabindex="-1"></a><span class="fu">qqnorm</span>(<span class="fu">resid</span>(initech_fit), <span class="at">main =</span> <span class="st">&quot;Normal Q-Q Plot&quot;</span>, <span class="at">col =</span> <span class="st">&quot;darkgrey&quot;</span>)</span>
+<span id="cb1030-8"><a href="transformations.html#cb1030-8" tabindex="-1"></a><span class="fu">qqline</span>(<span class="fu">resid</span>(initech_fit), <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
 <p><img src="transformations_files/figure-html/unnamed-chunk-7-1.png" width="960" style="display: block; margin: auto;" /></p>
 <p>However, from the fitted versus residuals plot it appears there is non-constant variance. Specifically, the variance increases as the fitted value increases.</p>
 <div id="variance-stabilizing-transformations" class="section level3 hasAnchor" number="14.1.1">
@@ -555,41 +555,41 @@ <h3><span class="header-section-number">14.1.1</span> Variance Stabilizing Trans
 \]</span></p>
 <p>which has the errors entering the model in a multiplicative fashion.</p>
 <p>Fitting this model in <code>R</code> requires only a minor modification to our formula specification.</p>
-<div class="sourceCode" id="cb1031"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1031-1"><a href="transformations.html#cb1031-1" aria-hidden="true" tabindex="-1"></a>initech_fit_log <span class="ot">=</span> <span class="fu">lm</span>(<span class="fu">log</span>(salary) <span class="sc">~</span> years, <span class="at">data =</span> initech)</span></code></pre></div>
+<div class="sourceCode" id="cb1031"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1031-1"><a href="transformations.html#cb1031-1" tabindex="-1"></a>initech_fit_log <span class="ot">=</span> <span class="fu">lm</span>(<span class="fu">log</span>(salary) <span class="sc">~</span> years, <span class="at">data =</span> initech)</span></code></pre></div>
 <p>Note that while <code>log(y)</code> is considered the new response variable, we do not actually create a new variable in <code>R</code>, but simply transform the variable inside the model formula.</p>
-<div class="sourceCode" id="cb1032"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1032-1"><a href="transformations.html#cb1032-1" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(<span class="fu">log</span>(salary) <span class="sc">~</span> years, <span class="at">data =</span> initech, <span class="at">col =</span> <span class="st">&quot;grey&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">cex =</span> <span class="fl">1.5</span>,</span>
-<span id="cb1032-2"><a href="transformations.html#cb1032-2" aria-hidden="true" tabindex="-1"></a>     <span class="at">main =</span> <span class="st">&quot;Salaries at Initech, By Seniority&quot;</span>)</span>
-<span id="cb1032-3"><a href="transformations.html#cb1032-3" aria-hidden="true" tabindex="-1"></a><span class="fu">abline</span>(initech_fit_log, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1032"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1032-1"><a href="transformations.html#cb1032-1" tabindex="-1"></a><span class="fu">plot</span>(<span class="fu">log</span>(salary) <span class="sc">~</span> years, <span class="at">data =</span> initech, <span class="at">col =</span> <span class="st">&quot;grey&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">cex =</span> <span class="fl">1.5</span>,</span>
+<span id="cb1032-2"><a href="transformations.html#cb1032-2" tabindex="-1"></a>     <span class="at">main =</span> <span class="st">&quot;Salaries at Initech, By Seniority&quot;</span>)</span>
+<span id="cb1032-3"><a href="transformations.html#cb1032-3" tabindex="-1"></a><span class="fu">abline</span>(initech_fit_log, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
 <p><img src="transformations_files/figure-html/unnamed-chunk-9-1.png" width="672" style="display: block; margin: auto;" /></p>
 <p>Plotting the data on the transformed log scale and adding the fitted line, the relationship again appears linear, and we can already see that the variation about the fitted line looks constant.</p>
-<div class="sourceCode" id="cb1033"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1033-1"><a href="transformations.html#cb1033-1" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(salary <span class="sc">~</span> years, <span class="at">data =</span> initech, <span class="at">col =</span> <span class="st">&quot;grey&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">cex =</span> <span class="fl">1.5</span>,</span>
-<span id="cb1033-2"><a href="transformations.html#cb1033-2" aria-hidden="true" tabindex="-1"></a>     <span class="at">main =</span> <span class="st">&quot;Salaries at Initech, By Seniority&quot;</span>)</span>
-<span id="cb1033-3"><a href="transformations.html#cb1033-3" aria-hidden="true" tabindex="-1"></a><span class="fu">curve</span>(<span class="fu">exp</span>(initech_fit_log<span class="sc">$</span>coef[<span class="dv">1</span>] <span class="sc">+</span> initech_fit_log<span class="sc">$</span>coef[<span class="dv">2</span>] <span class="sc">*</span> x),</span>
-<span id="cb1033-4"><a href="transformations.html#cb1033-4" aria-hidden="true" tabindex="-1"></a>      <span class="at">from =</span> <span class="dv">0</span>, <span class="at">to =</span> <span class="dv">30</span>, <span class="at">add =</span> <span class="cn">TRUE</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1033"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1033-1"><a href="transformations.html#cb1033-1" tabindex="-1"></a><span class="fu">plot</span>(salary <span class="sc">~</span> years, <span class="at">data =</span> initech, <span class="at">col =</span> <span class="st">&quot;grey&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">cex =</span> <span class="fl">1.5</span>,</span>
+<span id="cb1033-2"><a href="transformations.html#cb1033-2" tabindex="-1"></a>     <span class="at">main =</span> <span class="st">&quot;Salaries at Initech, By Seniority&quot;</span>)</span>
+<span id="cb1033-3"><a href="transformations.html#cb1033-3" tabindex="-1"></a><span class="fu">curve</span>(<span class="fu">exp</span>(initech_fit_log<span class="sc">$</span>coef[<span class="dv">1</span>] <span class="sc">+</span> initech_fit_log<span class="sc">$</span>coef[<span class="dv">2</span>] <span class="sc">*</span> x),</span>
+<span id="cb1033-4"><a href="transformations.html#cb1033-4" tabindex="-1"></a>      <span class="at">from =</span> <span class="dv">0</span>, <span class="at">to =</span> <span class="dv">30</span>, <span class="at">add =</span> <span class="cn">TRUE</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
 <p><img src="transformations_files/figure-html/unnamed-chunk-10-1.png" width="672" style="display: block; margin: auto;" /></p>
 <p>By plotting the data on the original scale, and adding the fitted regression, we see an exponential relationship. However, this is still a <em>linear</em> model, since the new transformed response, <span class="math inline">\(\log(y)\)</span>, is still a <em>linear</em> combination of the predictors.</p>
-<div class="sourceCode" id="cb1034"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1034-1"><a href="transformations.html#cb1034-1" aria-hidden="true" tabindex="-1"></a><span class="fu">par</span>(<span class="at">mfrow =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>))</span>
-<span id="cb1034-2"><a href="transformations.html#cb1034-2" aria-hidden="true" tabindex="-1"></a></span>
-<span id="cb1034-3"><a href="transformations.html#cb1034-3" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(<span class="fu">fitted</span>(initech_fit_log), <span class="fu">resid</span>(initech_fit_log), <span class="at">col =</span> <span class="st">&quot;grey&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>,</span>
-<span id="cb1034-4"><a href="transformations.html#cb1034-4" aria-hidden="true" tabindex="-1"></a>     <span class="at">xlab =</span> <span class="st">&quot;Fitted&quot;</span>, <span class="at">ylab =</span> <span class="st">&quot;Residuals&quot;</span>, <span class="at">main =</span> <span class="st">&quot;Fitted versus Residuals&quot;</span>)</span>
-<span id="cb1034-5"><a href="transformations.html#cb1034-5" aria-hidden="true" tabindex="-1"></a><span class="fu">abline</span>(<span class="at">h =</span> <span class="dv">0</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span>
-<span id="cb1034-6"><a href="transformations.html#cb1034-6" aria-hidden="true" tabindex="-1"></a></span>
-<span id="cb1034-7"><a href="transformations.html#cb1034-7" aria-hidden="true" tabindex="-1"></a><span class="fu">qqnorm</span>(<span class="fu">resid</span>(initech_fit_log), <span class="at">main =</span> <span class="st">&quot;Normal Q-Q Plot&quot;</span>, <span class="at">col =</span> <span class="st">&quot;darkgrey&quot;</span>)</span>
-<span id="cb1034-8"><a href="transformations.html#cb1034-8" aria-hidden="true" tabindex="-1"></a><span class="fu">qqline</span>(<span class="fu">resid</span>(initech_fit_log), <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1034"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1034-1"><a href="transformations.html#cb1034-1" tabindex="-1"></a><span class="fu">par</span>(<span class="at">mfrow =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>))</span>
+<span id="cb1034-2"><a href="transformations.html#cb1034-2" tabindex="-1"></a></span>
+<span id="cb1034-3"><a href="transformations.html#cb1034-3" tabindex="-1"></a><span class="fu">plot</span>(<span class="fu">fitted</span>(initech_fit_log), <span class="fu">resid</span>(initech_fit_log), <span class="at">col =</span> <span class="st">&quot;grey&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>,</span>
+<span id="cb1034-4"><a href="transformations.html#cb1034-4" tabindex="-1"></a>     <span class="at">xlab =</span> <span class="st">&quot;Fitted&quot;</span>, <span class="at">ylab =</span> <span class="st">&quot;Residuals&quot;</span>, <span class="at">main =</span> <span class="st">&quot;Fitted versus Residuals&quot;</span>)</span>
+<span id="cb1034-5"><a href="transformations.html#cb1034-5" tabindex="-1"></a><span class="fu">abline</span>(<span class="at">h =</span> <span class="dv">0</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span>
+<span id="cb1034-6"><a href="transformations.html#cb1034-6" tabindex="-1"></a></span>
+<span id="cb1034-7"><a href="transformations.html#cb1034-7" tabindex="-1"></a><span class="fu">qqnorm</span>(<span class="fu">resid</span>(initech_fit_log), <span class="at">main =</span> <span class="st">&quot;Normal Q-Q Plot&quot;</span>, <span class="at">col =</span> <span class="st">&quot;darkgrey&quot;</span>)</span>
+<span id="cb1034-8"><a href="transformations.html#cb1034-8" tabindex="-1"></a><span class="fu">qqline</span>(<span class="fu">resid</span>(initech_fit_log), <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
 <p><img src="transformations_files/figure-html/unnamed-chunk-11-1.png" width="960" style="display: block; margin: auto;" /></p>
 <p>The fitted versus residuals plot looks much better. It appears the constant variance assumption is no longer violated.</p>
 <p>Comparing the RMSE using the original and transformed response, we also see that the log transformed model simply fits better, with a smaller average squared error.</p>
-<div class="sourceCode" id="cb1035"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1035-1"><a href="transformations.html#cb1035-1" aria-hidden="true" tabindex="-1"></a><span class="fu">sqrt</span>(<span class="fu">mean</span>(<span class="fu">resid</span>(initech_fit) <span class="sc">^</span> <span class="dv">2</span>))</span></code></pre></div>
+<div class="sourceCode" id="cb1035"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1035-1"><a href="transformations.html#cb1035-1" tabindex="-1"></a><span class="fu">sqrt</span>(<span class="fu">mean</span>(<span class="fu">resid</span>(initech_fit) <span class="sc">^</span> <span class="dv">2</span>))</span></code></pre></div>
 <pre><code>## [1] 27080.16</code></pre>
-<div class="sourceCode" id="cb1037"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1037-1"><a href="transformations.html#cb1037-1" aria-hidden="true" tabindex="-1"></a><span class="fu">sqrt</span>(<span class="fu">mean</span>(<span class="fu">resid</span>(initech_fit_log) <span class="sc">^</span> <span class="dv">2</span>))</span></code></pre></div>
+<div class="sourceCode" id="cb1037"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1037-1"><a href="transformations.html#cb1037-1" tabindex="-1"></a><span class="fu">sqrt</span>(<span class="fu">mean</span>(<span class="fu">resid</span>(initech_fit_log) <span class="sc">^</span> <span class="dv">2</span>))</span></code></pre></div>
 <pre><code>## [1] 0.1934907</code></pre>
 <p>But wait, that isn’t fair, this difference is simply due to the different scales being used.</p>
-<div class="sourceCode" id="cb1039"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1039-1"><a href="transformations.html#cb1039-1" aria-hidden="true" tabindex="-1"></a><span class="fu">sqrt</span>(<span class="fu">mean</span>((initech<span class="sc">$</span>salary <span class="sc">-</span> <span class="fu">fitted</span>(initech_fit)) <span class="sc">^</span> <span class="dv">2</span>))</span></code></pre></div>
+<div class="sourceCode" id="cb1039"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1039-1"><a href="transformations.html#cb1039-1" tabindex="-1"></a><span class="fu">sqrt</span>(<span class="fu">mean</span>((initech<span class="sc">$</span>salary <span class="sc">-</span> <span class="fu">fitted</span>(initech_fit)) <span class="sc">^</span> <span class="dv">2</span>))</span></code></pre></div>
 <pre><code>## [1] 27080.16</code></pre>
-<div class="sourceCode" id="cb1041"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1041-1"><a href="transformations.html#cb1041-1" aria-hidden="true" tabindex="-1"></a><span class="fu">sqrt</span>(<span class="fu">mean</span>((initech<span class="sc">$</span>salary <span class="sc">-</span> <span class="fu">exp</span>(<span class="fu">fitted</span>(initech_fit_log))) <span class="sc">^</span> <span class="dv">2</span>))</span></code></pre></div>
+<div class="sourceCode" id="cb1041"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1041-1"><a href="transformations.html#cb1041-1" tabindex="-1"></a><span class="fu">sqrt</span>(<span class="fu">mean</span>((initech<span class="sc">$</span>salary <span class="sc">-</span> <span class="fu">exp</span>(<span class="fu">fitted</span>(initech_fit_log))) <span class="sc">^</span> <span class="dv">2</span>))</span></code></pre></div>
 <pre><code>## [1] 24280.36</code></pre>
 <p>Transforming the fitted values of the log model back to the data scale, we do indeed see that it fits better!</p>
-<div class="sourceCode" id="cb1043"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1043-1"><a href="transformations.html#cb1043-1" aria-hidden="true" tabindex="-1"></a><span class="fu">summary</span>(initech_fit_log)</span></code></pre></div>
+<div class="sourceCode" id="cb1043"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1043-1"><a href="transformations.html#cb1043-1" tabindex="-1"></a><span class="fu">summary</span>(initech_fit_log)</span></code></pre></div>
 <pre><code>## 
 ## Call:
 ## lm(formula = log(salary) ~ years, data = initech)
@@ -639,62 +639,62 @@ <h3><span class="header-section-number">14.1.2</span> Box-Cox Transformations<a
     \left\{ \lambda :  L(\lambda) &gt; L(\hat{\lambda}) - \frac{1}{2}\chi_{1,\alpha}^2  \right\}   
 \]</span></p>
 <p>which <code>R</code> will plot for us to help quickly select an appropriate <span class="math inline">\(\lambda\)</span> value. We often choose a “nice” value from within the confidence interval, instead of the value of <span class="math inline">\(\lambda\)</span> that truly maximizes the likelihood.</p>
-<div class="sourceCode" id="cb1045"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1045-1"><a href="transformations.html#cb1045-1" aria-hidden="true" tabindex="-1"></a><span class="fu">library</span>(MASS)</span>
-<span id="cb1045-2"><a href="transformations.html#cb1045-2" aria-hidden="true" tabindex="-1"></a><span class="fu">library</span>(faraway)</span></code></pre></div>
+<div class="sourceCode" id="cb1045"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1045-1"><a href="transformations.html#cb1045-1" tabindex="-1"></a><span class="fu">library</span>(MASS)</span>
+<span id="cb1045-2"><a href="transformations.html#cb1045-2" tabindex="-1"></a><span class="fu">library</span>(faraway)</span></code></pre></div>
 <p>Here we need the <code>MASS</code> package for the <code>boxcox()</code> function, and we will consider a couple of datasets from the <code>faraway</code> package.</p>
 <p>First we will use the <code>savings</code> dataset as an example of using the Box-Cox method to justify the use of no transformation. We fit an additive multiple regression model with <code>sr</code> as the response and each of the other variables as predictors.</p>
-<div class="sourceCode" id="cb1046"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1046-1"><a href="transformations.html#cb1046-1" aria-hidden="true" tabindex="-1"></a>savings_model <span class="ot">=</span> <span class="fu">lm</span>(sr <span class="sc">~</span> ., <span class="at">data =</span> savings)</span></code></pre></div>
+<div class="sourceCode" id="cb1046"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1046-1"><a href="transformations.html#cb1046-1" tabindex="-1"></a>savings_model <span class="ot">=</span> <span class="fu">lm</span>(sr <span class="sc">~</span> ., <span class="at">data =</span> savings)</span></code></pre></div>
 <p>We then use the <code>boxcox()</code> function to find the best transformation of the form considered by the Box-Cox method.</p>
-<div class="sourceCode" id="cb1047"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1047-1"><a href="transformations.html#cb1047-1" aria-hidden="true" tabindex="-1"></a><span class="fu">boxcox</span>(savings_model, <span class="at">plotit =</span> <span class="cn">TRUE</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1047"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1047-1"><a href="transformations.html#cb1047-1" tabindex="-1"></a><span class="fu">boxcox</span>(savings_model, <span class="at">plotit =</span> <span class="cn">TRUE</span>)</span></code></pre></div>
 <p><img src="transformations_files/figure-html/unnamed-chunk-17-1.png" width="672" style="display: block; margin: auto;" /></p>
 <p><code>R</code> automatically plots the log-Likelihood as a function of possible <span class="math inline">\(\lambda\)</span> values. It indicates both the value that maximizes the log-likelihood, as well as a confidence interval for the <span class="math inline">\(\lambda\)</span> value that maximizes the log-likelihood.</p>
-<div class="sourceCode" id="cb1048"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1048-1"><a href="transformations.html#cb1048-1" aria-hidden="true" tabindex="-1"></a><span class="fu">boxcox</span>(savings_model, <span class="at">plotit =</span> <span class="cn">TRUE</span>, <span class="at">lambda =</span> <span class="fu">seq</span>(<span class="fl">0.5</span>, <span class="fl">1.5</span>, <span class="at">by =</span> <span class="fl">0.1</span>))</span></code></pre></div>
+<div class="sourceCode" id="cb1048"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1048-1"><a href="transformations.html#cb1048-1" tabindex="-1"></a><span class="fu">boxcox</span>(savings_model, <span class="at">plotit =</span> <span class="cn">TRUE</span>, <span class="at">lambda =</span> <span class="fu">seq</span>(<span class="fl">0.5</span>, <span class="fl">1.5</span>, <span class="at">by =</span> <span class="fl">0.1</span>))</span></code></pre></div>
 <p><img src="transformations_files/figure-html/unnamed-chunk-18-1.png" width="672" style="display: block; margin: auto;" /></p>
 <p>Note that we can specify a range of <span class="math inline">\(\lambda\)</span> values to consider and thus be plotted. We often specify a range that is more visually interesting. Here we see that <span class="math inline">\(\lambda = 1\)</span> is both in the confidence interval, and is extremely close to the maximum. This suggests a transformation of the form</p>
 <p><span class="math display">\[
 \frac{y^\lambda - 1}{\lambda} = \frac{y^1 - 1}{1} = y - 1.
 \]</span></p>
 <p>This is essentially not a transformation. It would not change the variance or make the model fit better. By subtracting 1 from every value, we would only change the intercept of the model, and the resulting errors would be the same.</p>
-<div class="sourceCode" id="cb1049"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1049-1"><a href="transformations.html#cb1049-1" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(<span class="fu">fitted</span>(savings_model), <span class="fu">resid</span>(savings_model), <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>,</span>
-<span id="cb1049-2"><a href="transformations.html#cb1049-2" aria-hidden="true" tabindex="-1"></a>     <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">cex =</span> <span class="fl">1.5</span>, <span class="at">xlab =</span> <span class="st">&quot;Fitted&quot;</span>, <span class="at">ylab =</span> <span class="st">&quot;Residuals&quot;</span>)</span>
-<span id="cb1049-3"><a href="transformations.html#cb1049-3" aria-hidden="true" tabindex="-1"></a><span class="fu">abline</span>(<span class="at">h =</span> <span class="dv">0</span>, <span class="at">lty =</span> <span class="dv">2</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1049"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1049-1"><a href="transformations.html#cb1049-1" tabindex="-1"></a><span class="fu">plot</span>(<span class="fu">fitted</span>(savings_model), <span class="fu">resid</span>(savings_model), <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>,</span>
+<span id="cb1049-2"><a href="transformations.html#cb1049-2" tabindex="-1"></a>     <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">cex =</span> <span class="fl">1.5</span>, <span class="at">xlab =</span> <span class="st">&quot;Fitted&quot;</span>, <span class="at">ylab =</span> <span class="st">&quot;Residuals&quot;</span>)</span>
+<span id="cb1049-3"><a href="transformations.html#cb1049-3" tabindex="-1"></a><span class="fu">abline</span>(<span class="at">h =</span> <span class="dv">0</span>, <span class="at">lty =</span> <span class="dv">2</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
 <p><img src="transformations_files/figure-html/unnamed-chunk-19-1.png" width="672" style="display: block; margin: auto;" /></p>
 <p>Looking at a fitted versus residuals plot verifies that there likely are not any issues with the assumptions of this model, which Breusch-Pagan and Shapiro-Wilk tests verify.</p>
-<div class="sourceCode" id="cb1050"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1050-1"><a href="transformations.html#cb1050-1" aria-hidden="true" tabindex="-1"></a><span class="fu">library</span>(lmtest)</span>
-<span id="cb1050-2"><a href="transformations.html#cb1050-2" aria-hidden="true" tabindex="-1"></a><span class="fu">bptest</span>(savings_model)</span></code></pre></div>
+<div class="sourceCode" id="cb1050"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1050-1"><a href="transformations.html#cb1050-1" tabindex="-1"></a><span class="fu">library</span>(lmtest)</span>
+<span id="cb1050-2"><a href="transformations.html#cb1050-2" tabindex="-1"></a><span class="fu">bptest</span>(savings_model)</span></code></pre></div>
 <pre><code>## 
 ##  studentized Breusch-Pagan test
 ## 
 ## data:  savings_model
 ## BP = 4.9852, df = 4, p-value = 0.2888</code></pre>
-<div class="sourceCode" id="cb1052"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1052-1"><a href="transformations.html#cb1052-1" aria-hidden="true" tabindex="-1"></a><span class="fu">shapiro.test</span>(<span class="fu">resid</span>(savings_model))</span></code></pre></div>
+<div class="sourceCode" id="cb1052"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1052-1"><a href="transformations.html#cb1052-1" tabindex="-1"></a><span class="fu">shapiro.test</span>(<span class="fu">resid</span>(savings_model))</span></code></pre></div>
 <pre><code>## 
 ##  Shapiro-Wilk normality test
 ## 
 ## data:  resid(savings_model)
 ## W = 0.98698, p-value = 0.8524</code></pre>
 <p>Now we will use the <code>gala</code> dataset as an example of using the Box-Cox method to justify a transformation other than <span class="math inline">\(\log\)</span>. We fit an additive multiple regression model with <code>Species</code> as the response and most of the other variables as predictors.</p>
-<div class="sourceCode" id="cb1054"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1054-1"><a href="transformations.html#cb1054-1" aria-hidden="true" tabindex="-1"></a>gala_model <span class="ot">=</span> <span class="fu">lm</span>(Species <span class="sc">~</span> Area <span class="sc">+</span> Elevation <span class="sc">+</span> Nearest <span class="sc">+</span> Scruz <span class="sc">+</span> Adjacent, <span class="at">data =</span> gala)</span></code></pre></div>
-<div class="sourceCode" id="cb1055"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1055-1"><a href="transformations.html#cb1055-1" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(<span class="fu">fitted</span>(gala_model), <span class="fu">resid</span>(gala_model), <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>,</span>
-<span id="cb1055-2"><a href="transformations.html#cb1055-2" aria-hidden="true" tabindex="-1"></a>     <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">cex =</span> <span class="fl">1.5</span>, <span class="at">xlab =</span> <span class="st">&quot;Fitted&quot;</span>, <span class="at">ylab =</span> <span class="st">&quot;Residuals&quot;</span>)</span>
-<span id="cb1055-3"><a href="transformations.html#cb1055-3" aria-hidden="true" tabindex="-1"></a><span class="fu">abline</span>(<span class="at">h =</span> <span class="dv">0</span>, <span class="at">lty =</span> <span class="dv">2</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1054"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1054-1"><a href="transformations.html#cb1054-1" tabindex="-1"></a>gala_model <span class="ot">=</span> <span class="fu">lm</span>(Species <span class="sc">~</span> Area <span class="sc">+</span> Elevation <span class="sc">+</span> Nearest <span class="sc">+</span> Scruz <span class="sc">+</span> Adjacent, <span class="at">data =</span> gala)</span></code></pre></div>
+<div class="sourceCode" id="cb1055"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1055-1"><a href="transformations.html#cb1055-1" tabindex="-1"></a><span class="fu">plot</span>(<span class="fu">fitted</span>(gala_model), <span class="fu">resid</span>(gala_model), <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>,</span>
+<span id="cb1055-2"><a href="transformations.html#cb1055-2" tabindex="-1"></a>     <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">cex =</span> <span class="fl">1.5</span>, <span class="at">xlab =</span> <span class="st">&quot;Fitted&quot;</span>, <span class="at">ylab =</span> <span class="st">&quot;Residuals&quot;</span>)</span>
+<span id="cb1055-3"><a href="transformations.html#cb1055-3" tabindex="-1"></a><span class="fu">abline</span>(<span class="at">h =</span> <span class="dv">0</span>, <span class="at">lty =</span> <span class="dv">2</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
 <p><img src="transformations_files/figure-html/unnamed-chunk-22-1.png" width="672" style="display: block; margin: auto;" /></p>
 <p>Even though there is not a lot of data for large fitted values, it still seems very clear that the constant variance assumption is violated.</p>
-<div class="sourceCode" id="cb1056"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1056-1"><a href="transformations.html#cb1056-1" aria-hidden="true" tabindex="-1"></a><span class="fu">boxcox</span>(gala_model, <span class="at">lambda =</span> <span class="fu">seq</span>(<span class="sc">-</span><span class="fl">0.25</span>, <span class="fl">0.75</span>, <span class="at">by =</span> <span class="fl">0.05</span>), <span class="at">plotit =</span> <span class="cn">TRUE</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1056"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1056-1"><a href="transformations.html#cb1056-1" tabindex="-1"></a><span class="fu">boxcox</span>(gala_model, <span class="at">lambda =</span> <span class="fu">seq</span>(<span class="sc">-</span><span class="fl">0.25</span>, <span class="fl">0.75</span>, <span class="at">by =</span> <span class="fl">0.05</span>), <span class="at">plotit =</span> <span class="cn">TRUE</span>)</span></code></pre></div>
 <p><img src="transformations_files/figure-html/unnamed-chunk-23-1.png" width="672" style="display: block; margin: auto;" /></p>
 <p>Using the Box-Cox method, we see that <span class="math inline">\(\lambda = 0.3\)</span> is both in the confidence interval, and is extremely close to the maximum, which suggests a transformation of the form</p>
 <p><span class="math display">\[
 \frac{y^\lambda - 1}{\lambda} = \frac{y^{0.3} - 1}{0.3}.
 \]</span></p>
 <p>We then fit a model with this transformation applied to the response.</p>
-<div class="sourceCode" id="cb1057"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1057-1"><a href="transformations.html#cb1057-1" aria-hidden="true" tabindex="-1"></a>gala_model_cox <span class="ot">=</span> <span class="fu">lm</span>((((Species <span class="sc">^</span> <span class="fl">0.3</span>) <span class="sc">-</span> <span class="dv">1</span>) <span class="sc">/</span> <span class="fl">0.3</span>) <span class="sc">~</span> Area <span class="sc">+</span> Elevation <span class="sc">+</span> Nearest <span class="sc">+</span> Scruz <span class="sc">+</span> Adjacent, <span class="at">data =</span> gala)</span></code></pre></div>
-<div class="sourceCode" id="cb1058"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1058-1"><a href="transformations.html#cb1058-1" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(<span class="fu">fitted</span>(gala_model_cox), <span class="fu">resid</span>(gala_model_cox), <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>,</span>
-<span id="cb1058-2"><a href="transformations.html#cb1058-2" aria-hidden="true" tabindex="-1"></a>     <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">cex =</span> <span class="fl">1.5</span>, <span class="at">xlab =</span> <span class="st">&quot;Fitted&quot;</span>, <span class="at">ylab =</span> <span class="st">&quot;Residuals&quot;</span>)</span>
-<span id="cb1058-3"><a href="transformations.html#cb1058-3" aria-hidden="true" tabindex="-1"></a><span class="fu">abline</span>(<span class="at">h =</span> <span class="dv">0</span>, <span class="at">lty =</span> <span class="dv">2</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1057"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1057-1"><a href="transformations.html#cb1057-1" tabindex="-1"></a>gala_model_cox <span class="ot">=</span> <span class="fu">lm</span>((((Species <span class="sc">^</span> <span class="fl">0.3</span>) <span class="sc">-</span> <span class="dv">1</span>) <span class="sc">/</span> <span class="fl">0.3</span>) <span class="sc">~</span> Area <span class="sc">+</span> Elevation <span class="sc">+</span> Nearest <span class="sc">+</span> Scruz <span class="sc">+</span> Adjacent, <span class="at">data =</span> gala)</span></code></pre></div>
+<div class="sourceCode" id="cb1058"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1058-1"><a href="transformations.html#cb1058-1" tabindex="-1"></a><span class="fu">plot</span>(<span class="fu">fitted</span>(gala_model_cox), <span class="fu">resid</span>(gala_model_cox), <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>,</span>
+<span id="cb1058-2"><a href="transformations.html#cb1058-2" tabindex="-1"></a>     <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">cex =</span> <span class="fl">1.5</span>, <span class="at">xlab =</span> <span class="st">&quot;Fitted&quot;</span>, <span class="at">ylab =</span> <span class="st">&quot;Residuals&quot;</span>)</span>
+<span id="cb1058-3"><a href="transformations.html#cb1058-3" tabindex="-1"></a><span class="fu">abline</span>(<span class="at">h =</span> <span class="dv">0</span>, <span class="at">lty =</span> <span class="dv">2</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
 <p><img src="transformations_files/figure-html/unnamed-chunk-25-1.png" width="672" style="display: block; margin: auto;" /></p>
 <p>The resulting fitted versus residuals plot looks much better!</p>
 <p>Lastly, we return to the <code>initech</code> data, and the <code>initech_fit</code> model we had used earlier. Recall that this was the untransformed model, that we used a <span class="math inline">\(\log\)</span> transform to fix.</p>
-<div class="sourceCode" id="cb1059"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1059-1"><a href="transformations.html#cb1059-1" aria-hidden="true" tabindex="-1"></a><span class="fu">boxcox</span>(initech_fit)</span></code></pre></div>
+<div class="sourceCode" id="cb1059"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1059-1"><a href="transformations.html#cb1059-1" tabindex="-1"></a><span class="fu">boxcox</span>(initech_fit)</span></code></pre></div>
 <p><img src="transformations_files/figure-html/unnamed-chunk-26-1.png" width="672" style="display: block; margin: auto;" /></p>
 <p>Using the Box-Cox method, we see that <span class="math inline">\(\lambda = 0\)</span> is both in the interval, and extremely close to the maximum, which suggests a transformation of the form</p>
 <p><span class="math display">\[
@@ -706,7 +706,7 @@ <h3><span class="header-section-number">14.1.2</span> Box-Cox Transformations<a
 <div id="predictor-transformation" class="section level2 hasAnchor" number="14.2">
 <h2><span class="header-section-number">14.2</span> Predictor Transformation<a href="transformations.html#predictor-transformation" class="anchor-section" aria-label="Anchor link to header"></a></h2>
 <p>In addition to transformation of the response variable, we can also consider transformations of predictor variables. Sometimes these transformations can help with violation of model assumptions, and other times they can be used to simply fit a more flexible model.</p>
-<div class="sourceCode" id="cb1060"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1060-1"><a href="transformations.html#cb1060-1" aria-hidden="true" tabindex="-1"></a><span class="fu">str</span>(autompg)</span></code></pre></div>
+<div class="sourceCode" id="cb1060"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1060-1"><a href="transformations.html#cb1060-1" tabindex="-1"></a><span class="fu">str</span>(autompg)</span></code></pre></div>
 <pre><code>## &#39;data.frame&#39;:    383 obs. of  9 variables:
 ##  $ mpg     : num  18 15 18 16 17 15 14 14 14 15 ...
 ##  $ cyl     : Factor w/ 3 levels &quot;4&quot;,&quot;6&quot;,&quot;8&quot;: 3 3 3 3 3 3 3 3 3 3 ...
@@ -718,31 +718,31 @@ <h2><span class="header-section-number">14.2</span> Predictor Transformation<a h
 ##  $ origin  : int  1 1 1 1 1 1 1 1 1 1 ...
 ##  $ domestic: num  1 1 1 1 1 1 1 1 1 1 ...</code></pre>
 <p>Recall the <code>autompg</code> dataset from the previous chapter. Here we will attempt to model <code>mpg</code> as a function of <code>hp</code>.</p>
-<div class="sourceCode" id="cb1062"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1062-1"><a href="transformations.html#cb1062-1" aria-hidden="true" tabindex="-1"></a><span class="fu">par</span>(<span class="at">mfrow =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>))</span>
-<span id="cb1062-2"><a href="transformations.html#cb1062-2" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(mpg <span class="sc">~</span> hp, <span class="at">data =</span> autompg, <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">cex =</span> <span class="fl">1.5</span>)</span>
-<span id="cb1062-3"><a href="transformations.html#cb1062-3" aria-hidden="true" tabindex="-1"></a>mpg_hp <span class="ot">=</span> <span class="fu">lm</span>(mpg <span class="sc">~</span> hp, <span class="at">data =</span> autompg)</span>
-<span id="cb1062-4"><a href="transformations.html#cb1062-4" aria-hidden="true" tabindex="-1"></a><span class="fu">abline</span>(mpg_hp, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span>
-<span id="cb1062-5"><a href="transformations.html#cb1062-5" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(<span class="fu">fitted</span>(mpg_hp), <span class="fu">resid</span>(mpg_hp), <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>,</span>
-<span id="cb1062-6"><a href="transformations.html#cb1062-6" aria-hidden="true" tabindex="-1"></a>     <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">cex =</span> <span class="fl">1.5</span>, <span class="at">xlab =</span> <span class="st">&quot;Fitted&quot;</span>, <span class="at">ylab =</span> <span class="st">&quot;Residuals&quot;</span>)</span>
-<span id="cb1062-7"><a href="transformations.html#cb1062-7" aria-hidden="true" tabindex="-1"></a><span class="fu">abline</span>(<span class="at">h =</span> <span class="dv">0</span>, <span class="at">lty =</span> <span class="dv">2</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1062"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1062-1"><a href="transformations.html#cb1062-1" tabindex="-1"></a><span class="fu">par</span>(<span class="at">mfrow =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>))</span>
+<span id="cb1062-2"><a href="transformations.html#cb1062-2" tabindex="-1"></a><span class="fu">plot</span>(mpg <span class="sc">~</span> hp, <span class="at">data =</span> autompg, <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">cex =</span> <span class="fl">1.5</span>)</span>
+<span id="cb1062-3"><a href="transformations.html#cb1062-3" tabindex="-1"></a>mpg_hp <span class="ot">=</span> <span class="fu">lm</span>(mpg <span class="sc">~</span> hp, <span class="at">data =</span> autompg)</span>
+<span id="cb1062-4"><a href="transformations.html#cb1062-4" tabindex="-1"></a><span class="fu">abline</span>(mpg_hp, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span>
+<span id="cb1062-5"><a href="transformations.html#cb1062-5" tabindex="-1"></a><span class="fu">plot</span>(<span class="fu">fitted</span>(mpg_hp), <span class="fu">resid</span>(mpg_hp), <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>,</span>
+<span id="cb1062-6"><a href="transformations.html#cb1062-6" tabindex="-1"></a>     <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">cex =</span> <span class="fl">1.5</span>, <span class="at">xlab =</span> <span class="st">&quot;Fitted&quot;</span>, <span class="at">ylab =</span> <span class="st">&quot;Residuals&quot;</span>)</span>
+<span id="cb1062-7"><a href="transformations.html#cb1062-7" tabindex="-1"></a><span class="fu">abline</span>(<span class="at">h =</span> <span class="dv">0</span>, <span class="at">lty =</span> <span class="dv">2</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
 <p><img src="transformations_files/figure-html/unnamed-chunk-29-1.png" width="768" style="display: block; margin: auto;" /></p>
 <p>We first attempt SLR, but we see a rather obvious pattern in the fitted versus residuals plot, which includes increasing variance, so we attempt a <span class="math inline">\(\log\)</span> transform of the response.</p>
-<div class="sourceCode" id="cb1063"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1063-1"><a href="transformations.html#cb1063-1" aria-hidden="true" tabindex="-1"></a><span class="fu">par</span>(<span class="at">mfrow =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>))</span>
-<span id="cb1063-2"><a href="transformations.html#cb1063-2" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(<span class="fu">log</span>(mpg) <span class="sc">~</span> hp, <span class="at">data =</span> autompg, <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">cex =</span> <span class="fl">1.5</span>)</span>
-<span id="cb1063-3"><a href="transformations.html#cb1063-3" aria-hidden="true" tabindex="-1"></a>mpg_hp_log <span class="ot">=</span> <span class="fu">lm</span>(<span class="fu">log</span>(mpg) <span class="sc">~</span> hp, <span class="at">data =</span> autompg)</span>
-<span id="cb1063-4"><a href="transformations.html#cb1063-4" aria-hidden="true" tabindex="-1"></a><span class="fu">abline</span>(mpg_hp_log, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span>
-<span id="cb1063-5"><a href="transformations.html#cb1063-5" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(<span class="fu">fitted</span>(mpg_hp_log), <span class="fu">resid</span>(mpg_hp_log), <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>,</span>
-<span id="cb1063-6"><a href="transformations.html#cb1063-6" aria-hidden="true" tabindex="-1"></a>     <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">cex =</span> <span class="fl">1.5</span>, <span class="at">xlab =</span> <span class="st">&quot;Fitted&quot;</span>, <span class="at">ylab =</span> <span class="st">&quot;Residuals&quot;</span>)</span>
-<span id="cb1063-7"><a href="transformations.html#cb1063-7" aria-hidden="true" tabindex="-1"></a><span class="fu">abline</span>(<span class="at">h =</span> <span class="dv">0</span>, <span class="at">lty =</span> <span class="dv">2</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1063"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1063-1"><a href="transformations.html#cb1063-1" tabindex="-1"></a><span class="fu">par</span>(<span class="at">mfrow =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>))</span>
+<span id="cb1063-2"><a href="transformations.html#cb1063-2" tabindex="-1"></a><span class="fu">plot</span>(<span class="fu">log</span>(mpg) <span class="sc">~</span> hp, <span class="at">data =</span> autompg, <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">cex =</span> <span class="fl">1.5</span>)</span>
+<span id="cb1063-3"><a href="transformations.html#cb1063-3" tabindex="-1"></a>mpg_hp_log <span class="ot">=</span> <span class="fu">lm</span>(<span class="fu">log</span>(mpg) <span class="sc">~</span> hp, <span class="at">data =</span> autompg)</span>
+<span id="cb1063-4"><a href="transformations.html#cb1063-4" tabindex="-1"></a><span class="fu">abline</span>(mpg_hp_log, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span>
+<span id="cb1063-5"><a href="transformations.html#cb1063-5" tabindex="-1"></a><span class="fu">plot</span>(<span class="fu">fitted</span>(mpg_hp_log), <span class="fu">resid</span>(mpg_hp_log), <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>,</span>
+<span id="cb1063-6"><a href="transformations.html#cb1063-6" tabindex="-1"></a>     <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">cex =</span> <span class="fl">1.5</span>, <span class="at">xlab =</span> <span class="st">&quot;Fitted&quot;</span>, <span class="at">ylab =</span> <span class="st">&quot;Residuals&quot;</span>)</span>
+<span id="cb1063-7"><a href="transformations.html#cb1063-7" tabindex="-1"></a><span class="fu">abline</span>(<span class="at">h =</span> <span class="dv">0</span>, <span class="at">lty =</span> <span class="dv">2</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
 <p><img src="transformations_files/figure-html/unnamed-chunk-30-1.png" width="768" style="display: block; margin: auto;" /></p>
 <p>After performing the <span class="math inline">\(\log\)</span> transform of the response, we still have some of the same issues with the fitted versus response. Now, we will try also <span class="math inline">\(\log\)</span> transforming the <strong>predictor</strong>.</p>
-<div class="sourceCode" id="cb1064"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1064-1"><a href="transformations.html#cb1064-1" aria-hidden="true" tabindex="-1"></a><span class="fu">par</span>(<span class="at">mfrow =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>))</span>
-<span id="cb1064-2"><a href="transformations.html#cb1064-2" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(<span class="fu">log</span>(mpg) <span class="sc">~</span> <span class="fu">log</span>(hp), <span class="at">data =</span> autompg, <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">cex =</span> <span class="fl">1.5</span>)</span>
-<span id="cb1064-3"><a href="transformations.html#cb1064-3" aria-hidden="true" tabindex="-1"></a>mpg_hp_loglog <span class="ot">=</span> <span class="fu">lm</span>(<span class="fu">log</span>(mpg) <span class="sc">~</span> <span class="fu">log</span>(hp), <span class="at">data =</span> autompg)</span>
-<span id="cb1064-4"><a href="transformations.html#cb1064-4" aria-hidden="true" tabindex="-1"></a><span class="fu">abline</span>(mpg_hp_loglog, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span>
-<span id="cb1064-5"><a href="transformations.html#cb1064-5" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(<span class="fu">fitted</span>(mpg_hp_loglog), <span class="fu">resid</span>(mpg_hp_loglog), <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>,</span>
-<span id="cb1064-6"><a href="transformations.html#cb1064-6" aria-hidden="true" tabindex="-1"></a>     <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">cex =</span> <span class="fl">1.5</span>, <span class="at">xlab =</span> <span class="st">&quot;Fitted&quot;</span>, <span class="at">ylab =</span> <span class="st">&quot;Residuals&quot;</span>)</span>
-<span id="cb1064-7"><a href="transformations.html#cb1064-7" aria-hidden="true" tabindex="-1"></a><span class="fu">abline</span>(<span class="at">h =</span> <span class="dv">0</span>, <span class="at">lty =</span> <span class="dv">2</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1064"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1064-1"><a href="transformations.html#cb1064-1" tabindex="-1"></a><span class="fu">par</span>(<span class="at">mfrow =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>))</span>
+<span id="cb1064-2"><a href="transformations.html#cb1064-2" tabindex="-1"></a><span class="fu">plot</span>(<span class="fu">log</span>(mpg) <span class="sc">~</span> <span class="fu">log</span>(hp), <span class="at">data =</span> autompg, <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">cex =</span> <span class="fl">1.5</span>)</span>
+<span id="cb1064-3"><a href="transformations.html#cb1064-3" tabindex="-1"></a>mpg_hp_loglog <span class="ot">=</span> <span class="fu">lm</span>(<span class="fu">log</span>(mpg) <span class="sc">~</span> <span class="fu">log</span>(hp), <span class="at">data =</span> autompg)</span>
+<span id="cb1064-4"><a href="transformations.html#cb1064-4" tabindex="-1"></a><span class="fu">abline</span>(mpg_hp_loglog, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span>
+<span id="cb1064-5"><a href="transformations.html#cb1064-5" tabindex="-1"></a><span class="fu">plot</span>(<span class="fu">fitted</span>(mpg_hp_loglog), <span class="fu">resid</span>(mpg_hp_loglog), <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>,</span>
+<span id="cb1064-6"><a href="transformations.html#cb1064-6" tabindex="-1"></a>     <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">cex =</span> <span class="fl">1.5</span>, <span class="at">xlab =</span> <span class="st">&quot;Fitted&quot;</span>, <span class="at">ylab =</span> <span class="st">&quot;Residuals&quot;</span>)</span>
+<span id="cb1064-7"><a href="transformations.html#cb1064-7" tabindex="-1"></a><span class="fu">abline</span>(<span class="at">h =</span> <span class="dv">0</span>, <span class="at">lty =</span> <span class="dv">2</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
 <p><img src="transformations_files/figure-html/unnamed-chunk-31-1.png" width="768" style="display: block; margin: auto;" /></p>
 <p>Here, our fitted versus residuals plot looks good.</p>
 <div id="polynomials" class="section level3 hasAnchor" number="14.2.1">
@@ -750,10 +750,10 @@ <h3><span class="header-section-number">14.2.1</span> Polynomials<a href="transf
 <p>Another very common “transformation” of a predictor variable is the use of polynomial transformations. They are extremely useful as they allow for more flexible models, but do not change the units of the variables.</p>
 <p>It should come as no surprise that sales of a product are related to the advertising budget for the product, but there are diminishing returns. A company cannot always expect linear returns based on an increased advertising budget.</p>
 <p>Consider monthly data for the sales of <em>Initech</em> widgets, <span class="math inline">\(y\)</span>, as a function of <em>Initech</em>’s advertising expenditure for said widget, <span class="math inline">\(x\)</span>, both in ten thousand dollars. The data can be found in <a href="data/marketing.csv"><code>marketing.csv</code></a>.</p>
-<div class="sourceCode" id="cb1065"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1065-1"><a href="transformations.html#cb1065-1" aria-hidden="true" tabindex="-1"></a>marketing <span class="ot">=</span> <span class="fu">read.csv</span>(<span class="st">&quot;data/marketing.csv&quot;</span>)</span></code></pre></div>
-<div class="sourceCode" id="cb1066"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1066-1"><a href="transformations.html#cb1066-1" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(sales <span class="sc">~</span> advert, <span class="at">data =</span> marketing, </span>
-<span id="cb1066-2"><a href="transformations.html#cb1066-2" aria-hidden="true" tabindex="-1"></a>     <span class="at">xlab =</span> <span class="st">&quot;Advert Spending (in $10,000)&quot;</span>, <span class="at">ylab =</span> <span class="st">&quot;Sales (in $10,000)&quot;</span>,</span>
-<span id="cb1066-3"><a href="transformations.html#cb1066-3" aria-hidden="true" tabindex="-1"></a>     <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">cex =</span> <span class="dv">2</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1065"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1065-1"><a href="transformations.html#cb1065-1" tabindex="-1"></a>marketing <span class="ot">=</span> <span class="fu">read.csv</span>(<span class="st">&quot;data/marketing.csv&quot;</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1066"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1066-1"><a href="transformations.html#cb1066-1" tabindex="-1"></a><span class="fu">plot</span>(sales <span class="sc">~</span> advert, <span class="at">data =</span> marketing, </span>
+<span id="cb1066-2"><a href="transformations.html#cb1066-2" tabindex="-1"></a>     <span class="at">xlab =</span> <span class="st">&quot;Advert Spending (in $10,000)&quot;</span>, <span class="at">ylab =</span> <span class="st">&quot;Sales (in $10,000)&quot;</span>,</span>
+<span id="cb1066-3"><a href="transformations.html#cb1066-3" tabindex="-1"></a>     <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">cex =</span> <span class="dv">2</span>)</span></code></pre></div>
 <p><img src="transformations_files/figure-html/unnamed-chunk-33-1.png" width="672" style="display: block; margin: auto;" /></p>
 <p>We would like to fit the model,</p>
 <p><span class="math display">\[
@@ -787,8 +787,8 @@ <h3><span class="header-section-number">14.2.1</span> Polynomials<a href="transf
 <p><span class="math display">\[
 \hat{\beta}_j \sim N\left(\beta_j, \sigma^2 C_{jj}  \right).
 \]</span></p>
-<div class="sourceCode" id="cb1067"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1067-1"><a href="transformations.html#cb1067-1" aria-hidden="true" tabindex="-1"></a>mark_mod <span class="ot">=</span> <span class="fu">lm</span>(sales <span class="sc">~</span> advert, <span class="at">data =</span> marketing)</span>
-<span id="cb1067-2"><a href="transformations.html#cb1067-2" aria-hidden="true" tabindex="-1"></a><span class="fu">summary</span>(mark_mod)</span></code></pre></div>
+<div class="sourceCode" id="cb1067"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1067-1"><a href="transformations.html#cb1067-1" tabindex="-1"></a>mark_mod <span class="ot">=</span> <span class="fu">lm</span>(sales <span class="sc">~</span> advert, <span class="at">data =</span> marketing)</span>
+<span id="cb1067-2"><a href="transformations.html#cb1067-2" tabindex="-1"></a><span class="fu">summary</span>(mark_mod)</span></code></pre></div>
 <pre><code>## 
 ## Call:
 ## lm(formula = sales ~ advert, data = marketing)
@@ -808,8 +808,8 @@ <h3><span class="header-section-number">14.2.1</span> Polynomials<a href="transf
 ## Multiple R-squared:  0.8949, Adjusted R-squared:  0.8894 
 ## F-statistic: 161.8 on 1 and 19 DF,  p-value: 9.646e-11</code></pre>
 <p>While the SLR model is significant, the fitted versus residuals plot would have a very clear pattern.</p>
-<div class="sourceCode" id="cb1069"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1069-1"><a href="transformations.html#cb1069-1" aria-hidden="true" tabindex="-1"></a>mark_mod_poly2 <span class="ot">=</span> <span class="fu">lm</span>(sales <span class="sc">~</span> advert <span class="sc">+</span> <span class="fu">I</span>(advert <span class="sc">^</span> <span class="dv">2</span>), <span class="at">data =</span> marketing)</span>
-<span id="cb1069-2"><a href="transformations.html#cb1069-2" aria-hidden="true" tabindex="-1"></a><span class="fu">summary</span>(mark_mod_poly2)</span></code></pre></div>
+<div class="sourceCode" id="cb1069"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1069-1"><a href="transformations.html#cb1069-1" tabindex="-1"></a>mark_mod_poly2 <span class="ot">=</span> <span class="fu">lm</span>(sales <span class="sc">~</span> advert <span class="sc">+</span> <span class="fu">I</span>(advert <span class="sc">^</span> <span class="dv">2</span>), <span class="at">data =</span> marketing)</span>
+<span id="cb1069-2"><a href="transformations.html#cb1069-2" tabindex="-1"></a><span class="fu">summary</span>(mark_mod_poly2)</span></code></pre></div>
 <pre><code>## 
 ## Call:
 ## lm(formula = sales ~ advert + I(advert^2), data = marketing)
@@ -830,14 +830,14 @@ <h3><span class="header-section-number">14.2.1</span> Polynomials<a href="transf
 ## Multiple R-squared:  0.9587, Adjusted R-squared:  0.9541 
 ## F-statistic:   209 on 2 and 18 DF,  p-value: 3.486e-13</code></pre>
 <p>To add the second order term we need to use the <code>I()</code> function in the model specification around our newly created predictor. We see that with the first order term in the model, the quadratic term is also significant.</p>
-<div class="sourceCode" id="cb1071"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1071-1"><a href="transformations.html#cb1071-1" aria-hidden="true" tabindex="-1"></a>n <span class="ot">=</span> <span class="fu">length</span>(marketing<span class="sc">$</span>advert)</span>
-<span id="cb1071-2"><a href="transformations.html#cb1071-2" aria-hidden="true" tabindex="-1"></a>X <span class="ot">=</span> <span class="fu">cbind</span>(<span class="fu">rep</span>(<span class="dv">1</span>, n), marketing<span class="sc">$</span>advert, marketing<span class="sc">$</span>advert <span class="sc">^</span> <span class="dv">2</span>)</span>
-<span id="cb1071-3"><a href="transformations.html#cb1071-3" aria-hidden="true" tabindex="-1"></a><span class="fu">t</span>(X) <span class="sc">%*%</span> X</span></code></pre></div>
+<div class="sourceCode" id="cb1071"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1071-1"><a href="transformations.html#cb1071-1" tabindex="-1"></a>n <span class="ot">=</span> <span class="fu">length</span>(marketing<span class="sc">$</span>advert)</span>
+<span id="cb1071-2"><a href="transformations.html#cb1071-2" tabindex="-1"></a>X <span class="ot">=</span> <span class="fu">cbind</span>(<span class="fu">rep</span>(<span class="dv">1</span>, n), marketing<span class="sc">$</span>advert, marketing<span class="sc">$</span>advert <span class="sc">^</span> <span class="dv">2</span>)</span>
+<span id="cb1071-3"><a href="transformations.html#cb1071-3" tabindex="-1"></a><span class="fu">t</span>(X) <span class="sc">%*%</span> X</span></code></pre></div>
 <pre><code>##         [,1]     [,2]      [,3]
 ## [1,]   21.00   120.70   1107.95
 ## [2,]  120.70  1107.95  12385.86
 ## [3,] 1107.95 12385.86 151369.12</code></pre>
-<div class="sourceCode" id="cb1073"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1073-1"><a href="transformations.html#cb1073-1" aria-hidden="true" tabindex="-1"></a><span class="fu">solve</span>(<span class="fu">t</span>(X) <span class="sc">%*%</span> X) <span class="sc">%*%</span> <span class="fu">t</span>(X) <span class="sc">%*%</span> marketing<span class="sc">$</span>sales</span></code></pre></div>
+<div class="sourceCode" id="cb1073"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1073-1"><a href="transformations.html#cb1073-1" tabindex="-1"></a><span class="fu">solve</span>(<span class="fu">t</span>(X) <span class="sc">%*%</span> X) <span class="sc">%*%</span> <span class="fu">t</span>(X) <span class="sc">%*%</span> marketing<span class="sc">$</span>sales</span></code></pre></div>
 <pre><code>##             [,1]
 ## [1,]  6.76161045
 ## [2,]  2.46230964
@@ -856,8 +856,8 @@ <h3><span class="header-section-number">14.2.1</span> Polynomials<a href="transf
   1      &amp; x_{n}  &amp; x_{n}^2  &amp; x_{n}^3    \\
   \end{bmatrix}
 \]</span></p>
-<div class="sourceCode" id="cb1075"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1075-1"><a href="transformations.html#cb1075-1" aria-hidden="true" tabindex="-1"></a>mark_mod_poly3 <span class="ot">=</span> <span class="fu">lm</span>(sales <span class="sc">~</span> advert <span class="sc">+</span> <span class="fu">I</span>(advert <span class="sc">^</span> <span class="dv">2</span>) <span class="sc">+</span> <span class="fu">I</span>(advert <span class="sc">^</span> <span class="dv">3</span>), <span class="at">data =</span> marketing)</span>
-<span id="cb1075-2"><a href="transformations.html#cb1075-2" aria-hidden="true" tabindex="-1"></a><span class="fu">summary</span>(mark_mod_poly3)</span></code></pre></div>
+<div class="sourceCode" id="cb1075"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1075-1"><a href="transformations.html#cb1075-1" tabindex="-1"></a>mark_mod_poly3 <span class="ot">=</span> <span class="fu">lm</span>(sales <span class="sc">~</span> advert <span class="sc">+</span> <span class="fu">I</span>(advert <span class="sc">^</span> <span class="dv">2</span>) <span class="sc">+</span> <span class="fu">I</span>(advert <span class="sc">^</span> <span class="dv">3</span>), <span class="at">data =</span> marketing)</span>
+<span id="cb1075-2"><a href="transformations.html#cb1075-2" tabindex="-1"></a><span class="fu">summary</span>(mark_mod_poly3)</span></code></pre></div>
 <pre><code>## 
 ## Call:
 ## lm(formula = sales ~ advert + I(advert^2) + I(advert^3), data = marketing)
@@ -879,38 +879,38 @@ <h3><span class="header-section-number">14.2.1</span> Polynomials<a href="transf
 ## Multiple R-squared:  0.9821, Adjusted R-squared:  0.9789 
 ## F-statistic: 310.2 on 3 and 17 DF,  p-value: 4.892e-15</code></pre>
 <p>Now we see that with the first and second order terms in the model, the third order term is also significant. But does this make sense practically? The following plot should gives hints as to why it doesn’t. (The model with the third order term doesn’t have diminishing returns!)</p>
-<div class="sourceCode" id="cb1077"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1077-1"><a href="transformations.html#cb1077-1" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(sales <span class="sc">~</span> advert, <span class="at">data =</span> marketing, </span>
-<span id="cb1077-2"><a href="transformations.html#cb1077-2" aria-hidden="true" tabindex="-1"></a>     <span class="at">xlab =</span> <span class="st">&quot;Advert Spending (in $10,000)&quot;</span>, <span class="at">ylab =</span> <span class="st">&quot;Sales (in $10,000)&quot;</span>,</span>
-<span id="cb1077-3"><a href="transformations.html#cb1077-3" aria-hidden="true" tabindex="-1"></a>     <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">cex =</span> <span class="dv">2</span>)</span>
-<span id="cb1077-4"><a href="transformations.html#cb1077-4" aria-hidden="true" tabindex="-1"></a><span class="fu">abline</span>(mark_mod, <span class="at">lty =</span> <span class="dv">2</span>, <span class="at">col =</span> <span class="st">&quot;green&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span>
-<span id="cb1077-5"><a href="transformations.html#cb1077-5" aria-hidden="true" tabindex="-1"></a>xplot <span class="ot">=</span> <span class="fu">seq</span>(<span class="dv">0</span>, <span class="dv">16</span>, <span class="at">by =</span> <span class="fl">0.01</span>)</span>
-<span id="cb1077-6"><a href="transformations.html#cb1077-6" aria-hidden="true" tabindex="-1"></a><span class="fu">lines</span>(xplot, <span class="fu">predict</span>(mark_mod_poly2, <span class="at">newdata =</span> <span class="fu">data.frame</span>(<span class="at">advert =</span> xplot)),</span>
-<span id="cb1077-7"><a href="transformations.html#cb1077-7" aria-hidden="true" tabindex="-1"></a>      <span class="at">col =</span> <span class="st">&quot;blue&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span>
-<span id="cb1077-8"><a href="transformations.html#cb1077-8" aria-hidden="true" tabindex="-1"></a><span class="fu">lines</span>(xplot, <span class="fu">predict</span>(mark_mod_poly3, <span class="at">newdata =</span> <span class="fu">data.frame</span>(<span class="at">advert =</span> xplot)),</span>
-<span id="cb1077-9"><a href="transformations.html#cb1077-9" aria-hidden="true" tabindex="-1"></a>      <span class="at">col =</span> <span class="st">&quot;red&quot;</span>, <span class="at">lty =</span> <span class="dv">3</span>, <span class="at">lwd =</span> <span class="dv">3</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1077"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1077-1"><a href="transformations.html#cb1077-1" tabindex="-1"></a><span class="fu">plot</span>(sales <span class="sc">~</span> advert, <span class="at">data =</span> marketing, </span>
+<span id="cb1077-2"><a href="transformations.html#cb1077-2" tabindex="-1"></a>     <span class="at">xlab =</span> <span class="st">&quot;Advert Spending (in $10,000)&quot;</span>, <span class="at">ylab =</span> <span class="st">&quot;Sales (in $10,000)&quot;</span>,</span>
+<span id="cb1077-3"><a href="transformations.html#cb1077-3" tabindex="-1"></a>     <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">cex =</span> <span class="dv">2</span>)</span>
+<span id="cb1077-4"><a href="transformations.html#cb1077-4" tabindex="-1"></a><span class="fu">abline</span>(mark_mod, <span class="at">lty =</span> <span class="dv">2</span>, <span class="at">col =</span> <span class="st">&quot;green&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span>
+<span id="cb1077-5"><a href="transformations.html#cb1077-5" tabindex="-1"></a>xplot <span class="ot">=</span> <span class="fu">seq</span>(<span class="dv">0</span>, <span class="dv">16</span>, <span class="at">by =</span> <span class="fl">0.01</span>)</span>
+<span id="cb1077-6"><a href="transformations.html#cb1077-6" tabindex="-1"></a><span class="fu">lines</span>(xplot, <span class="fu">predict</span>(mark_mod_poly2, <span class="at">newdata =</span> <span class="fu">data.frame</span>(<span class="at">advert =</span> xplot)),</span>
+<span id="cb1077-7"><a href="transformations.html#cb1077-7" tabindex="-1"></a>      <span class="at">col =</span> <span class="st">&quot;blue&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span>
+<span id="cb1077-8"><a href="transformations.html#cb1077-8" tabindex="-1"></a><span class="fu">lines</span>(xplot, <span class="fu">predict</span>(mark_mod_poly3, <span class="at">newdata =</span> <span class="fu">data.frame</span>(<span class="at">advert =</span> xplot)),</span>
+<span id="cb1077-9"><a href="transformations.html#cb1077-9" tabindex="-1"></a>      <span class="at">col =</span> <span class="st">&quot;red&quot;</span>, <span class="at">lty =</span> <span class="dv">3</span>, <span class="at">lwd =</span> <span class="dv">3</span>)</span></code></pre></div>
 <p><img src="transformations_files/figure-html/unnamed-chunk-38-1.png" width="672" style="display: block; margin: auto;" /></p>
 <p>The previous plot was made using base graphics in <code>R</code>. The next plot was made using the package <a href="https://ggplot2.tidyverse.org/" target="_blank"><code>ggplot2</code></a>, an increasingly popular plotting method in <code>R</code>.</p>
-<div class="sourceCode" id="cb1078"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1078-1"><a href="transformations.html#cb1078-1" aria-hidden="true" tabindex="-1"></a><span class="fu">library</span>(ggplot2)</span>
-<span id="cb1078-2"><a href="transformations.html#cb1078-2" aria-hidden="true" tabindex="-1"></a><span class="fu">ggplot</span>(<span class="at">data =</span> marketing, <span class="fu">aes</span>(<span class="at">x =</span> advert, <span class="at">y =</span> sales)) <span class="sc">+</span></span>
-<span id="cb1078-3"><a href="transformations.html#cb1078-3" aria-hidden="true" tabindex="-1"></a>  <span class="fu">stat_smooth</span>(<span class="at">method =</span> <span class="st">&quot;lm&quot;</span>, <span class="at">se =</span> <span class="cn">FALSE</span>, <span class="at">color =</span> <span class="st">&quot;green&quot;</span>, <span class="at">formula =</span> y <span class="sc">~</span> x) <span class="sc">+</span></span>
-<span id="cb1078-4"><a href="transformations.html#cb1078-4" aria-hidden="true" tabindex="-1"></a>  <span class="fu">stat_smooth</span>(<span class="at">method =</span> <span class="st">&quot;lm&quot;</span>, <span class="at">se =</span> <span class="cn">FALSE</span>, <span class="at">color =</span> <span class="st">&quot;blue&quot;</span>, <span class="at">formula =</span> y <span class="sc">~</span> x <span class="sc">+</span> <span class="fu">I</span>(x <span class="sc">^</span> <span class="dv">2</span>)) <span class="sc">+</span></span>
-<span id="cb1078-5"><a href="transformations.html#cb1078-5" aria-hidden="true" tabindex="-1"></a>  <span class="fu">stat_smooth</span>(<span class="at">method =</span> <span class="st">&quot;lm&quot;</span>, <span class="at">se =</span> <span class="cn">FALSE</span>, <span class="at">color =</span> <span class="st">&quot;red&quot;</span>, <span class="at">formula =</span> y <span class="sc">~</span> x <span class="sc">+</span> <span class="fu">I</span>(x <span class="sc">^</span> <span class="dv">2</span>)<span class="sc">+</span> <span class="fu">I</span>(x <span class="sc">^</span> <span class="dv">3</span>)) <span class="sc">+</span></span>
-<span id="cb1078-6"><a href="transformations.html#cb1078-6" aria-hidden="true" tabindex="-1"></a>  <span class="fu">geom_point</span>(<span class="at">colour =</span> <span class="st">&quot;black&quot;</span>, <span class="at">size =</span> <span class="dv">3</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1078"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1078-1"><a href="transformations.html#cb1078-1" tabindex="-1"></a><span class="fu">library</span>(ggplot2)</span>
+<span id="cb1078-2"><a href="transformations.html#cb1078-2" tabindex="-1"></a><span class="fu">ggplot</span>(<span class="at">data =</span> marketing, <span class="fu">aes</span>(<span class="at">x =</span> advert, <span class="at">y =</span> sales)) <span class="sc">+</span></span>
+<span id="cb1078-3"><a href="transformations.html#cb1078-3" tabindex="-1"></a>  <span class="fu">stat_smooth</span>(<span class="at">method =</span> <span class="st">&quot;lm&quot;</span>, <span class="at">se =</span> <span class="cn">FALSE</span>, <span class="at">color =</span> <span class="st">&quot;green&quot;</span>, <span class="at">formula =</span> y <span class="sc">~</span> x) <span class="sc">+</span></span>
+<span id="cb1078-4"><a href="transformations.html#cb1078-4" tabindex="-1"></a>  <span class="fu">stat_smooth</span>(<span class="at">method =</span> <span class="st">&quot;lm&quot;</span>, <span class="at">se =</span> <span class="cn">FALSE</span>, <span class="at">color =</span> <span class="st">&quot;blue&quot;</span>, <span class="at">formula =</span> y <span class="sc">~</span> x <span class="sc">+</span> <span class="fu">I</span>(x <span class="sc">^</span> <span class="dv">2</span>)) <span class="sc">+</span></span>
+<span id="cb1078-5"><a href="transformations.html#cb1078-5" tabindex="-1"></a>  <span class="fu">stat_smooth</span>(<span class="at">method =</span> <span class="st">&quot;lm&quot;</span>, <span class="at">se =</span> <span class="cn">FALSE</span>, <span class="at">color =</span> <span class="st">&quot;red&quot;</span>, <span class="at">formula =</span> y <span class="sc">~</span> x <span class="sc">+</span> <span class="fu">I</span>(x <span class="sc">^</span> <span class="dv">2</span>)<span class="sc">+</span> <span class="fu">I</span>(x <span class="sc">^</span> <span class="dv">3</span>)) <span class="sc">+</span></span>
+<span id="cb1078-6"><a href="transformations.html#cb1078-6" tabindex="-1"></a>  <span class="fu">geom_point</span>(<span class="at">colour =</span> <span class="st">&quot;black&quot;</span>, <span class="at">size =</span> <span class="dv">3</span>)</span></code></pre></div>
 <p><img src="transformations_files/figure-html/unnamed-chunk-39-1.png" width="672" style="display: block; margin: auto;" /></p>
 <p>Note we could fit a polynomial of an arbitrary order,</p>
 <p><span class="math display">\[
   Y_i = \beta_0 + \beta_1 x_i + \beta_2 x_i^2 + \cdots + \beta_{p-1}x_i^{p-1} + \epsilon_i
 \]</span></p>
 <p>However, we should be careful about over-fitting, since with a polynomial of degree one less than the number of observations, it is sometimes possible to fit a model perfectly.</p>
-<div class="sourceCode" id="cb1079"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1079-1"><a href="transformations.html#cb1079-1" aria-hidden="true" tabindex="-1"></a><span class="fu">set.seed</span>(<span class="dv">1234</span>)</span>
-<span id="cb1079-2"><a href="transformations.html#cb1079-2" aria-hidden="true" tabindex="-1"></a>x <span class="ot">=</span> <span class="fu">seq</span>(<span class="dv">0</span>, <span class="dv">10</span>)</span>
-<span id="cb1079-3"><a href="transformations.html#cb1079-3" aria-hidden="true" tabindex="-1"></a>y <span class="ot">=</span> <span class="dv">3</span> <span class="sc">+</span> x <span class="sc">+</span> <span class="dv">4</span> <span class="sc">*</span> x <span class="sc">^</span> <span class="dv">2</span> <span class="sc">+</span> <span class="fu">rnorm</span>(<span class="dv">11</span>, <span class="dv">0</span>, <span class="dv">20</span>)</span>
-<span id="cb1079-4"><a href="transformations.html#cb1079-4" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(x, y, <span class="at">ylim =</span> <span class="fu">c</span>(<span class="sc">-</span><span class="dv">300</span>, <span class="dv">400</span>), <span class="at">cex =</span> <span class="dv">2</span>, <span class="at">pch =</span> <span class="dv">20</span>)</span>
-<span id="cb1079-5"><a href="transformations.html#cb1079-5" aria-hidden="true" tabindex="-1"></a>fit <span class="ot">=</span> <span class="fu">lm</span>(y <span class="sc">~</span> x <span class="sc">+</span> <span class="fu">I</span>(x <span class="sc">^</span> <span class="dv">2</span>))</span>
-<span id="cb1079-6"><a href="transformations.html#cb1079-6" aria-hidden="true" tabindex="-1"></a><span class="co">#summary(fit)</span></span>
-<span id="cb1079-7"><a href="transformations.html#cb1079-7" aria-hidden="true" tabindex="-1"></a>fit_perf <span class="ot">=</span> <span class="fu">lm</span>(y <span class="sc">~</span> x <span class="sc">+</span> <span class="fu">I</span>(x <span class="sc">^</span> <span class="dv">2</span>) <span class="sc">+</span> <span class="fu">I</span>(x <span class="sc">^</span> <span class="dv">3</span>) <span class="sc">+</span> <span class="fu">I</span>(x <span class="sc">^</span> <span class="dv">4</span>) <span class="sc">+</span> <span class="fu">I</span>(x <span class="sc">^</span> <span class="dv">5</span>) <span class="sc">+</span> <span class="fu">I</span>(x <span class="sc">^</span> <span class="dv">6</span>)</span>
-<span id="cb1079-8"><a href="transformations.html#cb1079-8" aria-hidden="true" tabindex="-1"></a>               <span class="sc">+</span> <span class="fu">I</span>(x <span class="sc">^</span> <span class="dv">7</span>) <span class="sc">+</span> <span class="fu">I</span>(x <span class="sc">^</span> <span class="dv">8</span>) <span class="sc">+</span> <span class="fu">I</span>(x <span class="sc">^</span> <span class="dv">9</span>) <span class="sc">+</span> <span class="fu">I</span>(x <span class="sc">^</span> <span class="dv">10</span>))</span>
-<span id="cb1079-9"><a href="transformations.html#cb1079-9" aria-hidden="true" tabindex="-1"></a><span class="fu">summary</span>(fit_perf)</span></code></pre></div>
+<div class="sourceCode" id="cb1079"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1079-1"><a href="transformations.html#cb1079-1" tabindex="-1"></a><span class="fu">set.seed</span>(<span class="dv">1234</span>)</span>
+<span id="cb1079-2"><a href="transformations.html#cb1079-2" tabindex="-1"></a>x <span class="ot">=</span> <span class="fu">seq</span>(<span class="dv">0</span>, <span class="dv">10</span>)</span>
+<span id="cb1079-3"><a href="transformations.html#cb1079-3" tabindex="-1"></a>y <span class="ot">=</span> <span class="dv">3</span> <span class="sc">+</span> x <span class="sc">+</span> <span class="dv">4</span> <span class="sc">*</span> x <span class="sc">^</span> <span class="dv">2</span> <span class="sc">+</span> <span class="fu">rnorm</span>(<span class="dv">11</span>, <span class="dv">0</span>, <span class="dv">20</span>)</span>
+<span id="cb1079-4"><a href="transformations.html#cb1079-4" tabindex="-1"></a><span class="fu">plot</span>(x, y, <span class="at">ylim =</span> <span class="fu">c</span>(<span class="sc">-</span><span class="dv">300</span>, <span class="dv">400</span>), <span class="at">cex =</span> <span class="dv">2</span>, <span class="at">pch =</span> <span class="dv">20</span>)</span>
+<span id="cb1079-5"><a href="transformations.html#cb1079-5" tabindex="-1"></a>fit <span class="ot">=</span> <span class="fu">lm</span>(y <span class="sc">~</span> x <span class="sc">+</span> <span class="fu">I</span>(x <span class="sc">^</span> <span class="dv">2</span>))</span>
+<span id="cb1079-6"><a href="transformations.html#cb1079-6" tabindex="-1"></a><span class="co">#summary(fit)</span></span>
+<span id="cb1079-7"><a href="transformations.html#cb1079-7" tabindex="-1"></a>fit_perf <span class="ot">=</span> <span class="fu">lm</span>(y <span class="sc">~</span> x <span class="sc">+</span> <span class="fu">I</span>(x <span class="sc">^</span> <span class="dv">2</span>) <span class="sc">+</span> <span class="fu">I</span>(x <span class="sc">^</span> <span class="dv">3</span>) <span class="sc">+</span> <span class="fu">I</span>(x <span class="sc">^</span> <span class="dv">4</span>) <span class="sc">+</span> <span class="fu">I</span>(x <span class="sc">^</span> <span class="dv">5</span>) <span class="sc">+</span> <span class="fu">I</span>(x <span class="sc">^</span> <span class="dv">6</span>)</span>
+<span id="cb1079-8"><a href="transformations.html#cb1079-8" tabindex="-1"></a>               <span class="sc">+</span> <span class="fu">I</span>(x <span class="sc">^</span> <span class="dv">7</span>) <span class="sc">+</span> <span class="fu">I</span>(x <span class="sc">^</span> <span class="dv">8</span>) <span class="sc">+</span> <span class="fu">I</span>(x <span class="sc">^</span> <span class="dv">9</span>) <span class="sc">+</span> <span class="fu">I</span>(x <span class="sc">^</span> <span class="dv">10</span>))</span>
+<span id="cb1079-9"><a href="transformations.html#cb1079-9" tabindex="-1"></a><span class="fu">summary</span>(fit_perf)</span></code></pre></div>
 <pre><code>## 
 ## Call:
 ## lm(formula = y ~ x + I(x^2) + I(x^3) + I(x^4) + I(x^5) + I(x^6) + 
@@ -936,39 +936,39 @@ <h3><span class="header-section-number">14.2.1</span> Polynomials<a href="transf
 ## Residual standard error: NaN on 0 degrees of freedom
 ## Multiple R-squared:      1,  Adjusted R-squared:    NaN 
 ## F-statistic:   NaN on 10 and 0 DF,  p-value: NA</code></pre>
-<div class="sourceCode" id="cb1081"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1081-1"><a href="transformations.html#cb1081-1" aria-hidden="true" tabindex="-1"></a>xplot <span class="ot">=</span> <span class="fu">seq</span>(<span class="dv">0</span>, <span class="dv">10</span>, <span class="at">by =</span> <span class="fl">0.1</span>)</span>
-<span id="cb1081-2"><a href="transformations.html#cb1081-2" aria-hidden="true" tabindex="-1"></a><span class="fu">lines</span>(xplot, <span class="fu">predict</span>(fit, <span class="at">newdata =</span> <span class="fu">data.frame</span>(<span class="at">x =</span> xplot)),</span>
-<span id="cb1081-3"><a href="transformations.html#cb1081-3" aria-hidden="true" tabindex="-1"></a>      <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>, <span class="at">lty =</span> <span class="dv">1</span>)</span>
-<span id="cb1081-4"><a href="transformations.html#cb1081-4" aria-hidden="true" tabindex="-1"></a><span class="fu">lines</span>(xplot, <span class="fu">predict</span>(fit_perf, <span class="at">newdata =</span> <span class="fu">data.frame</span>(<span class="at">x =</span> xplot)),</span>
-<span id="cb1081-5"><a href="transformations.html#cb1081-5" aria-hidden="true" tabindex="-1"></a>      <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>, <span class="at">lty =</span> <span class="dv">2</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1081"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1081-1"><a href="transformations.html#cb1081-1" tabindex="-1"></a>xplot <span class="ot">=</span> <span class="fu">seq</span>(<span class="dv">0</span>, <span class="dv">10</span>, <span class="at">by =</span> <span class="fl">0.1</span>)</span>
+<span id="cb1081-2"><a href="transformations.html#cb1081-2" tabindex="-1"></a><span class="fu">lines</span>(xplot, <span class="fu">predict</span>(fit, <span class="at">newdata =</span> <span class="fu">data.frame</span>(<span class="at">x =</span> xplot)),</span>
+<span id="cb1081-3"><a href="transformations.html#cb1081-3" tabindex="-1"></a>      <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>, <span class="at">lty =</span> <span class="dv">1</span>)</span>
+<span id="cb1081-4"><a href="transformations.html#cb1081-4" tabindex="-1"></a><span class="fu">lines</span>(xplot, <span class="fu">predict</span>(fit_perf, <span class="at">newdata =</span> <span class="fu">data.frame</span>(<span class="at">x =</span> xplot)),</span>
+<span id="cb1081-5"><a href="transformations.html#cb1081-5" tabindex="-1"></a>      <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>, <span class="at">lty =</span> <span class="dv">2</span>)</span></code></pre></div>
 <p><img src="transformations_files/figure-html/unnamed-chunk-40-1.png" width="672" style="display: block; margin: auto;" /></p>
 <p>Notice in the summary, <code>R</code> could not calculate standard errors. This is a result of being “out” of degrees of freedom. With 11 <span class="math inline">\(\beta\)</span> parameters and 11 data points, we use up all the degrees of freedom before we can estimate <span class="math inline">\(\sigma\)</span>.</p>
 <p>In this example, the true relationship is quadratic, but the order 10 polynomial’s fit is “perfect”. Next chapter we will focus on the trade-off between goodness of fit (minimizing errors) and complexity of model.</p>
 <p>Suppose you work for an automobile manufacturer which makes a large luxury sedan. You would like to know how the car performs from a fuel efficiency standpoint when it is driven at various speeds. Instead of testing the car at every conceivable speed (which would be impossible) you create an experiment where the car is driven at speeds of interest in increments of 5 miles per hour.</p>
 <p>Our goal then, is to fit a model to this data in order to be able to predict fuel efficiency when driving at certain speeds. The data from this example can be found in <a href="data/fuel_econ.csv"><code>fuel_econ.csv</code></a>.</p>
-<div class="sourceCode" id="cb1082"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1082-1"><a href="transformations.html#cb1082-1" aria-hidden="true" tabindex="-1"></a>econ <span class="ot">=</span> <span class="fu">read.csv</span>(<span class="st">&quot;data/fuel_econ.csv&quot;</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1082"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1082-1"><a href="transformations.html#cb1082-1" tabindex="-1"></a>econ <span class="ot">=</span> <span class="fu">read.csv</span>(<span class="st">&quot;data/fuel_econ.csv&quot;</span>)</span></code></pre></div>
 <p>In this example, we will be frequently looking at the fitted versus residuals plot, so we <em>should</em> write a function to make our life easier, but this is left as an exercise for homework.</p>
 <p>We will also be adding fitted curves to scatterplots repeatedly, so smartly we will write a function to do so.</p>
-<div class="sourceCode" id="cb1083"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1083-1"><a href="transformations.html#cb1083-1" aria-hidden="true" tabindex="-1"></a>plot_econ_curve <span class="ot">=</span> <span class="cf">function</span>(model) {</span>
-<span id="cb1083-2"><a href="transformations.html#cb1083-2" aria-hidden="true" tabindex="-1"></a>  <span class="fu">plot</span>(mpg <span class="sc">~</span> mph, <span class="at">data =</span> econ, <span class="at">xlab =</span> <span class="st">&quot;Speed (Miles per Hour)&quot;</span>, </span>
-<span id="cb1083-3"><a href="transformations.html#cb1083-3" aria-hidden="true" tabindex="-1"></a>       <span class="at">ylab =</span> <span class="st">&quot;Fuel Efficiency (Miles per Gallon)&quot;</span>, <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, </span>
-<span id="cb1083-4"><a href="transformations.html#cb1083-4" aria-hidden="true" tabindex="-1"></a>       <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">cex =</span> <span class="dv">2</span>)</span>
-<span id="cb1083-5"><a href="transformations.html#cb1083-5" aria-hidden="true" tabindex="-1"></a>  xplot <span class="ot">=</span> <span class="fu">seq</span>(<span class="dv">10</span>, <span class="dv">75</span>, <span class="at">by =</span> <span class="fl">0.1</span>)</span>
-<span id="cb1083-6"><a href="transformations.html#cb1083-6" aria-hidden="true" tabindex="-1"></a>  <span class="fu">lines</span>(xplot, <span class="fu">predict</span>(model, <span class="at">newdata =</span> <span class="fu">data.frame</span>(<span class="at">mph =</span> xplot)),</span>
-<span id="cb1083-7"><a href="transformations.html#cb1083-7" aria-hidden="true" tabindex="-1"></a>        <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>, <span class="at">lty =</span> <span class="dv">1</span>)</span>
-<span id="cb1083-8"><a href="transformations.html#cb1083-8" aria-hidden="true" tabindex="-1"></a>}</span></code></pre></div>
+<div class="sourceCode" id="cb1083"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1083-1"><a href="transformations.html#cb1083-1" tabindex="-1"></a>plot_econ_curve <span class="ot">=</span> <span class="cf">function</span>(model) {</span>
+<span id="cb1083-2"><a href="transformations.html#cb1083-2" tabindex="-1"></a>  <span class="fu">plot</span>(mpg <span class="sc">~</span> mph, <span class="at">data =</span> econ, <span class="at">xlab =</span> <span class="st">&quot;Speed (Miles per Hour)&quot;</span>, </span>
+<span id="cb1083-3"><a href="transformations.html#cb1083-3" tabindex="-1"></a>       <span class="at">ylab =</span> <span class="st">&quot;Fuel Efficiency (Miles per Gallon)&quot;</span>, <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, </span>
+<span id="cb1083-4"><a href="transformations.html#cb1083-4" tabindex="-1"></a>       <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">cex =</span> <span class="dv">2</span>)</span>
+<span id="cb1083-5"><a href="transformations.html#cb1083-5" tabindex="-1"></a>  xplot <span class="ot">=</span> <span class="fu">seq</span>(<span class="dv">10</span>, <span class="dv">75</span>, <span class="at">by =</span> <span class="fl">0.1</span>)</span>
+<span id="cb1083-6"><a href="transformations.html#cb1083-6" tabindex="-1"></a>  <span class="fu">lines</span>(xplot, <span class="fu">predict</span>(model, <span class="at">newdata =</span> <span class="fu">data.frame</span>(<span class="at">mph =</span> xplot)),</span>
+<span id="cb1083-7"><a href="transformations.html#cb1083-7" tabindex="-1"></a>        <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>, <span class="at">lty =</span> <span class="dv">1</span>)</span>
+<span id="cb1083-8"><a href="transformations.html#cb1083-8" tabindex="-1"></a>}</span></code></pre></div>
 <p>So now we first fit a simple linear regression to this data.</p>
-<div class="sourceCode" id="cb1084"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1084-1"><a href="transformations.html#cb1084-1" aria-hidden="true" tabindex="-1"></a>fit1 <span class="ot">=</span> <span class="fu">lm</span>(mpg <span class="sc">~</span> mph, <span class="at">data =</span> econ)</span></code></pre></div>
-<div class="sourceCode" id="cb1085"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1085-1"><a href="transformations.html#cb1085-1" aria-hidden="true" tabindex="-1"></a><span class="fu">par</span>(<span class="at">mfrow =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>))</span>
-<span id="cb1085-2"><a href="transformations.html#cb1085-2" aria-hidden="true" tabindex="-1"></a><span class="fu">plot_econ_curve</span>(fit1)</span>
-<span id="cb1085-3"><a href="transformations.html#cb1085-3" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(<span class="fu">fitted</span>(fit1), <span class="fu">resid</span>(fit1), <span class="at">xlab =</span> <span class="st">&quot;Fitted&quot;</span>, <span class="at">ylab =</span> <span class="st">&quot;Residuals&quot;</span>, </span>
-<span id="cb1085-4"><a href="transformations.html#cb1085-4" aria-hidden="true" tabindex="-1"></a>     <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">cex =</span> <span class="dv">2</span>)</span>
-<span id="cb1085-5"><a href="transformations.html#cb1085-5" aria-hidden="true" tabindex="-1"></a>  <span class="fu">abline</span>(<span class="at">h =</span> <span class="dv">0</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1084"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1084-1"><a href="transformations.html#cb1084-1" tabindex="-1"></a>fit1 <span class="ot">=</span> <span class="fu">lm</span>(mpg <span class="sc">~</span> mph, <span class="at">data =</span> econ)</span></code></pre></div>
+<div class="sourceCode" id="cb1085"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1085-1"><a href="transformations.html#cb1085-1" tabindex="-1"></a><span class="fu">par</span>(<span class="at">mfrow =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>))</span>
+<span id="cb1085-2"><a href="transformations.html#cb1085-2" tabindex="-1"></a><span class="fu">plot_econ_curve</span>(fit1)</span>
+<span id="cb1085-3"><a href="transformations.html#cb1085-3" tabindex="-1"></a><span class="fu">plot</span>(<span class="fu">fitted</span>(fit1), <span class="fu">resid</span>(fit1), <span class="at">xlab =</span> <span class="st">&quot;Fitted&quot;</span>, <span class="at">ylab =</span> <span class="st">&quot;Residuals&quot;</span>, </span>
+<span id="cb1085-4"><a href="transformations.html#cb1085-4" tabindex="-1"></a>     <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">cex =</span> <span class="dv">2</span>)</span>
+<span id="cb1085-5"><a href="transformations.html#cb1085-5" tabindex="-1"></a>  <span class="fu">abline</span>(<span class="at">h =</span> <span class="dv">0</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
 <p><img src="transformations_files/figure-html/unnamed-chunk-44-1.png" width="768" style="display: block; margin: auto;" /></p>
 <p>Pretty clearly we can do better. Yes fuel efficiency does increase as speed increases, but only up to a certain point.</p>
 <p>We will now add polynomial terms until we fit a suitable fit.</p>
-<div class="sourceCode" id="cb1086"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1086-1"><a href="transformations.html#cb1086-1" aria-hidden="true" tabindex="-1"></a>fit2 <span class="ot">=</span> <span class="fu">lm</span>(mpg <span class="sc">~</span> mph <span class="sc">+</span> <span class="fu">I</span>(mph <span class="sc">^</span> <span class="dv">2</span>), <span class="at">data =</span> econ)</span>
-<span id="cb1086-2"><a href="transformations.html#cb1086-2" aria-hidden="true" tabindex="-1"></a><span class="fu">summary</span>(fit2)</span></code></pre></div>
+<div class="sourceCode" id="cb1086"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1086-1"><a href="transformations.html#cb1086-1" tabindex="-1"></a>fit2 <span class="ot">=</span> <span class="fu">lm</span>(mpg <span class="sc">~</span> mph <span class="sc">+</span> <span class="fu">I</span>(mph <span class="sc">^</span> <span class="dv">2</span>), <span class="at">data =</span> econ)</span>
+<span id="cb1086-2"><a href="transformations.html#cb1086-2" tabindex="-1"></a><span class="fu">summary</span>(fit2)</span></code></pre></div>
 <pre><code>## 
 ## Call:
 ## lm(formula = mpg ~ mph + I(mph^2), data = econ)
@@ -988,15 +988,15 @@ <h3><span class="header-section-number">14.2.1</span> Polynomials<a href="transf
 ## Residual standard error: 1.663 on 25 degrees of freedom
 ## Multiple R-squared:  0.9188, Adjusted R-squared:  0.9123 
 ## F-statistic: 141.5 on 2 and 25 DF,  p-value: 2.338e-14</code></pre>
-<div class="sourceCode" id="cb1088"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1088-1"><a href="transformations.html#cb1088-1" aria-hidden="true" tabindex="-1"></a><span class="fu">par</span>(<span class="at">mfrow =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>))</span>
-<span id="cb1088-2"><a href="transformations.html#cb1088-2" aria-hidden="true" tabindex="-1"></a><span class="fu">plot_econ_curve</span>(fit2)</span>
-<span id="cb1088-3"><a href="transformations.html#cb1088-3" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(<span class="fu">fitted</span>(fit2), <span class="fu">resid</span>(fit2), <span class="at">xlab =</span> <span class="st">&quot;Fitted&quot;</span>, <span class="at">ylab =</span> <span class="st">&quot;Residuals&quot;</span>, </span>
-<span id="cb1088-4"><a href="transformations.html#cb1088-4" aria-hidden="true" tabindex="-1"></a>     <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">cex =</span> <span class="dv">2</span>)</span>
-<span id="cb1088-5"><a href="transformations.html#cb1088-5" aria-hidden="true" tabindex="-1"></a>  <span class="fu">abline</span>(<span class="at">h =</span> <span class="dv">0</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1088"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1088-1"><a href="transformations.html#cb1088-1" tabindex="-1"></a><span class="fu">par</span>(<span class="at">mfrow =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>))</span>
+<span id="cb1088-2"><a href="transformations.html#cb1088-2" tabindex="-1"></a><span class="fu">plot_econ_curve</span>(fit2)</span>
+<span id="cb1088-3"><a href="transformations.html#cb1088-3" tabindex="-1"></a><span class="fu">plot</span>(<span class="fu">fitted</span>(fit2), <span class="fu">resid</span>(fit2), <span class="at">xlab =</span> <span class="st">&quot;Fitted&quot;</span>, <span class="at">ylab =</span> <span class="st">&quot;Residuals&quot;</span>, </span>
+<span id="cb1088-4"><a href="transformations.html#cb1088-4" tabindex="-1"></a>     <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">cex =</span> <span class="dv">2</span>)</span>
+<span id="cb1088-5"><a href="transformations.html#cb1088-5" tabindex="-1"></a>  <span class="fu">abline</span>(<span class="at">h =</span> <span class="dv">0</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
 <p><img src="transformations_files/figure-html/unnamed-chunk-46-1.png" width="768" style="display: block; margin: auto;" /></p>
 <p>While this model clearly fits much better, and the second order term is significant, we still see a pattern in the fitted versus residuals plot which suggests higher order terms will help. Also, we would expect the curve to flatten as speed increases or decreases, not go sharply downward as we see here.</p>
-<div class="sourceCode" id="cb1089"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1089-1"><a href="transformations.html#cb1089-1" aria-hidden="true" tabindex="-1"></a>fit3 <span class="ot">=</span> <span class="fu">lm</span>(mpg <span class="sc">~</span> mph <span class="sc">+</span> <span class="fu">I</span>(mph <span class="sc">^</span> <span class="dv">2</span>) <span class="sc">+</span> <span class="fu">I</span>(mph <span class="sc">^</span> <span class="dv">3</span>), <span class="at">data =</span> econ)</span>
-<span id="cb1089-2"><a href="transformations.html#cb1089-2" aria-hidden="true" tabindex="-1"></a><span class="fu">summary</span>(fit3)</span></code></pre></div>
+<div class="sourceCode" id="cb1089"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1089-1"><a href="transformations.html#cb1089-1" tabindex="-1"></a>fit3 <span class="ot">=</span> <span class="fu">lm</span>(mpg <span class="sc">~</span> mph <span class="sc">+</span> <span class="fu">I</span>(mph <span class="sc">^</span> <span class="dv">2</span>) <span class="sc">+</span> <span class="fu">I</span>(mph <span class="sc">^</span> <span class="dv">3</span>), <span class="at">data =</span> econ)</span>
+<span id="cb1089-2"><a href="transformations.html#cb1089-2" tabindex="-1"></a><span class="fu">summary</span>(fit3)</span></code></pre></div>
 <pre><code>## 
 ## Call:
 ## lm(formula = mpg ~ mph + I(mph^2) + I(mph^3), data = econ)
@@ -1017,15 +1017,15 @@ <h3><span class="header-section-number">14.2.1</span> Polynomials<a href="transf
 ## Residual standard error: 1.697 on 24 degrees of freedom
 ## Multiple R-squared:  0.9188, Adjusted R-squared:  0.9087 
 ## F-statistic: 90.56 on 3 and 24 DF,  p-value: 3.17e-13</code></pre>
-<div class="sourceCode" id="cb1091"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1091-1"><a href="transformations.html#cb1091-1" aria-hidden="true" tabindex="-1"></a><span class="fu">par</span>(<span class="at">mfrow =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>))</span>
-<span id="cb1091-2"><a href="transformations.html#cb1091-2" aria-hidden="true" tabindex="-1"></a><span class="fu">plot_econ_curve</span>(fit3)</span>
-<span id="cb1091-3"><a href="transformations.html#cb1091-3" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(<span class="fu">fitted</span>(fit3), <span class="fu">resid</span>(fit3), <span class="at">xlab =</span> <span class="st">&quot;Fitted&quot;</span>, <span class="at">ylab =</span> <span class="st">&quot;Residuals&quot;</span>, </span>
-<span id="cb1091-4"><a href="transformations.html#cb1091-4" aria-hidden="true" tabindex="-1"></a>     <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">cex =</span> <span class="dv">2</span>)</span>
-<span id="cb1091-5"><a href="transformations.html#cb1091-5" aria-hidden="true" tabindex="-1"></a>  <span class="fu">abline</span>(<span class="at">h =</span> <span class="dv">0</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1091"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1091-1"><a href="transformations.html#cb1091-1" tabindex="-1"></a><span class="fu">par</span>(<span class="at">mfrow =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>))</span>
+<span id="cb1091-2"><a href="transformations.html#cb1091-2" tabindex="-1"></a><span class="fu">plot_econ_curve</span>(fit3)</span>
+<span id="cb1091-3"><a href="transformations.html#cb1091-3" tabindex="-1"></a><span class="fu">plot</span>(<span class="fu">fitted</span>(fit3), <span class="fu">resid</span>(fit3), <span class="at">xlab =</span> <span class="st">&quot;Fitted&quot;</span>, <span class="at">ylab =</span> <span class="st">&quot;Residuals&quot;</span>, </span>
+<span id="cb1091-4"><a href="transformations.html#cb1091-4" tabindex="-1"></a>     <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">cex =</span> <span class="dv">2</span>)</span>
+<span id="cb1091-5"><a href="transformations.html#cb1091-5" tabindex="-1"></a>  <span class="fu">abline</span>(<span class="at">h =</span> <span class="dv">0</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
 <p><img src="transformations_files/figure-html/unnamed-chunk-48-1.png" width="768" style="display: block; margin: auto;" /></p>
 <p>Adding the third order term doesn’t seem to help at all. The fitted curve hardly changes. This makes sense, since what we would like is for the curve to flatten at the extremes. For this we will need an even degree polynomial term.</p>
-<div class="sourceCode" id="cb1092"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1092-1"><a href="transformations.html#cb1092-1" aria-hidden="true" tabindex="-1"></a>fit4 <span class="ot">=</span> <span class="fu">lm</span>(mpg <span class="sc">~</span> mph <span class="sc">+</span> <span class="fu">I</span>(mph <span class="sc">^</span> <span class="dv">2</span>) <span class="sc">+</span> <span class="fu">I</span>(mph <span class="sc">^</span> <span class="dv">3</span>) <span class="sc">+</span> <span class="fu">I</span>(mph <span class="sc">^</span> <span class="dv">4</span>), <span class="at">data =</span> econ)</span>
-<span id="cb1092-2"><a href="transformations.html#cb1092-2" aria-hidden="true" tabindex="-1"></a><span class="fu">summary</span>(fit4)</span></code></pre></div>
+<div class="sourceCode" id="cb1092"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1092-1"><a href="transformations.html#cb1092-1" tabindex="-1"></a>fit4 <span class="ot">=</span> <span class="fu">lm</span>(mpg <span class="sc">~</span> mph <span class="sc">+</span> <span class="fu">I</span>(mph <span class="sc">^</span> <span class="dv">2</span>) <span class="sc">+</span> <span class="fu">I</span>(mph <span class="sc">^</span> <span class="dv">3</span>) <span class="sc">+</span> <span class="fu">I</span>(mph <span class="sc">^</span> <span class="dv">4</span>), <span class="at">data =</span> econ)</span>
+<span id="cb1092-2"><a href="transformations.html#cb1092-2" tabindex="-1"></a><span class="fu">summary</span>(fit4)</span></code></pre></div>
 <pre><code>## 
 ## Call:
 ## lm(formula = mpg ~ mph + I(mph^2) + I(mph^3) + I(mph^4), data = econ)
@@ -1047,15 +1047,15 @@ <h3><span class="header-section-number">14.2.1</span> Polynomials<a href="transf
 ## Residual standard error: 0.9307 on 23 degrees of freedom
 ## Multiple R-squared:  0.9766, Adjusted R-squared:  0.9726 
 ## F-statistic: 240.2 on 4 and 23 DF,  p-value: &lt; 2.2e-16</code></pre>
-<div class="sourceCode" id="cb1094"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1094-1"><a href="transformations.html#cb1094-1" aria-hidden="true" tabindex="-1"></a><span class="fu">par</span>(<span class="at">mfrow =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>))</span>
-<span id="cb1094-2"><a href="transformations.html#cb1094-2" aria-hidden="true" tabindex="-1"></a><span class="fu">plot_econ_curve</span>(fit4)</span>
-<span id="cb1094-3"><a href="transformations.html#cb1094-3" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(<span class="fu">fitted</span>(fit4), <span class="fu">resid</span>(fit4), <span class="at">xlab =</span> <span class="st">&quot;Fitted&quot;</span>, <span class="at">ylab =</span> <span class="st">&quot;Residuals&quot;</span>, </span>
-<span id="cb1094-4"><a href="transformations.html#cb1094-4" aria-hidden="true" tabindex="-1"></a>     <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">cex =</span> <span class="dv">2</span>)</span>
-<span id="cb1094-5"><a href="transformations.html#cb1094-5" aria-hidden="true" tabindex="-1"></a>  <span class="fu">abline</span>(<span class="at">h =</span> <span class="dv">0</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1094"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1094-1"><a href="transformations.html#cb1094-1" tabindex="-1"></a><span class="fu">par</span>(<span class="at">mfrow =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>))</span>
+<span id="cb1094-2"><a href="transformations.html#cb1094-2" tabindex="-1"></a><span class="fu">plot_econ_curve</span>(fit4)</span>
+<span id="cb1094-3"><a href="transformations.html#cb1094-3" tabindex="-1"></a><span class="fu">plot</span>(<span class="fu">fitted</span>(fit4), <span class="fu">resid</span>(fit4), <span class="at">xlab =</span> <span class="st">&quot;Fitted&quot;</span>, <span class="at">ylab =</span> <span class="st">&quot;Residuals&quot;</span>, </span>
+<span id="cb1094-4"><a href="transformations.html#cb1094-4" tabindex="-1"></a>     <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">cex =</span> <span class="dv">2</span>)</span>
+<span id="cb1094-5"><a href="transformations.html#cb1094-5" tabindex="-1"></a>  <span class="fu">abline</span>(<span class="at">h =</span> <span class="dv">0</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
 <p><img src="transformations_files/figure-html/unnamed-chunk-50-1.png" width="768" style="display: block; margin: auto;" /></p>
 <p>Now we are making progress. The fourth order term is significant with the other terms in the model. Also we are starting to see what we expected for low and high speed. However, there still seems to be a bit of a pattern in the residuals, so we will again try more higher order terms. We will add the fifth and sixth together, since adding the fifth will be similar to adding the third.</p>
-<div class="sourceCode" id="cb1095"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1095-1"><a href="transformations.html#cb1095-1" aria-hidden="true" tabindex="-1"></a>fit6 <span class="ot">=</span> <span class="fu">lm</span>(mpg <span class="sc">~</span> mph <span class="sc">+</span> <span class="fu">I</span>(mph <span class="sc">^</span> <span class="dv">2</span>) <span class="sc">+</span> <span class="fu">I</span>(mph <span class="sc">^</span> <span class="dv">3</span>) <span class="sc">+</span> <span class="fu">I</span>(mph <span class="sc">^</span> <span class="dv">4</span>) <span class="sc">+</span> <span class="fu">I</span>(mph <span class="sc">^</span> <span class="dv">5</span>) <span class="sc">+</span> <span class="fu">I</span>(mph<span class="sc">^</span><span class="dv">6</span>), <span class="at">data =</span> econ)</span>
-<span id="cb1095-2"><a href="transformations.html#cb1095-2" aria-hidden="true" tabindex="-1"></a><span class="fu">summary</span>(fit6)</span></code></pre></div>
+<div class="sourceCode" id="cb1095"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1095-1"><a href="transformations.html#cb1095-1" tabindex="-1"></a>fit6 <span class="ot">=</span> <span class="fu">lm</span>(mpg <span class="sc">~</span> mph <span class="sc">+</span> <span class="fu">I</span>(mph <span class="sc">^</span> <span class="dv">2</span>) <span class="sc">+</span> <span class="fu">I</span>(mph <span class="sc">^</span> <span class="dv">3</span>) <span class="sc">+</span> <span class="fu">I</span>(mph <span class="sc">^</span> <span class="dv">4</span>) <span class="sc">+</span> <span class="fu">I</span>(mph <span class="sc">^</span> <span class="dv">5</span>) <span class="sc">+</span> <span class="fu">I</span>(mph<span class="sc">^</span><span class="dv">6</span>), <span class="at">data =</span> econ)</span>
+<span id="cb1095-2"><a href="transformations.html#cb1095-2" tabindex="-1"></a><span class="fu">summary</span>(fit6)</span></code></pre></div>
 <pre><code>## 
 ## Call:
 ## lm(formula = mpg ~ mph + I(mph^2) + I(mph^3) + I(mph^4) + I(mph^5) + 
@@ -1080,17 +1080,17 @@ <h3><span class="header-section-number">14.2.1</span> Polynomials<a href="transf
 ## Residual standard error: 0.8657 on 21 degrees of freedom
 ## Multiple R-squared:  0.9815, Adjusted R-squared:  0.9762 
 ## F-statistic:   186 on 6 and 21 DF,  p-value: &lt; 2.2e-16</code></pre>
-<div class="sourceCode" id="cb1097"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1097-1"><a href="transformations.html#cb1097-1" aria-hidden="true" tabindex="-1"></a><span class="fu">par</span>(<span class="at">mfrow =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>))</span>
-<span id="cb1097-2"><a href="transformations.html#cb1097-2" aria-hidden="true" tabindex="-1"></a><span class="fu">plot_econ_curve</span>(fit6)</span>
-<span id="cb1097-3"><a href="transformations.html#cb1097-3" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(<span class="fu">fitted</span>(fit6), <span class="fu">resid</span>(fit6), <span class="at">xlab =</span> <span class="st">&quot;Fitted&quot;</span>, <span class="at">ylab =</span> <span class="st">&quot;Residuals&quot;</span>, </span>
-<span id="cb1097-4"><a href="transformations.html#cb1097-4" aria-hidden="true" tabindex="-1"></a>     <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">cex =</span> <span class="dv">2</span>)</span>
-<span id="cb1097-5"><a href="transformations.html#cb1097-5" aria-hidden="true" tabindex="-1"></a>  <span class="fu">abline</span>(<span class="at">h =</span> <span class="dv">0</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1097"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1097-1"><a href="transformations.html#cb1097-1" tabindex="-1"></a><span class="fu">par</span>(<span class="at">mfrow =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>))</span>
+<span id="cb1097-2"><a href="transformations.html#cb1097-2" tabindex="-1"></a><span class="fu">plot_econ_curve</span>(fit6)</span>
+<span id="cb1097-3"><a href="transformations.html#cb1097-3" tabindex="-1"></a><span class="fu">plot</span>(<span class="fu">fitted</span>(fit6), <span class="fu">resid</span>(fit6), <span class="at">xlab =</span> <span class="st">&quot;Fitted&quot;</span>, <span class="at">ylab =</span> <span class="st">&quot;Residuals&quot;</span>, </span>
+<span id="cb1097-4"><a href="transformations.html#cb1097-4" tabindex="-1"></a>     <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">cex =</span> <span class="dv">2</span>)</span>
+<span id="cb1097-5"><a href="transformations.html#cb1097-5" tabindex="-1"></a>  <span class="fu">abline</span>(<span class="at">h =</span> <span class="dv">0</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
 <p><img src="transformations_files/figure-html/unnamed-chunk-52-1.png" width="768" style="display: block; margin: auto;" /></p>
 <p>Again the sixth order term is significant with the other terms in the model and here we see less pattern in the residuals plot. Let’s now test which of the previous two models we prefer. We will test</p>
 <p><span class="math display">\[
 H_0: \beta_5 = \beta_6 = 0.
 \]</span></p>
-<div class="sourceCode" id="cb1098"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1098-1"><a href="transformations.html#cb1098-1" aria-hidden="true" tabindex="-1"></a><span class="fu">anova</span>(fit4, fit6)</span></code></pre></div>
+<div class="sourceCode" id="cb1098"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1098-1"><a href="transformations.html#cb1098-1" tabindex="-1"></a><span class="fu">anova</span>(fit4, fit6)</span></code></pre></div>
 <pre><code>## Analysis of Variance Table
 ## 
 ## Model 1: mpg ~ mph + I(mph^2) + I(mph^3) + I(mph^4)
@@ -1101,9 +1101,9 @@ <h3><span class="header-section-number">14.2.1</span> Polynomials<a href="transf
 ## ---
 ## Signif. codes:  0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1</code></pre>
 <p>So, this test does not reject the null hypothesis at a level of significance of <span class="math inline">\(\alpha = 0.05\)</span>, however the p-value is still rather small, and the fitted versus residuals plot is much better for the model with the sixth order term. This makes the sixth order model a good choice. We could repeat this process one more time.</p>
-<div class="sourceCode" id="cb1100"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1100-1"><a href="transformations.html#cb1100-1" aria-hidden="true" tabindex="-1"></a>fit8 <span class="ot">=</span> <span class="fu">lm</span>(mpg <span class="sc">~</span> mph <span class="sc">+</span> <span class="fu">I</span>(mph <span class="sc">^</span> <span class="dv">2</span>) <span class="sc">+</span> <span class="fu">I</span>(mph <span class="sc">^</span> <span class="dv">3</span>) <span class="sc">+</span> <span class="fu">I</span>(mph <span class="sc">^</span> <span class="dv">4</span>) <span class="sc">+</span> <span class="fu">I</span>(mph <span class="sc">^</span> <span class="dv">5</span>)</span>
-<span id="cb1100-2"><a href="transformations.html#cb1100-2" aria-hidden="true" tabindex="-1"></a>          <span class="sc">+</span> <span class="fu">I</span>(mph <span class="sc">^</span> <span class="dv">6</span>) <span class="sc">+</span> <span class="fu">I</span>(mph <span class="sc">^</span> <span class="dv">7</span>) <span class="sc">+</span> <span class="fu">I</span>(mph <span class="sc">^</span> <span class="dv">8</span>), <span class="at">data =</span> econ)</span>
-<span id="cb1100-3"><a href="transformations.html#cb1100-3" aria-hidden="true" tabindex="-1"></a><span class="fu">summary</span>(fit8)</span></code></pre></div>
+<div class="sourceCode" id="cb1100"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1100-1"><a href="transformations.html#cb1100-1" tabindex="-1"></a>fit8 <span class="ot">=</span> <span class="fu">lm</span>(mpg <span class="sc">~</span> mph <span class="sc">+</span> <span class="fu">I</span>(mph <span class="sc">^</span> <span class="dv">2</span>) <span class="sc">+</span> <span class="fu">I</span>(mph <span class="sc">^</span> <span class="dv">3</span>) <span class="sc">+</span> <span class="fu">I</span>(mph <span class="sc">^</span> <span class="dv">4</span>) <span class="sc">+</span> <span class="fu">I</span>(mph <span class="sc">^</span> <span class="dv">5</span>)</span>
+<span id="cb1100-2"><a href="transformations.html#cb1100-2" tabindex="-1"></a>          <span class="sc">+</span> <span class="fu">I</span>(mph <span class="sc">^</span> <span class="dv">6</span>) <span class="sc">+</span> <span class="fu">I</span>(mph <span class="sc">^</span> <span class="dv">7</span>) <span class="sc">+</span> <span class="fu">I</span>(mph <span class="sc">^</span> <span class="dv">8</span>), <span class="at">data =</span> econ)</span>
+<span id="cb1100-3"><a href="transformations.html#cb1100-3" tabindex="-1"></a><span class="fu">summary</span>(fit8)</span></code></pre></div>
 <pre><code>## 
 ## Call:
 ## lm(formula = mpg ~ mph + I(mph^2) + I(mph^3) + I(mph^4) + I(mph^5) + 
@@ -1128,13 +1128,13 @@ <h3><span class="header-section-number">14.2.1</span> Polynomials<a href="transf
 ## Residual standard error: 0.9034 on 19 degrees of freedom
 ## Multiple R-squared:  0.9818, Adjusted R-squared:  0.9741 
 ## F-statistic: 128.1 on 8 and 19 DF,  p-value: 7.074e-15</code></pre>
-<div class="sourceCode" id="cb1102"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1102-1"><a href="transformations.html#cb1102-1" aria-hidden="true" tabindex="-1"></a><span class="fu">par</span>(<span class="at">mfrow =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>))</span>
-<span id="cb1102-2"><a href="transformations.html#cb1102-2" aria-hidden="true" tabindex="-1"></a><span class="fu">plot_econ_curve</span>(fit8)</span>
-<span id="cb1102-3"><a href="transformations.html#cb1102-3" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(<span class="fu">fitted</span>(fit8), <span class="fu">resid</span>(fit8), <span class="at">xlab =</span> <span class="st">&quot;Fitted&quot;</span>, <span class="at">ylab =</span> <span class="st">&quot;Residuals&quot;</span>, </span>
-<span id="cb1102-4"><a href="transformations.html#cb1102-4" aria-hidden="true" tabindex="-1"></a>     <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">cex =</span> <span class="dv">2</span>)</span>
-<span id="cb1102-5"><a href="transformations.html#cb1102-5" aria-hidden="true" tabindex="-1"></a>  <span class="fu">abline</span>(<span class="at">h =</span> <span class="dv">0</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1102"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1102-1"><a href="transformations.html#cb1102-1" tabindex="-1"></a><span class="fu">par</span>(<span class="at">mfrow =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>))</span>
+<span id="cb1102-2"><a href="transformations.html#cb1102-2" tabindex="-1"></a><span class="fu">plot_econ_curve</span>(fit8)</span>
+<span id="cb1102-3"><a href="transformations.html#cb1102-3" tabindex="-1"></a><span class="fu">plot</span>(<span class="fu">fitted</span>(fit8), <span class="fu">resid</span>(fit8), <span class="at">xlab =</span> <span class="st">&quot;Fitted&quot;</span>, <span class="at">ylab =</span> <span class="st">&quot;Residuals&quot;</span>, </span>
+<span id="cb1102-4"><a href="transformations.html#cb1102-4" tabindex="-1"></a>     <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">cex =</span> <span class="dv">2</span>)</span>
+<span id="cb1102-5"><a href="transformations.html#cb1102-5" tabindex="-1"></a>  <span class="fu">abline</span>(<span class="at">h =</span> <span class="dv">0</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
 <p><img src="transformations_files/figure-html/unnamed-chunk-55-1.png" width="768" style="display: block; margin: auto;" /></p>
-<div class="sourceCode" id="cb1103"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1103-1"><a href="transformations.html#cb1103-1" aria-hidden="true" tabindex="-1"></a><span class="fu">summary</span>(fit8)</span></code></pre></div>
+<div class="sourceCode" id="cb1103"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1103-1"><a href="transformations.html#cb1103-1" tabindex="-1"></a><span class="fu">summary</span>(fit8)</span></code></pre></div>
 <pre><code>## 
 ## Call:
 ## lm(formula = mpg ~ mph + I(mph^2) + I(mph^3) + I(mph^4) + I(mph^5) + 
@@ -1159,7 +1159,7 @@ <h3><span class="header-section-number">14.2.1</span> Polynomials<a href="transf
 ## Residual standard error: 0.9034 on 19 degrees of freedom
 ## Multiple R-squared:  0.9818, Adjusted R-squared:  0.9741 
 ## F-statistic: 128.1 on 8 and 19 DF,  p-value: 7.074e-15</code></pre>
-<div class="sourceCode" id="cb1105"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1105-1"><a href="transformations.html#cb1105-1" aria-hidden="true" tabindex="-1"></a><span class="fu">anova</span>(fit6, fit8)</span></code></pre></div>
+<div class="sourceCode" id="cb1105"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1105-1"><a href="transformations.html#cb1105-1" tabindex="-1"></a><span class="fu">anova</span>(fit6, fit8)</span></code></pre></div>
 <pre><code>## Analysis of Variance Table
 ## 
 ## Model 1: mpg ~ mph + I(mph^2) + I(mph^3) + I(mph^4) + I(mph^5) + I(mph^6)
@@ -1170,22 +1170,22 @@ <h3><span class="header-section-number">14.2.1</span> Polynomials<a href="transf
 ## 2     19 15.506  2    0.2324 0.1424 0.8682</code></pre>
 <p>Here we would clearly stick with <code>fit6</code>. The eighth order term is not significant with the other terms in the model and the F-test does not reject.</p>
 <p>As an aside, be aware that there is a quicker way to specify a model with many higher order terms.</p>
-<div class="sourceCode" id="cb1107"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1107-1"><a href="transformations.html#cb1107-1" aria-hidden="true" tabindex="-1"></a>fit6_alt <span class="ot">=</span> <span class="fu">lm</span>(mpg <span class="sc">~</span> <span class="fu">poly</span>(mph, <span class="dv">6</span>), <span class="at">data =</span> econ)</span>
-<span id="cb1107-2"><a href="transformations.html#cb1107-2" aria-hidden="true" tabindex="-1"></a><span class="fu">all.equal</span>(<span class="fu">fitted</span>(fit6), <span class="fu">fitted</span>(fit6_alt))</span></code></pre></div>
+<div class="sourceCode" id="cb1107"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1107-1"><a href="transformations.html#cb1107-1" tabindex="-1"></a>fit6_alt <span class="ot">=</span> <span class="fu">lm</span>(mpg <span class="sc">~</span> <span class="fu">poly</span>(mph, <span class="dv">6</span>), <span class="at">data =</span> econ)</span>
+<span id="cb1107-2"><a href="transformations.html#cb1107-2" tabindex="-1"></a><span class="fu">all.equal</span>(<span class="fu">fitted</span>(fit6), <span class="fu">fitted</span>(fit6_alt))</span></code></pre></div>
 <pre><code>## [1] TRUE</code></pre>
 <p>We first verify that this method produces the same fitted values. However, the estimated coefficients are different.</p>
-<div class="sourceCode" id="cb1109"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1109-1"><a href="transformations.html#cb1109-1" aria-hidden="true" tabindex="-1"></a><span class="fu">coef</span>(fit6)</span></code></pre></div>
+<div class="sourceCode" id="cb1109"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1109-1"><a href="transformations.html#cb1109-1" tabindex="-1"></a><span class="fu">coef</span>(fit6)</span></code></pre></div>
 <pre><code>##   (Intercept)           mph      I(mph^2)      I(mph^3)      I(mph^4) 
 ## -4.206224e+00  4.203382e+00 -3.521452e-01  1.579340e-02 -3.472665e-04 
 ##      I(mph^5)      I(mph^6) 
 ##  3.585201e-06 -1.401995e-08</code></pre>
-<div class="sourceCode" id="cb1111"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1111-1"><a href="transformations.html#cb1111-1" aria-hidden="true" tabindex="-1"></a><span class="fu">coef</span>(fit6_alt)</span></code></pre></div>
+<div class="sourceCode" id="cb1111"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1111-1"><a href="transformations.html#cb1111-1" tabindex="-1"></a><span class="fu">coef</span>(fit6_alt)</span></code></pre></div>
 <pre><code>##   (Intercept) poly(mph, 6)1 poly(mph, 6)2 poly(mph, 6)3 poly(mph, 6)4 
 ##   24.40714286    4.16769628  -27.66685755    0.13446747    7.01671480 
 ## poly(mph, 6)5 poly(mph, 6)6 
 ##    0.09288754   -2.04307796</code></pre>
 <p>This is because <code>poly()</code> uses <em>orthogonal polynomials</em>, which solves an issue we will discuss in the next chapter.</p>
-<div class="sourceCode" id="cb1113"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1113-1"><a href="transformations.html#cb1113-1" aria-hidden="true" tabindex="-1"></a><span class="fu">summary</span>(fit6)</span></code></pre></div>
+<div class="sourceCode" id="cb1113"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1113-1"><a href="transformations.html#cb1113-1" tabindex="-1"></a><span class="fu">summary</span>(fit6)</span></code></pre></div>
 <pre><code>## 
 ## Call:
 ## lm(formula = mpg ~ mph + I(mph^2) + I(mph^3) + I(mph^4) + I(mph^5) + 
@@ -1210,7 +1210,7 @@ <h3><span class="header-section-number">14.2.1</span> Polynomials<a href="transf
 ## Residual standard error: 0.8657 on 21 degrees of freedom
 ## Multiple R-squared:  0.9815, Adjusted R-squared:  0.9762 
 ## F-statistic:   186 on 6 and 21 DF,  p-value: &lt; 2.2e-16</code></pre>
-<div class="sourceCode" id="cb1115"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1115-1"><a href="transformations.html#cb1115-1" aria-hidden="true" tabindex="-1"></a><span class="fu">summary</span>(fit6_alt)</span></code></pre></div>
+<div class="sourceCode" id="cb1115"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1115-1"><a href="transformations.html#cb1115-1" tabindex="-1"></a><span class="fu">summary</span>(fit6_alt)</span></code></pre></div>
 <pre><code>## 
 ## Call:
 ## lm(formula = mpg ~ poly(mph, 6), data = econ)
@@ -1236,8 +1236,8 @@ <h3><span class="header-section-number">14.2.1</span> Polynomials<a href="transf
 ## F-statistic:   186 on 6 and 21 DF,  p-value: &lt; 2.2e-16</code></pre>
 <p>Notice though that the p-value for testing the degree 6 term is the same. Because of this, for the most part we can use these interchangeably.</p>
 <p>To use <code>poly()</code> to obtain the same results as using <code>I()</code> repeatedly, we would need to set <code>raw = TRUE</code>.</p>
-<div class="sourceCode" id="cb1117"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1117-1"><a href="transformations.html#cb1117-1" aria-hidden="true" tabindex="-1"></a>fit6_alt2 <span class="ot">=</span> <span class="fu">lm</span>(mpg <span class="sc">~</span> <span class="fu">poly</span>(mph, <span class="dv">6</span>, <span class="at">raw =</span> <span class="cn">TRUE</span>), <span class="at">data =</span> econ)</span>
-<span id="cb1117-2"><a href="transformations.html#cb1117-2" aria-hidden="true" tabindex="-1"></a><span class="fu">coef</span>(fit6_alt2)</span></code></pre></div>
+<div class="sourceCode" id="cb1117"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1117-1"><a href="transformations.html#cb1117-1" tabindex="-1"></a>fit6_alt2 <span class="ot">=</span> <span class="fu">lm</span>(mpg <span class="sc">~</span> <span class="fu">poly</span>(mph, <span class="dv">6</span>, <span class="at">raw =</span> <span class="cn">TRUE</span>), <span class="at">data =</span> econ)</span>
+<span id="cb1117-2"><a href="transformations.html#cb1117-2" tabindex="-1"></a><span class="fu">coef</span>(fit6_alt2)</span></code></pre></div>
 <pre><code>##               (Intercept) poly(mph, 6, raw = TRUE)1 poly(mph, 6, raw = TRUE)2 
 ##             -4.206224e+00              4.203382e+00             -3.521452e-01 
 ## poly(mph, 6, raw = TRUE)3 poly(mph, 6, raw = TRUE)4 poly(mph, 6, raw = TRUE)5 
@@ -1250,12 +1250,12 @@ <h3><span class="header-section-number">14.2.1</span> Polynomials<a href="transf
 </div>
 <div id="response-transformations" class="section level2 unnumbered hasAnchor">
 <h2>Response Transformations<a href="transformations.html#response-transformations" class="anchor-section" aria-label="Anchor link to header"></a></h2>
-<div class="sourceCode" id="cb1119"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1119-1"><a href="transformations.html#cb1119-1" aria-hidden="true" tabindex="-1"></a>initech <span class="ot">=</span> <span class="fu">read.csv</span>(<span class="st">&quot;data/initech.csv&quot;</span>)</span></code></pre></div>
-<div class="sourceCode" id="cb1120"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1120-1"><a href="transformations.html#cb1120-1" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(salary <span class="sc">~</span> years, <span class="at">data =</span> initech, <span class="at">col =</span> <span class="st">&quot;grey&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">cex =</span> <span class="fl">1.5</span>,</span>
-<span id="cb1120-2"><a href="transformations.html#cb1120-2" aria-hidden="true" tabindex="-1"></a>     <span class="at">main =</span> <span class="st">&quot;Salaries at Initech, By Seniority&quot;</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1119"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1119-1"><a href="transformations.html#cb1119-1" tabindex="-1"></a>initech <span class="ot">=</span> <span class="fu">read.csv</span>(<span class="st">&quot;data/initech.csv&quot;</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1120"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1120-1"><a href="transformations.html#cb1120-1" tabindex="-1"></a><span class="fu">plot</span>(salary <span class="sc">~</span> years, <span class="at">data =</span> initech, <span class="at">col =</span> <span class="st">&quot;grey&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">cex =</span> <span class="fl">1.5</span>,</span>
+<span id="cb1120-2"><a href="transformations.html#cb1120-2" tabindex="-1"></a>     <span class="at">main =</span> <span class="st">&quot;Salaries at Initech, By Seniority&quot;</span>)</span></code></pre></div>
 <p><img src="transformations_files/figure-html/unnamed-chunk-62-1.png" width="672" style="display: block; margin: auto;" /></p>
-<div class="sourceCode" id="cb1121"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1121-1"><a href="transformations.html#cb1121-1" aria-hidden="true" tabindex="-1"></a>initech_fit <span class="ot">=</span> <span class="fu">lm</span>(salary <span class="sc">~</span> years, <span class="at">data =</span> initech)</span>
-<span id="cb1121-2"><a href="transformations.html#cb1121-2" aria-hidden="true" tabindex="-1"></a><span class="fu">summary</span>(initech_fit)</span></code></pre></div>
+<div class="sourceCode" id="cb1121"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1121-1"><a href="transformations.html#cb1121-1" tabindex="-1"></a>initech_fit <span class="ot">=</span> <span class="fu">lm</span>(salary <span class="sc">~</span> years, <span class="at">data =</span> initech)</span>
+<span id="cb1121-2"><a href="transformations.html#cb1121-2" tabindex="-1"></a><span class="fu">summary</span>(initech_fit)</span></code></pre></div>
 <pre><code>## 
 ## Call:
 ## lm(formula = salary ~ years, data = initech)
@@ -1274,66 +1274,66 @@ <h2>Response Transformations<a href="transformations.html#response-transformatio
 ## Residual standard error: 27360 on 98 degrees of freedom
 ## Multiple R-squared:  0.8341, Adjusted R-squared:  0.8324 
 ## F-statistic: 492.8 on 1 and 98 DF,  p-value: &lt; 2.2e-16</code></pre>
-<div class="sourceCode" id="cb1123"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1123-1"><a href="transformations.html#cb1123-1" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(salary <span class="sc">~</span> years, <span class="at">data =</span> initech, <span class="at">col =</span> <span class="st">&quot;grey&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">cex =</span> <span class="fl">1.5</span>,</span>
-<span id="cb1123-2"><a href="transformations.html#cb1123-2" aria-hidden="true" tabindex="-1"></a>     <span class="at">main =</span> <span class="st">&quot;Salaries at Initech, By Seniority&quot;</span>)</span>
-<span id="cb1123-3"><a href="transformations.html#cb1123-3" aria-hidden="true" tabindex="-1"></a><span class="fu">abline</span>(initech_fit, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1123"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1123-1"><a href="transformations.html#cb1123-1" tabindex="-1"></a><span class="fu">plot</span>(salary <span class="sc">~</span> years, <span class="at">data =</span> initech, <span class="at">col =</span> <span class="st">&quot;grey&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">cex =</span> <span class="fl">1.5</span>,</span>
+<span id="cb1123-2"><a href="transformations.html#cb1123-2" tabindex="-1"></a>     <span class="at">main =</span> <span class="st">&quot;Salaries at Initech, By Seniority&quot;</span>)</span>
+<span id="cb1123-3"><a href="transformations.html#cb1123-3" tabindex="-1"></a><span class="fu">abline</span>(initech_fit, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
 <p><img src="transformations_files/figure-html/unnamed-chunk-64-1.png" width="672" style="display: block; margin: auto;" /></p>
-<div class="sourceCode" id="cb1124"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1124-1"><a href="transformations.html#cb1124-1" aria-hidden="true" tabindex="-1"></a><span class="fu">par</span>(<span class="at">mfrow =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>))</span>
-<span id="cb1124-2"><a href="transformations.html#cb1124-2" aria-hidden="true" tabindex="-1"></a></span>
-<span id="cb1124-3"><a href="transformations.html#cb1124-3" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(<span class="fu">fitted</span>(initech_fit), <span class="fu">resid</span>(initech_fit), <span class="at">col =</span> <span class="st">&quot;grey&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>,</span>
-<span id="cb1124-4"><a href="transformations.html#cb1124-4" aria-hidden="true" tabindex="-1"></a>     <span class="at">xlab =</span> <span class="st">&quot;Fitted&quot;</span>, <span class="at">ylab =</span> <span class="st">&quot;Residuals&quot;</span>, <span class="at">main =</span> <span class="st">&quot;Fitted versus Residuals&quot;</span>)</span>
-<span id="cb1124-5"><a href="transformations.html#cb1124-5" aria-hidden="true" tabindex="-1"></a><span class="fu">abline</span>(<span class="at">h =</span> <span class="dv">0</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span>
-<span id="cb1124-6"><a href="transformations.html#cb1124-6" aria-hidden="true" tabindex="-1"></a></span>
-<span id="cb1124-7"><a href="transformations.html#cb1124-7" aria-hidden="true" tabindex="-1"></a><span class="fu">qqnorm</span>(<span class="fu">resid</span>(initech_fit), <span class="at">main =</span> <span class="st">&quot;Normal Q-Q Plot&quot;</span>, <span class="at">col =</span> <span class="st">&quot;darkgrey&quot;</span>)</span>
-<span id="cb1124-8"><a href="transformations.html#cb1124-8" aria-hidden="true" tabindex="-1"></a><span class="fu">qqline</span>(<span class="fu">resid</span>(initech_fit), <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1124"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1124-1"><a href="transformations.html#cb1124-1" tabindex="-1"></a><span class="fu">par</span>(<span class="at">mfrow =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>))</span>
+<span id="cb1124-2"><a href="transformations.html#cb1124-2" tabindex="-1"></a></span>
+<span id="cb1124-3"><a href="transformations.html#cb1124-3" tabindex="-1"></a><span class="fu">plot</span>(<span class="fu">fitted</span>(initech_fit), <span class="fu">resid</span>(initech_fit), <span class="at">col =</span> <span class="st">&quot;grey&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>,</span>
+<span id="cb1124-4"><a href="transformations.html#cb1124-4" tabindex="-1"></a>     <span class="at">xlab =</span> <span class="st">&quot;Fitted&quot;</span>, <span class="at">ylab =</span> <span class="st">&quot;Residuals&quot;</span>, <span class="at">main =</span> <span class="st">&quot;Fitted versus Residuals&quot;</span>)</span>
+<span id="cb1124-5"><a href="transformations.html#cb1124-5" tabindex="-1"></a><span class="fu">abline</span>(<span class="at">h =</span> <span class="dv">0</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span>
+<span id="cb1124-6"><a href="transformations.html#cb1124-6" tabindex="-1"></a></span>
+<span id="cb1124-7"><a href="transformations.html#cb1124-7" tabindex="-1"></a><span class="fu">qqnorm</span>(<span class="fu">resid</span>(initech_fit), <span class="at">main =</span> <span class="st">&quot;Normal Q-Q Plot&quot;</span>, <span class="at">col =</span> <span class="st">&quot;darkgrey&quot;</span>)</span>
+<span id="cb1124-8"><a href="transformations.html#cb1124-8" tabindex="-1"></a><span class="fu">qqline</span>(<span class="fu">resid</span>(initech_fit), <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
 <p><img src="transformations_files/figure-html/unnamed-chunk-65-1.png" width="960" style="display: block; margin: auto;" /></p>
-<div class="sourceCode" id="cb1125"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1125-1"><a href="transformations.html#cb1125-1" aria-hidden="true" tabindex="-1"></a>initech_fit_log <span class="ot">=</span> <span class="fu">lm</span>(<span class="fu">log</span>(salary) <span class="sc">~</span> years, <span class="at">data =</span> initech)</span></code></pre></div>
+<div class="sourceCode" id="cb1125"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1125-1"><a href="transformations.html#cb1125-1" tabindex="-1"></a>initech_fit_log <span class="ot">=</span> <span class="fu">lm</span>(<span class="fu">log</span>(salary) <span class="sc">~</span> years, <span class="at">data =</span> initech)</span></code></pre></div>
 <p><span class="math display">\[
 \log(Y_i) = \beta_0 + \beta_1 x_i + \epsilon_i
 \]</span></p>
-<div class="sourceCode" id="cb1126"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1126-1"><a href="transformations.html#cb1126-1" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(<span class="fu">log</span>(salary) <span class="sc">~</span> years, <span class="at">data =</span> initech, <span class="at">col =</span> <span class="st">&quot;grey&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">cex =</span> <span class="fl">1.5</span>,</span>
-<span id="cb1126-2"><a href="transformations.html#cb1126-2" aria-hidden="true" tabindex="-1"></a>     <span class="at">main =</span> <span class="st">&quot;Salaries at Initech, By Seniority&quot;</span>)</span>
-<span id="cb1126-3"><a href="transformations.html#cb1126-3" aria-hidden="true" tabindex="-1"></a><span class="fu">abline</span>(initech_fit_log, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1126"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1126-1"><a href="transformations.html#cb1126-1" tabindex="-1"></a><span class="fu">plot</span>(<span class="fu">log</span>(salary) <span class="sc">~</span> years, <span class="at">data =</span> initech, <span class="at">col =</span> <span class="st">&quot;grey&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">cex =</span> <span class="fl">1.5</span>,</span>
+<span id="cb1126-2"><a href="transformations.html#cb1126-2" tabindex="-1"></a>     <span class="at">main =</span> <span class="st">&quot;Salaries at Initech, By Seniority&quot;</span>)</span>
+<span id="cb1126-3"><a href="transformations.html#cb1126-3" tabindex="-1"></a><span class="fu">abline</span>(initech_fit_log, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
 <p><img src="transformations_files/figure-html/unnamed-chunk-67-1.png" width="672" style="display: block; margin: auto;" /></p>
 <p><span class="math display">\[
 Y_i = \exp(\beta_0 + \beta_1 x_i) \cdot \exp(\epsilon_i)
 \]</span></p>
-<div class="sourceCode" id="cb1127"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1127-1"><a href="transformations.html#cb1127-1" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(salary <span class="sc">~</span> years, <span class="at">data =</span> initech, <span class="at">col =</span> <span class="st">&quot;grey&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">cex =</span> <span class="fl">1.5</span>,</span>
-<span id="cb1127-2"><a href="transformations.html#cb1127-2" aria-hidden="true" tabindex="-1"></a>     <span class="at">main =</span> <span class="st">&quot;Salaries at Initech, By Seniority&quot;</span>)</span>
-<span id="cb1127-3"><a href="transformations.html#cb1127-3" aria-hidden="true" tabindex="-1"></a><span class="fu">curve</span>(<span class="fu">exp</span>(initech_fit_log<span class="sc">$</span>coef[<span class="dv">1</span>] <span class="sc">+</span> initech_fit_log<span class="sc">$</span>coef[<span class="dv">2</span>] <span class="sc">*</span> x),</span>
-<span id="cb1127-4"><a href="transformations.html#cb1127-4" aria-hidden="true" tabindex="-1"></a>      <span class="at">from =</span> <span class="dv">0</span>, <span class="at">to =</span> <span class="dv">30</span>, <span class="at">add =</span> <span class="cn">TRUE</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1127"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1127-1"><a href="transformations.html#cb1127-1" tabindex="-1"></a><span class="fu">plot</span>(salary <span class="sc">~</span> years, <span class="at">data =</span> initech, <span class="at">col =</span> <span class="st">&quot;grey&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">cex =</span> <span class="fl">1.5</span>,</span>
+<span id="cb1127-2"><a href="transformations.html#cb1127-2" tabindex="-1"></a>     <span class="at">main =</span> <span class="st">&quot;Salaries at Initech, By Seniority&quot;</span>)</span>
+<span id="cb1127-3"><a href="transformations.html#cb1127-3" tabindex="-1"></a><span class="fu">curve</span>(<span class="fu">exp</span>(initech_fit_log<span class="sc">$</span>coef[<span class="dv">1</span>] <span class="sc">+</span> initech_fit_log<span class="sc">$</span>coef[<span class="dv">2</span>] <span class="sc">*</span> x),</span>
+<span id="cb1127-4"><a href="transformations.html#cb1127-4" tabindex="-1"></a>      <span class="at">from =</span> <span class="dv">0</span>, <span class="at">to =</span> <span class="dv">30</span>, <span class="at">add =</span> <span class="cn">TRUE</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
 <p><img src="transformations_files/figure-html/unnamed-chunk-68-1.png" width="672" style="display: block; margin: auto;" /></p>
-<div class="sourceCode" id="cb1128"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1128-1"><a href="transformations.html#cb1128-1" aria-hidden="true" tabindex="-1"></a><span class="fu">par</span>(<span class="at">mfrow =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>))</span>
-<span id="cb1128-2"><a href="transformations.html#cb1128-2" aria-hidden="true" tabindex="-1"></a></span>
-<span id="cb1128-3"><a href="transformations.html#cb1128-3" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(<span class="fu">fitted</span>(initech_fit_log), <span class="fu">resid</span>(initech_fit_log), <span class="at">col =</span> <span class="st">&quot;grey&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>,</span>
-<span id="cb1128-4"><a href="transformations.html#cb1128-4" aria-hidden="true" tabindex="-1"></a>     <span class="at">xlab =</span> <span class="st">&quot;Fitted&quot;</span>, <span class="at">ylab =</span> <span class="st">&quot;Residuals&quot;</span>, <span class="at">main =</span> <span class="st">&quot;Fitted versus Residuals&quot;</span>)</span>
-<span id="cb1128-5"><a href="transformations.html#cb1128-5" aria-hidden="true" tabindex="-1"></a><span class="fu">abline</span>(<span class="at">h =</span> <span class="dv">0</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span>
-<span id="cb1128-6"><a href="transformations.html#cb1128-6" aria-hidden="true" tabindex="-1"></a></span>
-<span id="cb1128-7"><a href="transformations.html#cb1128-7" aria-hidden="true" tabindex="-1"></a><span class="fu">qqnorm</span>(<span class="fu">resid</span>(initech_fit_log), <span class="at">main =</span> <span class="st">&quot;Normal Q-Q Plot&quot;</span>, <span class="at">col =</span> <span class="st">&quot;darkgrey&quot;</span>)</span>
-<span id="cb1128-8"><a href="transformations.html#cb1128-8" aria-hidden="true" tabindex="-1"></a><span class="fu">qqline</span>(<span class="fu">resid</span>(initech_fit_log), <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1128"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1128-1"><a href="transformations.html#cb1128-1" tabindex="-1"></a><span class="fu">par</span>(<span class="at">mfrow =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>))</span>
+<span id="cb1128-2"><a href="transformations.html#cb1128-2" tabindex="-1"></a></span>
+<span id="cb1128-3"><a href="transformations.html#cb1128-3" tabindex="-1"></a><span class="fu">plot</span>(<span class="fu">fitted</span>(initech_fit_log), <span class="fu">resid</span>(initech_fit_log), <span class="at">col =</span> <span class="st">&quot;grey&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>,</span>
+<span id="cb1128-4"><a href="transformations.html#cb1128-4" tabindex="-1"></a>     <span class="at">xlab =</span> <span class="st">&quot;Fitted&quot;</span>, <span class="at">ylab =</span> <span class="st">&quot;Residuals&quot;</span>, <span class="at">main =</span> <span class="st">&quot;Fitted versus Residuals&quot;</span>)</span>
+<span id="cb1128-5"><a href="transformations.html#cb1128-5" tabindex="-1"></a><span class="fu">abline</span>(<span class="at">h =</span> <span class="dv">0</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span>
+<span id="cb1128-6"><a href="transformations.html#cb1128-6" tabindex="-1"></a></span>
+<span id="cb1128-7"><a href="transformations.html#cb1128-7" tabindex="-1"></a><span class="fu">qqnorm</span>(<span class="fu">resid</span>(initech_fit_log), <span class="at">main =</span> <span class="st">&quot;Normal Q-Q Plot&quot;</span>, <span class="at">col =</span> <span class="st">&quot;darkgrey&quot;</span>)</span>
+<span id="cb1128-8"><a href="transformations.html#cb1128-8" tabindex="-1"></a><span class="fu">qqline</span>(<span class="fu">resid</span>(initech_fit_log), <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
 <p><img src="transformations_files/figure-html/unnamed-chunk-69-1.png" width="960" style="display: block; margin: auto;" /></p>
-<div class="sourceCode" id="cb1129"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1129-1"><a href="transformations.html#cb1129-1" aria-hidden="true" tabindex="-1"></a><span class="fu">sqrt</span>(<span class="fu">mean</span>(<span class="fu">resid</span>(initech_fit) <span class="sc">^</span> <span class="dv">2</span>))</span></code></pre></div>
+<div class="sourceCode" id="cb1129"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1129-1"><a href="transformations.html#cb1129-1" tabindex="-1"></a><span class="fu">sqrt</span>(<span class="fu">mean</span>(<span class="fu">resid</span>(initech_fit) <span class="sc">^</span> <span class="dv">2</span>))</span></code></pre></div>
 <pre><code>## [1] 27080.16</code></pre>
-<div class="sourceCode" id="cb1131"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1131-1"><a href="transformations.html#cb1131-1" aria-hidden="true" tabindex="-1"></a><span class="fu">sqrt</span>(<span class="fu">mean</span>(<span class="fu">resid</span>(initech_fit_log) <span class="sc">^</span> <span class="dv">2</span>))</span></code></pre></div>
+<div class="sourceCode" id="cb1131"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1131-1"><a href="transformations.html#cb1131-1" tabindex="-1"></a><span class="fu">sqrt</span>(<span class="fu">mean</span>(<span class="fu">resid</span>(initech_fit_log) <span class="sc">^</span> <span class="dv">2</span>))</span></code></pre></div>
 <pre><code>## [1] 0.1934907</code></pre>
-<div class="sourceCode" id="cb1133"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1133-1"><a href="transformations.html#cb1133-1" aria-hidden="true" tabindex="-1"></a><span class="fu">sqrt</span>(<span class="fu">mean</span>((initech<span class="sc">$</span>salary <span class="sc">-</span> <span class="fu">fitted</span>(initech_fit)) <span class="sc">^</span> <span class="dv">2</span>))</span></code></pre></div>
+<div class="sourceCode" id="cb1133"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1133-1"><a href="transformations.html#cb1133-1" tabindex="-1"></a><span class="fu">sqrt</span>(<span class="fu">mean</span>((initech<span class="sc">$</span>salary <span class="sc">-</span> <span class="fu">fitted</span>(initech_fit)) <span class="sc">^</span> <span class="dv">2</span>))</span></code></pre></div>
 <pre><code>## [1] 27080.16</code></pre>
-<div class="sourceCode" id="cb1135"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1135-1"><a href="transformations.html#cb1135-1" aria-hidden="true" tabindex="-1"></a><span class="fu">sqrt</span>(<span class="fu">mean</span>((initech<span class="sc">$</span>salary <span class="sc">-</span> <span class="fu">exp</span>(<span class="fu">fitted</span>(initech_fit_log))) <span class="sc">^</span> <span class="dv">2</span>))</span></code></pre></div>
+<div class="sourceCode" id="cb1135"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1135-1"><a href="transformations.html#cb1135-1" tabindex="-1"></a><span class="fu">sqrt</span>(<span class="fu">mean</span>((initech<span class="sc">$</span>salary <span class="sc">-</span> <span class="fu">exp</span>(<span class="fu">fitted</span>(initech_fit_log))) <span class="sc">^</span> <span class="dv">2</span>))</span></code></pre></div>
 <pre><code>## [1] 24280.36</code></pre>
 </div>
 <div id="predictor-transformations" class="section level2 unnumbered hasAnchor">
 <h2>Predictor Transformations<a href="transformations.html#predictor-transformations" class="anchor-section" aria-label="Anchor link to header"></a></h2>
 <div id="a-quadratic-model" class="section level3 hasAnchor" number="14.2.2">
 <h3><span class="header-section-number">14.2.2</span> A Quadratic Model<a href="transformations.html#a-quadratic-model" class="anchor-section" aria-label="Anchor link to header"></a></h3>
-<div class="sourceCode" id="cb1137"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1137-1"><a href="transformations.html#cb1137-1" aria-hidden="true" tabindex="-1"></a>sim_quad <span class="ot">=</span> <span class="cf">function</span>(<span class="at">sample_size =</span> <span class="dv">500</span>) {</span>
-<span id="cb1137-2"><a href="transformations.html#cb1137-2" aria-hidden="true" tabindex="-1"></a>  x <span class="ot">=</span> <span class="fu">runif</span>(<span class="at">n =</span> sample_size) <span class="sc">*</span> <span class="dv">5</span></span>
-<span id="cb1137-3"><a href="transformations.html#cb1137-3" aria-hidden="true" tabindex="-1"></a>  y <span class="ot">=</span> <span class="dv">3</span> <span class="sc">+</span> <span class="dv">5</span> <span class="sc">*</span> x <span class="sc">^</span> <span class="dv">2</span> <span class="sc">+</span> <span class="fu">rnorm</span>(<span class="at">n =</span> sample_size, <span class="at">mean =</span> <span class="dv">0</span>, <span class="at">sd =</span> <span class="dv">5</span>)</span>
-<span id="cb1137-4"><a href="transformations.html#cb1137-4" aria-hidden="true" tabindex="-1"></a>  <span class="fu">data.frame</span>(x, y)</span>
-<span id="cb1137-5"><a href="transformations.html#cb1137-5" aria-hidden="true" tabindex="-1"></a>}</span></code></pre></div>
-<div class="sourceCode" id="cb1138"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1138-1"><a href="transformations.html#cb1138-1" aria-hidden="true" tabindex="-1"></a><span class="fu">set.seed</span>(<span class="dv">314</span>)</span>
-<span id="cb1138-2"><a href="transformations.html#cb1138-2" aria-hidden="true" tabindex="-1"></a>quad_data <span class="ot">=</span> <span class="fu">sim_quad</span>(<span class="at">sample_size =</span> <span class="dv">200</span>)</span></code></pre></div>
-<div class="sourceCode" id="cb1139"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1139-1"><a href="transformations.html#cb1139-1" aria-hidden="true" tabindex="-1"></a>lin_fit <span class="ot">=</span> <span class="fu">lm</span>(y <span class="sc">~</span> x, <span class="at">data =</span> quad_data)</span>
-<span id="cb1139-2"><a href="transformations.html#cb1139-2" aria-hidden="true" tabindex="-1"></a><span class="fu">summary</span>(lin_fit)</span></code></pre></div>
+<div class="sourceCode" id="cb1137"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1137-1"><a href="transformations.html#cb1137-1" tabindex="-1"></a>sim_quad <span class="ot">=</span> <span class="cf">function</span>(<span class="at">sample_size =</span> <span class="dv">500</span>) {</span>
+<span id="cb1137-2"><a href="transformations.html#cb1137-2" tabindex="-1"></a>  x <span class="ot">=</span> <span class="fu">runif</span>(<span class="at">n =</span> sample_size) <span class="sc">*</span> <span class="dv">5</span></span>
+<span id="cb1137-3"><a href="transformations.html#cb1137-3" tabindex="-1"></a>  y <span class="ot">=</span> <span class="dv">3</span> <span class="sc">+</span> <span class="dv">5</span> <span class="sc">*</span> x <span class="sc">^</span> <span class="dv">2</span> <span class="sc">+</span> <span class="fu">rnorm</span>(<span class="at">n =</span> sample_size, <span class="at">mean =</span> <span class="dv">0</span>, <span class="at">sd =</span> <span class="dv">5</span>)</span>
+<span id="cb1137-4"><a href="transformations.html#cb1137-4" tabindex="-1"></a>  <span class="fu">data.frame</span>(x, y)</span>
+<span id="cb1137-5"><a href="transformations.html#cb1137-5" tabindex="-1"></a>}</span></code></pre></div>
+<div class="sourceCode" id="cb1138"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1138-1"><a href="transformations.html#cb1138-1" tabindex="-1"></a><span class="fu">set.seed</span>(<span class="dv">314</span>)</span>
+<span id="cb1138-2"><a href="transformations.html#cb1138-2" tabindex="-1"></a>quad_data <span class="ot">=</span> <span class="fu">sim_quad</span>(<span class="at">sample_size =</span> <span class="dv">200</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1139"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1139-1"><a href="transformations.html#cb1139-1" tabindex="-1"></a>lin_fit <span class="ot">=</span> <span class="fu">lm</span>(y <span class="sc">~</span> x, <span class="at">data =</span> quad_data)</span>
+<span id="cb1139-2"><a href="transformations.html#cb1139-2" tabindex="-1"></a><span class="fu">summary</span>(lin_fit)</span></code></pre></div>
 <pre><code>## 
 ## Call:
 ## lm(formula = y ~ x, data = quad_data)
@@ -1352,24 +1352,24 @@ <h3><span class="header-section-number">14.2.2</span> A Quadratic Model<a href="
 ## Residual standard error: 10.79 on 198 degrees of freedom
 ## Multiple R-squared:  0.9163, Adjusted R-squared:  0.9158 
 ## F-statistic:  2167 on 1 and 198 DF,  p-value: &lt; 2.2e-16</code></pre>
-<div class="sourceCode" id="cb1141"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1141-1"><a href="transformations.html#cb1141-1" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(y <span class="sc">~</span> x, <span class="at">data =</span> quad_data, <span class="at">col =</span> <span class="st">&quot;grey&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">cex =</span> <span class="fl">1.5</span>,</span>
-<span id="cb1141-2"><a href="transformations.html#cb1141-2" aria-hidden="true" tabindex="-1"></a>     <span class="at">main =</span> <span class="st">&quot;Simulated Quadratic Data&quot;</span>)</span>
-<span id="cb1141-3"><a href="transformations.html#cb1141-3" aria-hidden="true" tabindex="-1"></a><span class="fu">abline</span>(lin_fit, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1141"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1141-1"><a href="transformations.html#cb1141-1" tabindex="-1"></a><span class="fu">plot</span>(y <span class="sc">~</span> x, <span class="at">data =</span> quad_data, <span class="at">col =</span> <span class="st">&quot;grey&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">cex =</span> <span class="fl">1.5</span>,</span>
+<span id="cb1141-2"><a href="transformations.html#cb1141-2" tabindex="-1"></a>     <span class="at">main =</span> <span class="st">&quot;Simulated Quadratic Data&quot;</span>)</span>
+<span id="cb1141-3"><a href="transformations.html#cb1141-3" tabindex="-1"></a><span class="fu">abline</span>(lin_fit, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
 <p><img src="transformations_files/figure-html/unnamed-chunk-75-1.png" width="672" style="display: block; margin: auto;" /></p>
-<div class="sourceCode" id="cb1142"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1142-1"><a href="transformations.html#cb1142-1" aria-hidden="true" tabindex="-1"></a><span class="fu">par</span>(<span class="at">mfrow =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>))</span>
-<span id="cb1142-2"><a href="transformations.html#cb1142-2" aria-hidden="true" tabindex="-1"></a></span>
-<span id="cb1142-3"><a href="transformations.html#cb1142-3" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(<span class="fu">fitted</span>(lin_fit), <span class="fu">resid</span>(lin_fit), <span class="at">col =</span> <span class="st">&quot;grey&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>,</span>
-<span id="cb1142-4"><a href="transformations.html#cb1142-4" aria-hidden="true" tabindex="-1"></a>     <span class="at">xlab =</span> <span class="st">&quot;Fitted&quot;</span>, <span class="at">ylab =</span> <span class="st">&quot;Residuals&quot;</span>, <span class="at">main =</span> <span class="st">&quot;Fitted versus Residuals&quot;</span>)</span>
-<span id="cb1142-5"><a href="transformations.html#cb1142-5" aria-hidden="true" tabindex="-1"></a><span class="fu">abline</span>(<span class="at">h =</span> <span class="dv">0</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span>
-<span id="cb1142-6"><a href="transformations.html#cb1142-6" aria-hidden="true" tabindex="-1"></a></span>
-<span id="cb1142-7"><a href="transformations.html#cb1142-7" aria-hidden="true" tabindex="-1"></a><span class="fu">qqnorm</span>(<span class="fu">resid</span>(lin_fit), <span class="at">main =</span> <span class="st">&quot;Normal Q-Q Plot&quot;</span>, <span class="at">col =</span> <span class="st">&quot;darkgrey&quot;</span>)</span>
-<span id="cb1142-8"><a href="transformations.html#cb1142-8" aria-hidden="true" tabindex="-1"></a><span class="fu">qqline</span>(<span class="fu">resid</span>(lin_fit), <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1142"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1142-1"><a href="transformations.html#cb1142-1" tabindex="-1"></a><span class="fu">par</span>(<span class="at">mfrow =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>))</span>
+<span id="cb1142-2"><a href="transformations.html#cb1142-2" tabindex="-1"></a></span>
+<span id="cb1142-3"><a href="transformations.html#cb1142-3" tabindex="-1"></a><span class="fu">plot</span>(<span class="fu">fitted</span>(lin_fit), <span class="fu">resid</span>(lin_fit), <span class="at">col =</span> <span class="st">&quot;grey&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>,</span>
+<span id="cb1142-4"><a href="transformations.html#cb1142-4" tabindex="-1"></a>     <span class="at">xlab =</span> <span class="st">&quot;Fitted&quot;</span>, <span class="at">ylab =</span> <span class="st">&quot;Residuals&quot;</span>, <span class="at">main =</span> <span class="st">&quot;Fitted versus Residuals&quot;</span>)</span>
+<span id="cb1142-5"><a href="transformations.html#cb1142-5" tabindex="-1"></a><span class="fu">abline</span>(<span class="at">h =</span> <span class="dv">0</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span>
+<span id="cb1142-6"><a href="transformations.html#cb1142-6" tabindex="-1"></a></span>
+<span id="cb1142-7"><a href="transformations.html#cb1142-7" tabindex="-1"></a><span class="fu">qqnorm</span>(<span class="fu">resid</span>(lin_fit), <span class="at">main =</span> <span class="st">&quot;Normal Q-Q Plot&quot;</span>, <span class="at">col =</span> <span class="st">&quot;darkgrey&quot;</span>)</span>
+<span id="cb1142-8"><a href="transformations.html#cb1142-8" tabindex="-1"></a><span class="fu">qqline</span>(<span class="fu">resid</span>(lin_fit), <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
 <p><img src="transformations_files/figure-html/unnamed-chunk-76-1.png" width="960" style="display: block; margin: auto;" /></p>
 <p><span class="math display">\[
 Y_i = \beta_0 + \beta_1 x_i + \beta_2 x_i^2 + \epsilon_i
 \]</span></p>
-<div class="sourceCode" id="cb1143"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1143-1"><a href="transformations.html#cb1143-1" aria-hidden="true" tabindex="-1"></a>quad_fit <span class="ot">=</span> <span class="fu">lm</span>(y <span class="sc">~</span> x <span class="sc">+</span> <span class="fu">I</span>(x<span class="sc">^</span><span class="dv">2</span>), <span class="at">data =</span> quad_data)</span>
-<span id="cb1143-2"><a href="transformations.html#cb1143-2" aria-hidden="true" tabindex="-1"></a><span class="fu">summary</span>(quad_fit)</span></code></pre></div>
+<div class="sourceCode" id="cb1143"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1143-1"><a href="transformations.html#cb1143-1" tabindex="-1"></a>quad_fit <span class="ot">=</span> <span class="fu">lm</span>(y <span class="sc">~</span> x <span class="sc">+</span> <span class="fu">I</span>(x<span class="sc">^</span><span class="dv">2</span>), <span class="at">data =</span> quad_data)</span>
+<span id="cb1143-2"><a href="transformations.html#cb1143-2" tabindex="-1"></a><span class="fu">summary</span>(quad_fit)</span></code></pre></div>
 <pre><code>## 
 ## Call:
 ## lm(formula = y ~ x + I(x^2), data = quad_data)
@@ -1389,49 +1389,49 @@ <h3><span class="header-section-number">14.2.2</span> A Quadratic Model<a href="
 ## Residual standard error: 4.531 on 197 degrees of freedom
 ## Multiple R-squared:  0.9853, Adjusted R-squared:  0.9852 
 ## F-statistic:  6608 on 2 and 197 DF,  p-value: &lt; 2.2e-16</code></pre>
-<div class="sourceCode" id="cb1145"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1145-1"><a href="transformations.html#cb1145-1" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(y <span class="sc">~</span> x, <span class="at">data =</span> quad_data, <span class="at">col =</span> <span class="st">&quot;grey&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">cex =</span> <span class="fl">1.5</span>,</span>
-<span id="cb1145-2"><a href="transformations.html#cb1145-2" aria-hidden="true" tabindex="-1"></a>     <span class="at">main =</span> <span class="st">&quot;Simulated Quadratic Data&quot;</span>)</span>
-<span id="cb1145-3"><a href="transformations.html#cb1145-3" aria-hidden="true" tabindex="-1"></a><span class="fu">curve</span>(quad_fit<span class="sc">$</span>coef[<span class="dv">1</span>] <span class="sc">+</span> quad_fit<span class="sc">$</span>coef[<span class="dv">2</span>] <span class="sc">*</span> x <span class="sc">+</span> quad_fit<span class="sc">$</span>coef[<span class="dv">3</span>] <span class="sc">*</span> x <span class="sc">^</span> <span class="dv">2</span>,</span>
-<span id="cb1145-4"><a href="transformations.html#cb1145-4" aria-hidden="true" tabindex="-1"></a>      <span class="at">from =</span> <span class="sc">-</span><span class="dv">5</span>, <span class="at">to =</span> <span class="dv">30</span>, <span class="at">add =</span> <span class="cn">TRUE</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1145"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1145-1"><a href="transformations.html#cb1145-1" tabindex="-1"></a><span class="fu">plot</span>(y <span class="sc">~</span> x, <span class="at">data =</span> quad_data, <span class="at">col =</span> <span class="st">&quot;grey&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">cex =</span> <span class="fl">1.5</span>,</span>
+<span id="cb1145-2"><a href="transformations.html#cb1145-2" tabindex="-1"></a>     <span class="at">main =</span> <span class="st">&quot;Simulated Quadratic Data&quot;</span>)</span>
+<span id="cb1145-3"><a href="transformations.html#cb1145-3" tabindex="-1"></a><span class="fu">curve</span>(quad_fit<span class="sc">$</span>coef[<span class="dv">1</span>] <span class="sc">+</span> quad_fit<span class="sc">$</span>coef[<span class="dv">2</span>] <span class="sc">*</span> x <span class="sc">+</span> quad_fit<span class="sc">$</span>coef[<span class="dv">3</span>] <span class="sc">*</span> x <span class="sc">^</span> <span class="dv">2</span>,</span>
+<span id="cb1145-4"><a href="transformations.html#cb1145-4" tabindex="-1"></a>      <span class="at">from =</span> <span class="sc">-</span><span class="dv">5</span>, <span class="at">to =</span> <span class="dv">30</span>, <span class="at">add =</span> <span class="cn">TRUE</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
 <p><img src="transformations_files/figure-html/unnamed-chunk-78-1.png" width="672" style="display: block; margin: auto;" /></p>
-<div class="sourceCode" id="cb1146"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1146-1"><a href="transformations.html#cb1146-1" aria-hidden="true" tabindex="-1"></a><span class="fu">par</span>(<span class="at">mfrow =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>))</span>
-<span id="cb1146-2"><a href="transformations.html#cb1146-2" aria-hidden="true" tabindex="-1"></a></span>
-<span id="cb1146-3"><a href="transformations.html#cb1146-3" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(<span class="fu">fitted</span>(quad_fit), <span class="fu">resid</span>(quad_fit), <span class="at">col =</span> <span class="st">&quot;grey&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>,</span>
-<span id="cb1146-4"><a href="transformations.html#cb1146-4" aria-hidden="true" tabindex="-1"></a>     <span class="at">xlab =</span> <span class="st">&quot;Fitted&quot;</span>, <span class="at">ylab =</span> <span class="st">&quot;Residuals&quot;</span>, <span class="at">main =</span> <span class="st">&quot;Fitted versus Residuals&quot;</span>)</span>
-<span id="cb1146-5"><a href="transformations.html#cb1146-5" aria-hidden="true" tabindex="-1"></a><span class="fu">abline</span>(<span class="at">h =</span> <span class="dv">0</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span>
-<span id="cb1146-6"><a href="transformations.html#cb1146-6" aria-hidden="true" tabindex="-1"></a></span>
-<span id="cb1146-7"><a href="transformations.html#cb1146-7" aria-hidden="true" tabindex="-1"></a><span class="fu">qqnorm</span>(<span class="fu">resid</span>(quad_fit), <span class="at">main =</span> <span class="st">&quot;Normal Q-Q Plot&quot;</span>, <span class="at">col =</span> <span class="st">&quot;darkgrey&quot;</span>)</span>
-<span id="cb1146-8"><a href="transformations.html#cb1146-8" aria-hidden="true" tabindex="-1"></a><span class="fu">qqline</span>(<span class="fu">resid</span>(quad_fit), <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1146"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1146-1"><a href="transformations.html#cb1146-1" tabindex="-1"></a><span class="fu">par</span>(<span class="at">mfrow =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>))</span>
+<span id="cb1146-2"><a href="transformations.html#cb1146-2" tabindex="-1"></a></span>
+<span id="cb1146-3"><a href="transformations.html#cb1146-3" tabindex="-1"></a><span class="fu">plot</span>(<span class="fu">fitted</span>(quad_fit), <span class="fu">resid</span>(quad_fit), <span class="at">col =</span> <span class="st">&quot;grey&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>,</span>
+<span id="cb1146-4"><a href="transformations.html#cb1146-4" tabindex="-1"></a>     <span class="at">xlab =</span> <span class="st">&quot;Fitted&quot;</span>, <span class="at">ylab =</span> <span class="st">&quot;Residuals&quot;</span>, <span class="at">main =</span> <span class="st">&quot;Fitted versus Residuals&quot;</span>)</span>
+<span id="cb1146-5"><a href="transformations.html#cb1146-5" tabindex="-1"></a><span class="fu">abline</span>(<span class="at">h =</span> <span class="dv">0</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span>
+<span id="cb1146-6"><a href="transformations.html#cb1146-6" tabindex="-1"></a></span>
+<span id="cb1146-7"><a href="transformations.html#cb1146-7" tabindex="-1"></a><span class="fu">qqnorm</span>(<span class="fu">resid</span>(quad_fit), <span class="at">main =</span> <span class="st">&quot;Normal Q-Q Plot&quot;</span>, <span class="at">col =</span> <span class="st">&quot;darkgrey&quot;</span>)</span>
+<span id="cb1146-8"><a href="transformations.html#cb1146-8" tabindex="-1"></a><span class="fu">qqline</span>(<span class="fu">resid</span>(quad_fit), <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
 <p><img src="transformations_files/figure-html/unnamed-chunk-79-1.png" width="960" style="display: block; margin: auto;" /></p>
 </div>
 <div id="overfitting-and-extrapolation" class="section level3 hasAnchor" number="14.2.3">
 <h3><span class="header-section-number">14.2.3</span> Overfitting and Extrapolation<a href="transformations.html#overfitting-and-extrapolation" class="anchor-section" aria-label="Anchor link to header"></a></h3>
-<div class="sourceCode" id="cb1147"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1147-1"><a href="transformations.html#cb1147-1" aria-hidden="true" tabindex="-1"></a>sim_for_perf <span class="ot">=</span> <span class="cf">function</span>() {</span>
-<span id="cb1147-2"><a href="transformations.html#cb1147-2" aria-hidden="true" tabindex="-1"></a>  x <span class="ot">=</span> <span class="fu">seq</span>(<span class="dv">0</span>, <span class="dv">10</span>)</span>
-<span id="cb1147-3"><a href="transformations.html#cb1147-3" aria-hidden="true" tabindex="-1"></a>  y <span class="ot">=</span> <span class="dv">3</span> <span class="sc">+</span> x <span class="sc">-</span> <span class="dv">4</span> <span class="sc">*</span> x <span class="sc">^</span> <span class="dv">2</span> <span class="sc">+</span> <span class="fu">rnorm</span>(<span class="at">n =</span> <span class="dv">11</span>, <span class="at">mean =</span> <span class="dv">0</span>, <span class="at">sd =</span> <span class="dv">25</span>)</span>
-<span id="cb1147-4"><a href="transformations.html#cb1147-4" aria-hidden="true" tabindex="-1"></a>  <span class="fu">data.frame</span>(x, y)</span>
-<span id="cb1147-5"><a href="transformations.html#cb1147-5" aria-hidden="true" tabindex="-1"></a>}</span></code></pre></div>
-<div class="sourceCode" id="cb1148"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1148-1"><a href="transformations.html#cb1148-1" aria-hidden="true" tabindex="-1"></a><span class="fu">set.seed</span>(<span class="dv">1234</span>)</span>
-<span id="cb1148-2"><a href="transformations.html#cb1148-2" aria-hidden="true" tabindex="-1"></a>data_for_perf <span class="ot">=</span> <span class="fu">sim_for_perf</span>()</span></code></pre></div>
-<div class="sourceCode" id="cb1149"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1149-1"><a href="transformations.html#cb1149-1" aria-hidden="true" tabindex="-1"></a>fit_correct <span class="ot">=</span> <span class="fu">lm</span>(y <span class="sc">~</span> x <span class="sc">+</span> <span class="fu">I</span>(x <span class="sc">^</span> <span class="dv">2</span>), <span class="at">data =</span> data_for_perf)</span>
-<span id="cb1149-2"><a href="transformations.html#cb1149-2" aria-hidden="true" tabindex="-1"></a>fit_perfect <span class="ot">=</span> <span class="fu">lm</span>(y <span class="sc">~</span> x <span class="sc">+</span> <span class="fu">I</span>(x <span class="sc">^</span> <span class="dv">2</span>) <span class="sc">+</span> <span class="fu">I</span>(x <span class="sc">^</span> <span class="dv">3</span>) <span class="sc">+</span> <span class="fu">I</span>(x <span class="sc">^</span> <span class="dv">4</span>) <span class="sc">+</span> <span class="fu">I</span>(x <span class="sc">^</span> <span class="dv">5</span>) <span class="sc">+</span> <span class="fu">I</span>(x <span class="sc">^</span> <span class="dv">6</span>) <span class="sc">+</span> </span>
-<span id="cb1149-3"><a href="transformations.html#cb1149-3" aria-hidden="true" tabindex="-1"></a>                 <span class="fu">I</span>(x <span class="sc">^</span> <span class="dv">7</span>) <span class="sc">+</span> <span class="fu">I</span>(x <span class="sc">^</span> <span class="dv">8</span>) <span class="sc">+</span> <span class="fu">I</span>(x <span class="sc">^</span> <span class="dv">9</span>) <span class="sc">+</span> <span class="fu">I</span>(x <span class="sc">^</span> <span class="dv">10</span>), </span>
-<span id="cb1149-4"><a href="transformations.html#cb1149-4" aria-hidden="true" tabindex="-1"></a>                 <span class="at">data =</span> data_for_perf)</span></code></pre></div>
-<div class="sourceCode" id="cb1150"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1150-1"><a href="transformations.html#cb1150-1" aria-hidden="true" tabindex="-1"></a>x_plot <span class="ot">=</span> <span class="fu">seq</span>(<span class="sc">-</span><span class="dv">5</span>, <span class="dv">15</span>, <span class="at">by =</span> <span class="fl">0.1</span>)</span>
-<span id="cb1150-2"><a href="transformations.html#cb1150-2" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(y <span class="sc">~</span> x, <span class="at">data =</span> data_for_perf, <span class="at">ylim =</span> <span class="fu">c</span>(<span class="sc">-</span><span class="dv">450</span>, <span class="dv">100</span>), <span class="at">cex =</span> <span class="dv">2</span>, <span class="at">pch =</span> <span class="dv">20</span>)</span>
-<span id="cb1150-3"><a href="transformations.html#cb1150-3" aria-hidden="true" tabindex="-1"></a><span class="fu">lines</span>(x_plot, <span class="fu">predict</span>(fit_correct, <span class="at">newdata =</span> <span class="fu">data.frame</span>(<span class="at">x =</span> x_plot)),</span>
-<span id="cb1150-4"><a href="transformations.html#cb1150-4" aria-hidden="true" tabindex="-1"></a>      <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>, <span class="at">lty =</span> <span class="dv">1</span>)</span>
-<span id="cb1150-5"><a href="transformations.html#cb1150-5" aria-hidden="true" tabindex="-1"></a><span class="fu">lines</span>(x_plot, <span class="fu">predict</span>(fit_perfect, <span class="at">newdata =</span> <span class="fu">data.frame</span>(<span class="at">x =</span> x_plot)),</span>
-<span id="cb1150-6"><a href="transformations.html#cb1150-6" aria-hidden="true" tabindex="-1"></a>      <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>, <span class="at">lty =</span> <span class="dv">2</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1147"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1147-1"><a href="transformations.html#cb1147-1" tabindex="-1"></a>sim_for_perf <span class="ot">=</span> <span class="cf">function</span>() {</span>
+<span id="cb1147-2"><a href="transformations.html#cb1147-2" tabindex="-1"></a>  x <span class="ot">=</span> <span class="fu">seq</span>(<span class="dv">0</span>, <span class="dv">10</span>)</span>
+<span id="cb1147-3"><a href="transformations.html#cb1147-3" tabindex="-1"></a>  y <span class="ot">=</span> <span class="dv">3</span> <span class="sc">+</span> x <span class="sc">-</span> <span class="dv">4</span> <span class="sc">*</span> x <span class="sc">^</span> <span class="dv">2</span> <span class="sc">+</span> <span class="fu">rnorm</span>(<span class="at">n =</span> <span class="dv">11</span>, <span class="at">mean =</span> <span class="dv">0</span>, <span class="at">sd =</span> <span class="dv">25</span>)</span>
+<span id="cb1147-4"><a href="transformations.html#cb1147-4" tabindex="-1"></a>  <span class="fu">data.frame</span>(x, y)</span>
+<span id="cb1147-5"><a href="transformations.html#cb1147-5" tabindex="-1"></a>}</span></code></pre></div>
+<div class="sourceCode" id="cb1148"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1148-1"><a href="transformations.html#cb1148-1" tabindex="-1"></a><span class="fu">set.seed</span>(<span class="dv">1234</span>)</span>
+<span id="cb1148-2"><a href="transformations.html#cb1148-2" tabindex="-1"></a>data_for_perf <span class="ot">=</span> <span class="fu">sim_for_perf</span>()</span></code></pre></div>
+<div class="sourceCode" id="cb1149"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1149-1"><a href="transformations.html#cb1149-1" tabindex="-1"></a>fit_correct <span class="ot">=</span> <span class="fu">lm</span>(y <span class="sc">~</span> x <span class="sc">+</span> <span class="fu">I</span>(x <span class="sc">^</span> <span class="dv">2</span>), <span class="at">data =</span> data_for_perf)</span>
+<span id="cb1149-2"><a href="transformations.html#cb1149-2" tabindex="-1"></a>fit_perfect <span class="ot">=</span> <span class="fu">lm</span>(y <span class="sc">~</span> x <span class="sc">+</span> <span class="fu">I</span>(x <span class="sc">^</span> <span class="dv">2</span>) <span class="sc">+</span> <span class="fu">I</span>(x <span class="sc">^</span> <span class="dv">3</span>) <span class="sc">+</span> <span class="fu">I</span>(x <span class="sc">^</span> <span class="dv">4</span>) <span class="sc">+</span> <span class="fu">I</span>(x <span class="sc">^</span> <span class="dv">5</span>) <span class="sc">+</span> <span class="fu">I</span>(x <span class="sc">^</span> <span class="dv">6</span>) <span class="sc">+</span> </span>
+<span id="cb1149-3"><a href="transformations.html#cb1149-3" tabindex="-1"></a>                 <span class="fu">I</span>(x <span class="sc">^</span> <span class="dv">7</span>) <span class="sc">+</span> <span class="fu">I</span>(x <span class="sc">^</span> <span class="dv">8</span>) <span class="sc">+</span> <span class="fu">I</span>(x <span class="sc">^</span> <span class="dv">9</span>) <span class="sc">+</span> <span class="fu">I</span>(x <span class="sc">^</span> <span class="dv">10</span>), </span>
+<span id="cb1149-4"><a href="transformations.html#cb1149-4" tabindex="-1"></a>                 <span class="at">data =</span> data_for_perf)</span></code></pre></div>
+<div class="sourceCode" id="cb1150"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1150-1"><a href="transformations.html#cb1150-1" tabindex="-1"></a>x_plot <span class="ot">=</span> <span class="fu">seq</span>(<span class="sc">-</span><span class="dv">5</span>, <span class="dv">15</span>, <span class="at">by =</span> <span class="fl">0.1</span>)</span>
+<span id="cb1150-2"><a href="transformations.html#cb1150-2" tabindex="-1"></a><span class="fu">plot</span>(y <span class="sc">~</span> x, <span class="at">data =</span> data_for_perf, <span class="at">ylim =</span> <span class="fu">c</span>(<span class="sc">-</span><span class="dv">450</span>, <span class="dv">100</span>), <span class="at">cex =</span> <span class="dv">2</span>, <span class="at">pch =</span> <span class="dv">20</span>)</span>
+<span id="cb1150-3"><a href="transformations.html#cb1150-3" tabindex="-1"></a><span class="fu">lines</span>(x_plot, <span class="fu">predict</span>(fit_correct, <span class="at">newdata =</span> <span class="fu">data.frame</span>(<span class="at">x =</span> x_plot)),</span>
+<span id="cb1150-4"><a href="transformations.html#cb1150-4" tabindex="-1"></a>      <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>, <span class="at">lty =</span> <span class="dv">1</span>)</span>
+<span id="cb1150-5"><a href="transformations.html#cb1150-5" tabindex="-1"></a><span class="fu">lines</span>(x_plot, <span class="fu">predict</span>(fit_perfect, <span class="at">newdata =</span> <span class="fu">data.frame</span>(<span class="at">x =</span> x_plot)),</span>
+<span id="cb1150-6"><a href="transformations.html#cb1150-6" tabindex="-1"></a>      <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>, <span class="at">lty =</span> <span class="dv">2</span>)</span></code></pre></div>
 <p><img src="transformations_files/figure-html/unnamed-chunk-83-1.png" width="768" style="display: block; margin: auto;" /></p>
 </div>
 <div id="comparing-polynomial-models" class="section level3 hasAnchor" number="14.2.4">
 <h3><span class="header-section-number">14.2.4</span> Comparing Polynomial Models<a href="transformations.html#comparing-polynomial-models" class="anchor-section" aria-label="Anchor link to header"></a></h3>
-<div class="sourceCode" id="cb1151"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1151-1"><a href="transformations.html#cb1151-1" aria-hidden="true" tabindex="-1"></a>sim_higher <span class="ot">=</span> <span class="cf">function</span>(<span class="at">sample_size =</span> <span class="dv">250</span>) {</span>
-<span id="cb1151-2"><a href="transformations.html#cb1151-2" aria-hidden="true" tabindex="-1"></a>  x <span class="ot">=</span> <span class="fu">runif</span>(<span class="at">n =</span> sample_size, <span class="at">min =</span> <span class="sc">-</span><span class="dv">1</span>, <span class="at">max =</span> <span class="dv">1</span>) <span class="sc">*</span> <span class="dv">2</span></span>
-<span id="cb1151-3"><a href="transformations.html#cb1151-3" aria-hidden="true" tabindex="-1"></a>  y <span class="ot">=</span> <span class="dv">3</span> <span class="sc">+</span> <span class="sc">-</span><span class="dv">6</span> <span class="sc">*</span> x <span class="sc">^</span> <span class="dv">2</span> <span class="sc">+</span> <span class="dv">1</span> <span class="sc">*</span> x <span class="sc">^</span> <span class="dv">4</span> <span class="sc">+</span> <span class="fu">rnorm</span>(<span class="at">n =</span> sample_size, <span class="at">mean =</span> <span class="dv">0</span>, <span class="at">sd =</span> <span class="dv">3</span>)</span>
-<span id="cb1151-4"><a href="transformations.html#cb1151-4" aria-hidden="true" tabindex="-1"></a>  <span class="fu">data.frame</span>(x, y)</span>
-<span id="cb1151-5"><a href="transformations.html#cb1151-5" aria-hidden="true" tabindex="-1"></a>}</span></code></pre></div>
+<div class="sourceCode" id="cb1151"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1151-1"><a href="transformations.html#cb1151-1" tabindex="-1"></a>sim_higher <span class="ot">=</span> <span class="cf">function</span>(<span class="at">sample_size =</span> <span class="dv">250</span>) {</span>
+<span id="cb1151-2"><a href="transformations.html#cb1151-2" tabindex="-1"></a>  x <span class="ot">=</span> <span class="fu">runif</span>(<span class="at">n =</span> sample_size, <span class="at">min =</span> <span class="sc">-</span><span class="dv">1</span>, <span class="at">max =</span> <span class="dv">1</span>) <span class="sc">*</span> <span class="dv">2</span></span>
+<span id="cb1151-3"><a href="transformations.html#cb1151-3" tabindex="-1"></a>  y <span class="ot">=</span> <span class="dv">3</span> <span class="sc">+</span> <span class="sc">-</span><span class="dv">6</span> <span class="sc">*</span> x <span class="sc">^</span> <span class="dv">2</span> <span class="sc">+</span> <span class="dv">1</span> <span class="sc">*</span> x <span class="sc">^</span> <span class="dv">4</span> <span class="sc">+</span> <span class="fu">rnorm</span>(<span class="at">n =</span> sample_size, <span class="at">mean =</span> <span class="dv">0</span>, <span class="at">sd =</span> <span class="dv">3</span>)</span>
+<span id="cb1151-4"><a href="transformations.html#cb1151-4" tabindex="-1"></a>  <span class="fu">data.frame</span>(x, y)</span>
+<span id="cb1151-5"><a href="transformations.html#cb1151-5" tabindex="-1"></a>}</span></code></pre></div>
 <p><span class="math display">\[
 Y_i = \beta_0 + \beta_1 x_i + \beta_2 x_i^2 + \epsilon_i
 \]</span></p>
@@ -1441,40 +1441,40 @@ <h3><span class="header-section-number">14.2.4</span> Comparing Polynomial Model
 <p><span class="math display">\[
 Y_i = \beta_0 + \beta_1 x_i + \beta_2 x_i^2 + \beta_3 x_i^3 + \beta_4 x_i^4 + \beta_5 x_i^5 + \beta_6 x_i^6 + \epsilon_i
 \]</span></p>
-<div class="sourceCode" id="cb1152"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1152-1"><a href="transformations.html#cb1152-1" aria-hidden="true" tabindex="-1"></a><span class="fu">set.seed</span>(<span class="dv">42</span>)</span>
-<span id="cb1152-2"><a href="transformations.html#cb1152-2" aria-hidden="true" tabindex="-1"></a>data_higher <span class="ot">=</span> <span class="fu">sim_higher</span>()</span></code></pre></div>
-<div class="sourceCode" id="cb1153"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1153-1"><a href="transformations.html#cb1153-1" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(y <span class="sc">~</span> x, <span class="at">data =</span> data_higher, <span class="at">col =</span> <span class="st">&quot;grey&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">cex =</span> <span class="fl">1.5</span>,</span>
-<span id="cb1153-2"><a href="transformations.html#cb1153-2" aria-hidden="true" tabindex="-1"></a>     <span class="at">main =</span> <span class="st">&quot;Simulated Quartic Data&quot;</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1152"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1152-1"><a href="transformations.html#cb1152-1" tabindex="-1"></a><span class="fu">set.seed</span>(<span class="dv">42</span>)</span>
+<span id="cb1152-2"><a href="transformations.html#cb1152-2" tabindex="-1"></a>data_higher <span class="ot">=</span> <span class="fu">sim_higher</span>()</span></code></pre></div>
+<div class="sourceCode" id="cb1153"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1153-1"><a href="transformations.html#cb1153-1" tabindex="-1"></a><span class="fu">plot</span>(y <span class="sc">~</span> x, <span class="at">data =</span> data_higher, <span class="at">col =</span> <span class="st">&quot;grey&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">cex =</span> <span class="fl">1.5</span>,</span>
+<span id="cb1153-2"><a href="transformations.html#cb1153-2" tabindex="-1"></a>     <span class="at">main =</span> <span class="st">&quot;Simulated Quartic Data&quot;</span>)</span></code></pre></div>
 <p><img src="transformations_files/figure-html/unnamed-chunk-86-1.png" width="672" style="display: block; margin: auto;" /></p>
-<div class="sourceCode" id="cb1154"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1154-1"><a href="transformations.html#cb1154-1" aria-hidden="true" tabindex="-1"></a>fit_2 <span class="ot">=</span> <span class="fu">lm</span>(y <span class="sc">~</span> <span class="fu">poly</span>(x, <span class="dv">2</span>), <span class="at">data =</span> data_higher)</span>
-<span id="cb1154-2"><a href="transformations.html#cb1154-2" aria-hidden="true" tabindex="-1"></a>fit_4 <span class="ot">=</span> <span class="fu">lm</span>(y <span class="sc">~</span> <span class="fu">poly</span>(x, <span class="dv">4</span>), <span class="at">data =</span> data_higher)</span></code></pre></div>
-<div class="sourceCode" id="cb1155"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1155-1"><a href="transformations.html#cb1155-1" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(y <span class="sc">~</span> x, <span class="at">data =</span> data_higher, <span class="at">col =</span> <span class="st">&quot;grey&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">cex =</span> <span class="fl">1.5</span>,</span>
-<span id="cb1155-2"><a href="transformations.html#cb1155-2" aria-hidden="true" tabindex="-1"></a>     <span class="at">main =</span> <span class="st">&quot;Simulated Quartic Data&quot;</span>)</span>
-<span id="cb1155-3"><a href="transformations.html#cb1155-3" aria-hidden="true" tabindex="-1"></a>x_plot <span class="ot">=</span> <span class="fu">seq</span>(<span class="sc">-</span><span class="dv">5</span>, <span class="dv">5</span>, <span class="at">by =</span> <span class="fl">0.05</span>)</span>
-<span id="cb1155-4"><a href="transformations.html#cb1155-4" aria-hidden="true" tabindex="-1"></a><span class="fu">lines</span>(x_plot, <span class="fu">predict</span>(fit_2, <span class="at">newdata =</span> <span class="fu">data.frame</span>(<span class="at">x =</span> x_plot)),</span>
-<span id="cb1155-5"><a href="transformations.html#cb1155-5" aria-hidden="true" tabindex="-1"></a>      <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>, <span class="at">lty =</span> <span class="dv">1</span>)</span>
-<span id="cb1155-6"><a href="transformations.html#cb1155-6" aria-hidden="true" tabindex="-1"></a><span class="fu">lines</span>(x_plot, <span class="fu">predict</span>(fit_4, <span class="at">newdata =</span> <span class="fu">data.frame</span>(<span class="at">x =</span> x_plot)),</span>
-<span id="cb1155-7"><a href="transformations.html#cb1155-7" aria-hidden="true" tabindex="-1"></a>      <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>, <span class="at">lty =</span> <span class="dv">2</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1154"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1154-1"><a href="transformations.html#cb1154-1" tabindex="-1"></a>fit_2 <span class="ot">=</span> <span class="fu">lm</span>(y <span class="sc">~</span> <span class="fu">poly</span>(x, <span class="dv">2</span>), <span class="at">data =</span> data_higher)</span>
+<span id="cb1154-2"><a href="transformations.html#cb1154-2" tabindex="-1"></a>fit_4 <span class="ot">=</span> <span class="fu">lm</span>(y <span class="sc">~</span> <span class="fu">poly</span>(x, <span class="dv">4</span>), <span class="at">data =</span> data_higher)</span></code></pre></div>
+<div class="sourceCode" id="cb1155"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1155-1"><a href="transformations.html#cb1155-1" tabindex="-1"></a><span class="fu">plot</span>(y <span class="sc">~</span> x, <span class="at">data =</span> data_higher, <span class="at">col =</span> <span class="st">&quot;grey&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">cex =</span> <span class="fl">1.5</span>,</span>
+<span id="cb1155-2"><a href="transformations.html#cb1155-2" tabindex="-1"></a>     <span class="at">main =</span> <span class="st">&quot;Simulated Quartic Data&quot;</span>)</span>
+<span id="cb1155-3"><a href="transformations.html#cb1155-3" tabindex="-1"></a>x_plot <span class="ot">=</span> <span class="fu">seq</span>(<span class="sc">-</span><span class="dv">5</span>, <span class="dv">5</span>, <span class="at">by =</span> <span class="fl">0.05</span>)</span>
+<span id="cb1155-4"><a href="transformations.html#cb1155-4" tabindex="-1"></a><span class="fu">lines</span>(x_plot, <span class="fu">predict</span>(fit_2, <span class="at">newdata =</span> <span class="fu">data.frame</span>(<span class="at">x =</span> x_plot)),</span>
+<span id="cb1155-5"><a href="transformations.html#cb1155-5" tabindex="-1"></a>      <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>, <span class="at">lty =</span> <span class="dv">1</span>)</span>
+<span id="cb1155-6"><a href="transformations.html#cb1155-6" tabindex="-1"></a><span class="fu">lines</span>(x_plot, <span class="fu">predict</span>(fit_4, <span class="at">newdata =</span> <span class="fu">data.frame</span>(<span class="at">x =</span> x_plot)),</span>
+<span id="cb1155-7"><a href="transformations.html#cb1155-7" tabindex="-1"></a>      <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>, <span class="at">lty =</span> <span class="dv">2</span>)</span></code></pre></div>
 <p><img src="transformations_files/figure-html/unnamed-chunk-88-1.png" width="672" style="display: block; margin: auto;" /></p>
-<div class="sourceCode" id="cb1156"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1156-1"><a href="transformations.html#cb1156-1" aria-hidden="true" tabindex="-1"></a><span class="fu">par</span>(<span class="at">mfrow =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>))</span>
-<span id="cb1156-2"><a href="transformations.html#cb1156-2" aria-hidden="true" tabindex="-1"></a></span>
-<span id="cb1156-3"><a href="transformations.html#cb1156-3" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(<span class="fu">fitted</span>(fit_2), <span class="fu">resid</span>(fit_2), <span class="at">col =</span> <span class="st">&quot;grey&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>,</span>
-<span id="cb1156-4"><a href="transformations.html#cb1156-4" aria-hidden="true" tabindex="-1"></a>     <span class="at">xlab =</span> <span class="st">&quot;Fitted&quot;</span>, <span class="at">ylab =</span> <span class="st">&quot;Residuals&quot;</span>, <span class="at">main =</span> <span class="st">&quot;Fitted versus Residuals&quot;</span>)</span>
-<span id="cb1156-5"><a href="transformations.html#cb1156-5" aria-hidden="true" tabindex="-1"></a><span class="fu">abline</span>(<span class="at">h =</span> <span class="dv">0</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span>
-<span id="cb1156-6"><a href="transformations.html#cb1156-6" aria-hidden="true" tabindex="-1"></a></span>
-<span id="cb1156-7"><a href="transformations.html#cb1156-7" aria-hidden="true" tabindex="-1"></a><span class="fu">qqnorm</span>(<span class="fu">resid</span>(fit_2), <span class="at">main =</span> <span class="st">&quot;Normal Q-Q Plot&quot;</span>, <span class="at">col =</span> <span class="st">&quot;darkgrey&quot;</span>)</span>
-<span id="cb1156-8"><a href="transformations.html#cb1156-8" aria-hidden="true" tabindex="-1"></a><span class="fu">qqline</span>(<span class="fu">resid</span>(fit_2), <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1156"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1156-1"><a href="transformations.html#cb1156-1" tabindex="-1"></a><span class="fu">par</span>(<span class="at">mfrow =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>))</span>
+<span id="cb1156-2"><a href="transformations.html#cb1156-2" tabindex="-1"></a></span>
+<span id="cb1156-3"><a href="transformations.html#cb1156-3" tabindex="-1"></a><span class="fu">plot</span>(<span class="fu">fitted</span>(fit_2), <span class="fu">resid</span>(fit_2), <span class="at">col =</span> <span class="st">&quot;grey&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>,</span>
+<span id="cb1156-4"><a href="transformations.html#cb1156-4" tabindex="-1"></a>     <span class="at">xlab =</span> <span class="st">&quot;Fitted&quot;</span>, <span class="at">ylab =</span> <span class="st">&quot;Residuals&quot;</span>, <span class="at">main =</span> <span class="st">&quot;Fitted versus Residuals&quot;</span>)</span>
+<span id="cb1156-5"><a href="transformations.html#cb1156-5" tabindex="-1"></a><span class="fu">abline</span>(<span class="at">h =</span> <span class="dv">0</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span>
+<span id="cb1156-6"><a href="transformations.html#cb1156-6" tabindex="-1"></a></span>
+<span id="cb1156-7"><a href="transformations.html#cb1156-7" tabindex="-1"></a><span class="fu">qqnorm</span>(<span class="fu">resid</span>(fit_2), <span class="at">main =</span> <span class="st">&quot;Normal Q-Q Plot&quot;</span>, <span class="at">col =</span> <span class="st">&quot;darkgrey&quot;</span>)</span>
+<span id="cb1156-8"><a href="transformations.html#cb1156-8" tabindex="-1"></a><span class="fu">qqline</span>(<span class="fu">resid</span>(fit_2), <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
 <p><img src="transformations_files/figure-html/unnamed-chunk-89-1.png" width="960" style="display: block; margin: auto;" /></p>
-<div class="sourceCode" id="cb1157"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1157-1"><a href="transformations.html#cb1157-1" aria-hidden="true" tabindex="-1"></a><span class="fu">par</span>(<span class="at">mfrow =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>))</span>
-<span id="cb1157-2"><a href="transformations.html#cb1157-2" aria-hidden="true" tabindex="-1"></a></span>
-<span id="cb1157-3"><a href="transformations.html#cb1157-3" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(<span class="fu">fitted</span>(fit_4), <span class="fu">resid</span>(fit_4), <span class="at">col =</span> <span class="st">&quot;grey&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>,</span>
-<span id="cb1157-4"><a href="transformations.html#cb1157-4" aria-hidden="true" tabindex="-1"></a>     <span class="at">xlab =</span> <span class="st">&quot;Fitted&quot;</span>, <span class="at">ylab =</span> <span class="st">&quot;Residuals&quot;</span>, <span class="at">main =</span> <span class="st">&quot;Fitted versus Residuals&quot;</span>)</span>
-<span id="cb1157-5"><a href="transformations.html#cb1157-5" aria-hidden="true" tabindex="-1"></a><span class="fu">abline</span>(<span class="at">h =</span> <span class="dv">0</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span>
-<span id="cb1157-6"><a href="transformations.html#cb1157-6" aria-hidden="true" tabindex="-1"></a></span>
-<span id="cb1157-7"><a href="transformations.html#cb1157-7" aria-hidden="true" tabindex="-1"></a><span class="fu">qqnorm</span>(<span class="fu">resid</span>(fit_4), <span class="at">main =</span> <span class="st">&quot;Normal Q-Q Plot&quot;</span>, <span class="at">col =</span> <span class="st">&quot;darkgrey&quot;</span>)</span>
-<span id="cb1157-8"><a href="transformations.html#cb1157-8" aria-hidden="true" tabindex="-1"></a><span class="fu">qqline</span>(<span class="fu">resid</span>(fit_4), <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1157"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1157-1"><a href="transformations.html#cb1157-1" tabindex="-1"></a><span class="fu">par</span>(<span class="at">mfrow =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>))</span>
+<span id="cb1157-2"><a href="transformations.html#cb1157-2" tabindex="-1"></a></span>
+<span id="cb1157-3"><a href="transformations.html#cb1157-3" tabindex="-1"></a><span class="fu">plot</span>(<span class="fu">fitted</span>(fit_4), <span class="fu">resid</span>(fit_4), <span class="at">col =</span> <span class="st">&quot;grey&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>,</span>
+<span id="cb1157-4"><a href="transformations.html#cb1157-4" tabindex="-1"></a>     <span class="at">xlab =</span> <span class="st">&quot;Fitted&quot;</span>, <span class="at">ylab =</span> <span class="st">&quot;Residuals&quot;</span>, <span class="at">main =</span> <span class="st">&quot;Fitted versus Residuals&quot;</span>)</span>
+<span id="cb1157-5"><a href="transformations.html#cb1157-5" tabindex="-1"></a><span class="fu">abline</span>(<span class="at">h =</span> <span class="dv">0</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span>
+<span id="cb1157-6"><a href="transformations.html#cb1157-6" tabindex="-1"></a></span>
+<span id="cb1157-7"><a href="transformations.html#cb1157-7" tabindex="-1"></a><span class="fu">qqnorm</span>(<span class="fu">resid</span>(fit_4), <span class="at">main =</span> <span class="st">&quot;Normal Q-Q Plot&quot;</span>, <span class="at">col =</span> <span class="st">&quot;darkgrey&quot;</span>)</span>
+<span id="cb1157-8"><a href="transformations.html#cb1157-8" tabindex="-1"></a><span class="fu">qqline</span>(<span class="fu">resid</span>(fit_4), <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
 <p><img src="transformations_files/figure-html/unnamed-chunk-90-1.png" width="960" style="display: block; margin: auto;" /></p>
-<div class="sourceCode" id="cb1158"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1158-1"><a href="transformations.html#cb1158-1" aria-hidden="true" tabindex="-1"></a><span class="fu">anova</span>(fit_2, fit_4)</span></code></pre></div>
+<div class="sourceCode" id="cb1158"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1158-1"><a href="transformations.html#cb1158-1" tabindex="-1"></a><span class="fu">anova</span>(fit_2, fit_4)</span></code></pre></div>
 <pre><code>## Analysis of Variance Table
 ## 
 ## Model 1: y ~ poly(x, 2)
@@ -1484,8 +1484,8 @@ <h3><span class="header-section-number">14.2.4</span> Comparing Polynomial Model
 ## 2    245 1912.6  2    421.51 26.997 2.536e-11 ***
 ## ---
 ## Signif. codes:  0 &#39;***&#39; 0.001 &#39;**&#39; 0.01 &#39;*&#39; 0.05 &#39;.&#39; 0.1 &#39; &#39; 1</code></pre>
-<div class="sourceCode" id="cb1160"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1160-1"><a href="transformations.html#cb1160-1" aria-hidden="true" tabindex="-1"></a>fit_6 <span class="ot">=</span> <span class="fu">lm</span>(y <span class="sc">~</span> <span class="fu">poly</span>(x, <span class="dv">6</span>), <span class="at">data =</span> data_higher)</span></code></pre></div>
-<div class="sourceCode" id="cb1161"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1161-1"><a href="transformations.html#cb1161-1" aria-hidden="true" tabindex="-1"></a><span class="fu">anova</span>(fit_4, fit_6)</span></code></pre></div>
+<div class="sourceCode" id="cb1160"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1160-1"><a href="transformations.html#cb1160-1" tabindex="-1"></a>fit_6 <span class="ot">=</span> <span class="fu">lm</span>(y <span class="sc">~</span> <span class="fu">poly</span>(x, <span class="dv">6</span>), <span class="at">data =</span> data_higher)</span></code></pre></div>
+<div class="sourceCode" id="cb1161"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1161-1"><a href="transformations.html#cb1161-1" tabindex="-1"></a><span class="fu">anova</span>(fit_4, fit_6)</span></code></pre></div>
 <pre><code>## Analysis of Variance Table
 ## 
 ## Model 1: y ~ poly(x, 4)
@@ -1499,37 +1499,37 @@ <h3><span class="header-section-number">14.2.5</span> <code>poly()</code> Functi
 <p><span class="math display">\[
 Y_i = \beta_0 + \beta_1 x_i + \beta_2 x_i^2 + \beta_3 x_i^3 + \beta_4 x_i^4 + \epsilon_i
 \]</span></p>
-<div class="sourceCode" id="cb1163"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1163-1"><a href="transformations.html#cb1163-1" aria-hidden="true" tabindex="-1"></a>fit_4a <span class="ot">=</span> <span class="fu">lm</span>(y <span class="sc">~</span> <span class="fu">poly</span>(x, <span class="at">degree =</span> <span class="dv">4</span>), <span class="at">data =</span> data_higher)</span>
-<span id="cb1163-2"><a href="transformations.html#cb1163-2" aria-hidden="true" tabindex="-1"></a>fit_4b <span class="ot">=</span> <span class="fu">lm</span>(y <span class="sc">~</span> <span class="fu">poly</span>(x, <span class="at">degree =</span> <span class="dv">4</span>, <span class="at">raw =</span> <span class="cn">TRUE</span>), <span class="at">data =</span> data_higher)</span>
-<span id="cb1163-3"><a href="transformations.html#cb1163-3" aria-hidden="true" tabindex="-1"></a>fit_4c <span class="ot">=</span> <span class="fu">lm</span>(y <span class="sc">~</span> x <span class="sc">+</span> <span class="fu">I</span>(x<span class="sc">^</span><span class="dv">2</span>) <span class="sc">+</span> <span class="fu">I</span>(x<span class="sc">^</span><span class="dv">3</span>) <span class="sc">+</span> <span class="fu">I</span>(x<span class="sc">^</span><span class="dv">4</span>), <span class="at">data =</span> data_higher)</span></code></pre></div>
-<div class="sourceCode" id="cb1164"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1164-1"><a href="transformations.html#cb1164-1" aria-hidden="true" tabindex="-1"></a><span class="fu">coef</span>(fit_4a)</span></code></pre></div>
+<div class="sourceCode" id="cb1163"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1163-1"><a href="transformations.html#cb1163-1" tabindex="-1"></a>fit_4a <span class="ot">=</span> <span class="fu">lm</span>(y <span class="sc">~</span> <span class="fu">poly</span>(x, <span class="at">degree =</span> <span class="dv">4</span>), <span class="at">data =</span> data_higher)</span>
+<span id="cb1163-2"><a href="transformations.html#cb1163-2" tabindex="-1"></a>fit_4b <span class="ot">=</span> <span class="fu">lm</span>(y <span class="sc">~</span> <span class="fu">poly</span>(x, <span class="at">degree =</span> <span class="dv">4</span>, <span class="at">raw =</span> <span class="cn">TRUE</span>), <span class="at">data =</span> data_higher)</span>
+<span id="cb1163-3"><a href="transformations.html#cb1163-3" tabindex="-1"></a>fit_4c <span class="ot">=</span> <span class="fu">lm</span>(y <span class="sc">~</span> x <span class="sc">+</span> <span class="fu">I</span>(x<span class="sc">^</span><span class="dv">2</span>) <span class="sc">+</span> <span class="fu">I</span>(x<span class="sc">^</span><span class="dv">3</span>) <span class="sc">+</span> <span class="fu">I</span>(x<span class="sc">^</span><span class="dv">4</span>), <span class="at">data =</span> data_higher)</span></code></pre></div>
+<div class="sourceCode" id="cb1164"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1164-1"><a href="transformations.html#cb1164-1" tabindex="-1"></a><span class="fu">coef</span>(fit_4a)</span></code></pre></div>
 <pre><code>##          (Intercept) poly(x, degree = 4)1 poly(x, degree = 4)2 
 ##            -1.980036            -2.053929           -49.344752 
 ## poly(x, degree = 4)3 poly(x, degree = 4)4 
 ##             0.669874            20.519759</code></pre>
-<div class="sourceCode" id="cb1166"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1166-1"><a href="transformations.html#cb1166-1" aria-hidden="true" tabindex="-1"></a><span class="fu">coef</span>(fit_4b)</span></code></pre></div>
+<div class="sourceCode" id="cb1166"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1166-1"><a href="transformations.html#cb1166-1" tabindex="-1"></a><span class="fu">coef</span>(fit_4b)</span></code></pre></div>
 <pre><code>##                      (Intercept) poly(x, degree = 4, raw = TRUE)1 
 ##                        2.9996256                       -0.3880250 
 ## poly(x, degree = 4, raw = TRUE)2 poly(x, degree = 4, raw = TRUE)3 
 ##                       -6.1511166                        0.1269046 
 ## poly(x, degree = 4, raw = TRUE)4 
 ##                        1.0282139</code></pre>
-<div class="sourceCode" id="cb1168"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1168-1"><a href="transformations.html#cb1168-1" aria-hidden="true" tabindex="-1"></a><span class="fu">coef</span>(fit_4c)</span></code></pre></div>
+<div class="sourceCode" id="cb1168"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1168-1"><a href="transformations.html#cb1168-1" tabindex="-1"></a><span class="fu">coef</span>(fit_4c)</span></code></pre></div>
 <pre><code>## (Intercept)           x      I(x^2)      I(x^3)      I(x^4) 
 ##   2.9996256  -0.3880250  -6.1511166   0.1269046   1.0282139</code></pre>
-<div class="sourceCode" id="cb1170"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1170-1"><a href="transformations.html#cb1170-1" aria-hidden="true" tabindex="-1"></a><span class="fu">unname</span>(<span class="fu">coef</span>(fit_4a))</span></code></pre></div>
+<div class="sourceCode" id="cb1170"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1170-1"><a href="transformations.html#cb1170-1" tabindex="-1"></a><span class="fu">unname</span>(<span class="fu">coef</span>(fit_4a))</span></code></pre></div>
 <pre><code>## [1]  -1.980036  -2.053929 -49.344752   0.669874  20.519759</code></pre>
-<div class="sourceCode" id="cb1172"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1172-1"><a href="transformations.html#cb1172-1" aria-hidden="true" tabindex="-1"></a><span class="fu">unname</span>(<span class="fu">coef</span>(fit_4b))</span></code></pre></div>
+<div class="sourceCode" id="cb1172"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1172-1"><a href="transformations.html#cb1172-1" tabindex="-1"></a><span class="fu">unname</span>(<span class="fu">coef</span>(fit_4b))</span></code></pre></div>
 <pre><code>## [1]  2.9996256 -0.3880250 -6.1511166  0.1269046  1.0282139</code></pre>
-<div class="sourceCode" id="cb1174"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1174-1"><a href="transformations.html#cb1174-1" aria-hidden="true" tabindex="-1"></a><span class="fu">unname</span>(<span class="fu">coef</span>(fit_4c))</span></code></pre></div>
+<div class="sourceCode" id="cb1174"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1174-1"><a href="transformations.html#cb1174-1" tabindex="-1"></a><span class="fu">unname</span>(<span class="fu">coef</span>(fit_4c))</span></code></pre></div>
 <pre><code>## [1]  2.9996256 -0.3880250 -6.1511166  0.1269046  1.0282139</code></pre>
-<div class="sourceCode" id="cb1176"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1176-1"><a href="transformations.html#cb1176-1" aria-hidden="true" tabindex="-1"></a><span class="fu">all.equal</span>(<span class="fu">fitted</span>(fit_4a),</span>
-<span id="cb1176-2"><a href="transformations.html#cb1176-2" aria-hidden="true" tabindex="-1"></a>          <span class="fu">fitted</span>(fit_4b))</span></code></pre></div>
+<div class="sourceCode" id="cb1176"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1176-1"><a href="transformations.html#cb1176-1" tabindex="-1"></a><span class="fu">all.equal</span>(<span class="fu">fitted</span>(fit_4a),</span>
+<span id="cb1176-2"><a href="transformations.html#cb1176-2" tabindex="-1"></a>          <span class="fu">fitted</span>(fit_4b))</span></code></pre></div>
 <pre><code>## [1] TRUE</code></pre>
-<div class="sourceCode" id="cb1178"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1178-1"><a href="transformations.html#cb1178-1" aria-hidden="true" tabindex="-1"></a><span class="fu">all.equal</span>(<span class="fu">resid</span>(fit_4a),</span>
-<span id="cb1178-2"><a href="transformations.html#cb1178-2" aria-hidden="true" tabindex="-1"></a>          <span class="fu">resid</span>(fit_4b))</span></code></pre></div>
+<div class="sourceCode" id="cb1178"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1178-1"><a href="transformations.html#cb1178-1" tabindex="-1"></a><span class="fu">all.equal</span>(<span class="fu">resid</span>(fit_4a),</span>
+<span id="cb1178-2"><a href="transformations.html#cb1178-2" tabindex="-1"></a>          <span class="fu">resid</span>(fit_4b))</span></code></pre></div>
 <pre><code>## [1] TRUE</code></pre>
-<div class="sourceCode" id="cb1180"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1180-1"><a href="transformations.html#cb1180-1" aria-hidden="true" tabindex="-1"></a><span class="fu">summary</span>(fit_4a)</span></code></pre></div>
+<div class="sourceCode" id="cb1180"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1180-1"><a href="transformations.html#cb1180-1" tabindex="-1"></a><span class="fu">summary</span>(fit_4a)</span></code></pre></div>
 <pre><code>## 
 ## Call:
 ## lm(formula = y ~ poly(x, degree = 4), data = data_higher)
@@ -1551,7 +1551,7 @@ <h3><span class="header-section-number">14.2.5</span> <code>poly()</code> Functi
 ## Residual standard error: 2.794 on 245 degrees of freedom
 ## Multiple R-squared:  0.5993, Adjusted R-squared:  0.5928 
 ## F-statistic: 91.61 on 4 and 245 DF,  p-value: &lt; 2.2e-16</code></pre>
-<div class="sourceCode" id="cb1182"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1182-1"><a href="transformations.html#cb1182-1" aria-hidden="true" tabindex="-1"></a><span class="fu">summary</span>(fit_4c)</span></code></pre></div>
+<div class="sourceCode" id="cb1182"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1182-1"><a href="transformations.html#cb1182-1" tabindex="-1"></a><span class="fu">summary</span>(fit_4c)</span></code></pre></div>
 <pre><code>## 
 ## Call:
 ## lm(formula = y ~ x + I(x^2) + I(x^3) + I(x^4), data = data_higher)
@@ -1576,89 +1576,89 @@ <h3><span class="header-section-number">14.2.5</span> <code>poly()</code> Functi
 </div>
 <div id="inhibit-function" class="section level3 hasAnchor" number="14.2.6">
 <h3><span class="header-section-number">14.2.6</span> Inhibit Function<a href="transformations.html#inhibit-function" class="anchor-section" aria-label="Anchor link to header"></a></h3>
-<div class="sourceCode" id="cb1184"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1184-1"><a href="transformations.html#cb1184-1" aria-hidden="true" tabindex="-1"></a><span class="fu">coef</span>(<span class="fu">lm</span>(y <span class="sc">~</span> x <span class="sc">+</span> x <span class="sc">^</span> <span class="dv">2</span>, <span class="at">data =</span> quad_data))</span></code></pre></div>
+<div class="sourceCode" id="cb1184"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1184-1"><a href="transformations.html#cb1184-1" tabindex="-1"></a><span class="fu">coef</span>(<span class="fu">lm</span>(y <span class="sc">~</span> x <span class="sc">+</span> x <span class="sc">^</span> <span class="dv">2</span>, <span class="at">data =</span> quad_data))</span></code></pre></div>
 <pre><code>## (Intercept)           x 
 ##   -18.32715    24.87163</code></pre>
-<div class="sourceCode" id="cb1186"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1186-1"><a href="transformations.html#cb1186-1" aria-hidden="true" tabindex="-1"></a><span class="fu">coef</span>(<span class="fu">lm</span>(y <span class="sc">~</span> x <span class="sc">+</span> <span class="fu">I</span>(x <span class="sc">^</span> <span class="dv">2</span>), <span class="at">data =</span> quad_data))</span></code></pre></div>
+<div class="sourceCode" id="cb1186"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1186-1"><a href="transformations.html#cb1186-1" tabindex="-1"></a><span class="fu">coef</span>(<span class="fu">lm</span>(y <span class="sc">~</span> x <span class="sc">+</span> <span class="fu">I</span>(x <span class="sc">^</span> <span class="dv">2</span>), <span class="at">data =</span> quad_data))</span></code></pre></div>
 <pre><code>## (Intercept)           x      I(x^2) 
 ##   3.0649446  -0.5108131   5.0739805</code></pre>
-<div class="sourceCode" id="cb1188"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1188-1"><a href="transformations.html#cb1188-1" aria-hidden="true" tabindex="-1"></a><span class="fu">coef</span>(<span class="fu">lm</span>(y <span class="sc">~</span> x <span class="sc">+</span> x<span class="sc">:</span>x, <span class="at">data =</span> quad_data))</span></code></pre></div>
+<div class="sourceCode" id="cb1188"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1188-1"><a href="transformations.html#cb1188-1" tabindex="-1"></a><span class="fu">coef</span>(<span class="fu">lm</span>(y <span class="sc">~</span> x <span class="sc">+</span> x<span class="sc">:</span>x, <span class="at">data =</span> quad_data))</span></code></pre></div>
 <pre><code>## (Intercept)           x 
 ##   -18.32715    24.87163</code></pre>
-<div class="sourceCode" id="cb1190"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1190-1"><a href="transformations.html#cb1190-1" aria-hidden="true" tabindex="-1"></a><span class="fu">coef</span>(<span class="fu">lm</span>(y <span class="sc">~</span> x <span class="sc">*</span> x, <span class="at">data =</span> quad_data))</span></code></pre></div>
+<div class="sourceCode" id="cb1190"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1190-1"><a href="transformations.html#cb1190-1" tabindex="-1"></a><span class="fu">coef</span>(<span class="fu">lm</span>(y <span class="sc">~</span> x <span class="sc">*</span> x, <span class="at">data =</span> quad_data))</span></code></pre></div>
 <pre><code>## (Intercept)           x 
 ##   -18.32715    24.87163</code></pre>
-<div class="sourceCode" id="cb1192"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1192-1"><a href="transformations.html#cb1192-1" aria-hidden="true" tabindex="-1"></a><span class="fu">coef</span>(<span class="fu">lm</span>(y <span class="sc">~</span> x <span class="sc">^</span> <span class="dv">2</span>, <span class="at">data =</span> quad_data))</span></code></pre></div>
+<div class="sourceCode" id="cb1192"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1192-1"><a href="transformations.html#cb1192-1" tabindex="-1"></a><span class="fu">coef</span>(<span class="fu">lm</span>(y <span class="sc">~</span> x <span class="sc">^</span> <span class="dv">2</span>, <span class="at">data =</span> quad_data))</span></code></pre></div>
 <pre><code>## (Intercept)           x 
 ##   -18.32715    24.87163</code></pre>
-<div class="sourceCode" id="cb1194"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1194-1"><a href="transformations.html#cb1194-1" aria-hidden="true" tabindex="-1"></a><span class="fu">coef</span>(<span class="fu">lm</span>(y <span class="sc">~</span> x <span class="sc">+</span> x <span class="sc">^</span> <span class="dv">2</span>, <span class="at">data =</span> quad_data))</span></code></pre></div>
+<div class="sourceCode" id="cb1194"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1194-1"><a href="transformations.html#cb1194-1" tabindex="-1"></a><span class="fu">coef</span>(<span class="fu">lm</span>(y <span class="sc">~</span> x <span class="sc">+</span> x <span class="sc">^</span> <span class="dv">2</span>, <span class="at">data =</span> quad_data))</span></code></pre></div>
 <pre><code>## (Intercept)           x 
 ##   -18.32715    24.87163</code></pre>
-<div class="sourceCode" id="cb1196"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1196-1"><a href="transformations.html#cb1196-1" aria-hidden="true" tabindex="-1"></a><span class="fu">coef</span>(<span class="fu">lm</span>(y <span class="sc">~</span> <span class="fu">I</span>(x <span class="sc">+</span> x), <span class="at">data =</span> quad_data))</span></code></pre></div>
+<div class="sourceCode" id="cb1196"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1196-1"><a href="transformations.html#cb1196-1" tabindex="-1"></a><span class="fu">coef</span>(<span class="fu">lm</span>(y <span class="sc">~</span> <span class="fu">I</span>(x <span class="sc">+</span> x), <span class="at">data =</span> quad_data))</span></code></pre></div>
 <pre><code>## (Intercept)    I(x + x) 
 ##   -18.32715    12.43582</code></pre>
-<div class="sourceCode" id="cb1198"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1198-1"><a href="transformations.html#cb1198-1" aria-hidden="true" tabindex="-1"></a><span class="fu">coef</span>(<span class="fu">lm</span>(y <span class="sc">~</span> x <span class="sc">+</span> x, <span class="at">data =</span> quad_data))</span></code></pre></div>
+<div class="sourceCode" id="cb1198"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1198-1"><a href="transformations.html#cb1198-1" tabindex="-1"></a><span class="fu">coef</span>(<span class="fu">lm</span>(y <span class="sc">~</span> x <span class="sc">+</span> x, <span class="at">data =</span> quad_data))</span></code></pre></div>
 <pre><code>## (Intercept)           x 
 ##   -18.32715    24.87163</code></pre>
 </div>
 <div id="data-example" class="section level3 hasAnchor" number="14.2.7">
 <h3><span class="header-section-number">14.2.7</span> Data Example<a href="transformations.html#data-example" class="anchor-section" aria-label="Anchor link to header"></a></h3>
-<div class="sourceCode" id="cb1200"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1200-1"><a href="transformations.html#cb1200-1" aria-hidden="true" tabindex="-1"></a><span class="fu">pairs</span>(autompg)</span></code></pre></div>
+<div class="sourceCode" id="cb1200"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1200-1"><a href="transformations.html#cb1200-1" tabindex="-1"></a><span class="fu">pairs</span>(autompg)</span></code></pre></div>
 <p><img src="transformations_files/figure-html/unnamed-chunk-105-1.png" width="960" style="display: block; margin: auto;" /></p>
-<div class="sourceCode" id="cb1201"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1201-1"><a href="transformations.html#cb1201-1" aria-hidden="true" tabindex="-1"></a>mpg_hp <span class="ot">=</span> <span class="fu">lm</span>(mpg <span class="sc">~</span> hp, <span class="at">data =</span> autompg)</span>
-<span id="cb1201-2"><a href="transformations.html#cb1201-2" aria-hidden="true" tabindex="-1"></a></span>
-<span id="cb1201-3"><a href="transformations.html#cb1201-3" aria-hidden="true" tabindex="-1"></a><span class="fu">par</span>(<span class="at">mfrow =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>))</span>
-<span id="cb1201-4"><a href="transformations.html#cb1201-4" aria-hidden="true" tabindex="-1"></a></span>
-<span id="cb1201-5"><a href="transformations.html#cb1201-5" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(mpg <span class="sc">~</span> hp, <span class="at">data =</span> autompg, <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">cex =</span> <span class="fl">1.5</span>)</span>
-<span id="cb1201-6"><a href="transformations.html#cb1201-6" aria-hidden="true" tabindex="-1"></a><span class="fu">abline</span>(mpg_hp, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span>
-<span id="cb1201-7"><a href="transformations.html#cb1201-7" aria-hidden="true" tabindex="-1"></a></span>
-<span id="cb1201-8"><a href="transformations.html#cb1201-8" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(<span class="fu">fitted</span>(mpg_hp), <span class="fu">resid</span>(mpg_hp), <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>,</span>
-<span id="cb1201-9"><a href="transformations.html#cb1201-9" aria-hidden="true" tabindex="-1"></a>     <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">cex =</span> <span class="fl">1.5</span>, <span class="at">xlab =</span> <span class="st">&quot;Fitted&quot;</span>, <span class="at">ylab =</span> <span class="st">&quot;Residuals&quot;</span>)</span>
-<span id="cb1201-10"><a href="transformations.html#cb1201-10" aria-hidden="true" tabindex="-1"></a><span class="fu">abline</span>(<span class="at">h =</span> <span class="dv">0</span>, <span class="at">lty =</span> <span class="dv">2</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1201"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1201-1"><a href="transformations.html#cb1201-1" tabindex="-1"></a>mpg_hp <span class="ot">=</span> <span class="fu">lm</span>(mpg <span class="sc">~</span> hp, <span class="at">data =</span> autompg)</span>
+<span id="cb1201-2"><a href="transformations.html#cb1201-2" tabindex="-1"></a></span>
+<span id="cb1201-3"><a href="transformations.html#cb1201-3" tabindex="-1"></a><span class="fu">par</span>(<span class="at">mfrow =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>))</span>
+<span id="cb1201-4"><a href="transformations.html#cb1201-4" tabindex="-1"></a></span>
+<span id="cb1201-5"><a href="transformations.html#cb1201-5" tabindex="-1"></a><span class="fu">plot</span>(mpg <span class="sc">~</span> hp, <span class="at">data =</span> autompg, <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">cex =</span> <span class="fl">1.5</span>)</span>
+<span id="cb1201-6"><a href="transformations.html#cb1201-6" tabindex="-1"></a><span class="fu">abline</span>(mpg_hp, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span>
+<span id="cb1201-7"><a href="transformations.html#cb1201-7" tabindex="-1"></a></span>
+<span id="cb1201-8"><a href="transformations.html#cb1201-8" tabindex="-1"></a><span class="fu">plot</span>(<span class="fu">fitted</span>(mpg_hp), <span class="fu">resid</span>(mpg_hp), <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>,</span>
+<span id="cb1201-9"><a href="transformations.html#cb1201-9" tabindex="-1"></a>     <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">cex =</span> <span class="fl">1.5</span>, <span class="at">xlab =</span> <span class="st">&quot;Fitted&quot;</span>, <span class="at">ylab =</span> <span class="st">&quot;Residuals&quot;</span>)</span>
+<span id="cb1201-10"><a href="transformations.html#cb1201-10" tabindex="-1"></a><span class="fu">abline</span>(<span class="at">h =</span> <span class="dv">0</span>, <span class="at">lty =</span> <span class="dv">2</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
 <p><img src="transformations_files/figure-html/unnamed-chunk-106-1.png" width="960" style="display: block; margin: auto;" /></p>
-<div class="sourceCode" id="cb1202"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1202-1"><a href="transformations.html#cb1202-1" aria-hidden="true" tabindex="-1"></a>mpg_hp_log <span class="ot">=</span> <span class="fu">lm</span>(mpg <span class="sc">~</span> hp <span class="sc">+</span> <span class="fu">I</span>(hp <span class="sc">^</span> <span class="dv">2</span>), <span class="at">data =</span> autompg)</span>
-<span id="cb1202-2"><a href="transformations.html#cb1202-2" aria-hidden="true" tabindex="-1"></a></span>
-<span id="cb1202-3"><a href="transformations.html#cb1202-3" aria-hidden="true" tabindex="-1"></a><span class="fu">par</span>(<span class="at">mfrow =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>))</span>
-<span id="cb1202-4"><a href="transformations.html#cb1202-4" aria-hidden="true" tabindex="-1"></a></span>
-<span id="cb1202-5"><a href="transformations.html#cb1202-5" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(mpg <span class="sc">~</span> hp, <span class="at">data =</span> autompg, <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">cex =</span> <span class="fl">1.5</span>)</span>
-<span id="cb1202-6"><a href="transformations.html#cb1202-6" aria-hidden="true" tabindex="-1"></a>xplot <span class="ot">=</span> <span class="fu">seq</span>(<span class="fu">min</span>(autompg<span class="sc">$</span>hp), <span class="fu">max</span>(autompg<span class="sc">$</span>hp), <span class="at">by =</span> <span class="fl">0.1</span>)</span>
-<span id="cb1202-7"><a href="transformations.html#cb1202-7" aria-hidden="true" tabindex="-1"></a><span class="fu">lines</span>(xplot, <span class="fu">predict</span>(mpg_hp_log, <span class="at">newdata =</span> <span class="fu">data.frame</span>(<span class="at">hp =</span> xplot)),</span>
-<span id="cb1202-8"><a href="transformations.html#cb1202-8" aria-hidden="true" tabindex="-1"></a>      <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>, <span class="at">lty =</span> <span class="dv">1</span>)</span>
-<span id="cb1202-9"><a href="transformations.html#cb1202-9" aria-hidden="true" tabindex="-1"></a></span>
-<span id="cb1202-10"><a href="transformations.html#cb1202-10" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(<span class="fu">fitted</span>(mpg_hp_log), <span class="fu">resid</span>(mpg_hp_log), <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>,</span>
-<span id="cb1202-11"><a href="transformations.html#cb1202-11" aria-hidden="true" tabindex="-1"></a>     <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">cex =</span> <span class="fl">1.5</span>, <span class="at">xlab =</span> <span class="st">&quot;Fitted&quot;</span>, <span class="at">ylab =</span> <span class="st">&quot;Residuals&quot;</span>)</span>
-<span id="cb1202-12"><a href="transformations.html#cb1202-12" aria-hidden="true" tabindex="-1"></a><span class="fu">abline</span>(<span class="at">h =</span> <span class="dv">0</span>, <span class="at">lty =</span> <span class="dv">2</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1202"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1202-1"><a href="transformations.html#cb1202-1" tabindex="-1"></a>mpg_hp_log <span class="ot">=</span> <span class="fu">lm</span>(mpg <span class="sc">~</span> hp <span class="sc">+</span> <span class="fu">I</span>(hp <span class="sc">^</span> <span class="dv">2</span>), <span class="at">data =</span> autompg)</span>
+<span id="cb1202-2"><a href="transformations.html#cb1202-2" tabindex="-1"></a></span>
+<span id="cb1202-3"><a href="transformations.html#cb1202-3" tabindex="-1"></a><span class="fu">par</span>(<span class="at">mfrow =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>))</span>
+<span id="cb1202-4"><a href="transformations.html#cb1202-4" tabindex="-1"></a></span>
+<span id="cb1202-5"><a href="transformations.html#cb1202-5" tabindex="-1"></a><span class="fu">plot</span>(mpg <span class="sc">~</span> hp, <span class="at">data =</span> autompg, <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">cex =</span> <span class="fl">1.5</span>)</span>
+<span id="cb1202-6"><a href="transformations.html#cb1202-6" tabindex="-1"></a>xplot <span class="ot">=</span> <span class="fu">seq</span>(<span class="fu">min</span>(autompg<span class="sc">$</span>hp), <span class="fu">max</span>(autompg<span class="sc">$</span>hp), <span class="at">by =</span> <span class="fl">0.1</span>)</span>
+<span id="cb1202-7"><a href="transformations.html#cb1202-7" tabindex="-1"></a><span class="fu">lines</span>(xplot, <span class="fu">predict</span>(mpg_hp_log, <span class="at">newdata =</span> <span class="fu">data.frame</span>(<span class="at">hp =</span> xplot)),</span>
+<span id="cb1202-8"><a href="transformations.html#cb1202-8" tabindex="-1"></a>      <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>, <span class="at">lty =</span> <span class="dv">1</span>)</span>
+<span id="cb1202-9"><a href="transformations.html#cb1202-9" tabindex="-1"></a></span>
+<span id="cb1202-10"><a href="transformations.html#cb1202-10" tabindex="-1"></a><span class="fu">plot</span>(<span class="fu">fitted</span>(mpg_hp_log), <span class="fu">resid</span>(mpg_hp_log), <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>,</span>
+<span id="cb1202-11"><a href="transformations.html#cb1202-11" tabindex="-1"></a>     <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">cex =</span> <span class="fl">1.5</span>, <span class="at">xlab =</span> <span class="st">&quot;Fitted&quot;</span>, <span class="at">ylab =</span> <span class="st">&quot;Residuals&quot;</span>)</span>
+<span id="cb1202-12"><a href="transformations.html#cb1202-12" tabindex="-1"></a><span class="fu">abline</span>(<span class="at">h =</span> <span class="dv">0</span>, <span class="at">lty =</span> <span class="dv">2</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
 <p><img src="transformations_files/figure-html/unnamed-chunk-107-1.png" width="960" style="display: block; margin: auto;" /></p>
-<div class="sourceCode" id="cb1203"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1203-1"><a href="transformations.html#cb1203-1" aria-hidden="true" tabindex="-1"></a>mpg_hp_log <span class="ot">=</span> <span class="fu">lm</span>(<span class="fu">log</span>(mpg) <span class="sc">~</span> hp <span class="sc">+</span> <span class="fu">I</span>(hp <span class="sc">^</span> <span class="dv">2</span>), <span class="at">data =</span> autompg)</span>
-<span id="cb1203-2"><a href="transformations.html#cb1203-2" aria-hidden="true" tabindex="-1"></a></span>
-<span id="cb1203-3"><a href="transformations.html#cb1203-3" aria-hidden="true" tabindex="-1"></a><span class="fu">par</span>(<span class="at">mfrow =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>))</span>
-<span id="cb1203-4"><a href="transformations.html#cb1203-4" aria-hidden="true" tabindex="-1"></a></span>
-<span id="cb1203-5"><a href="transformations.html#cb1203-5" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(<span class="fu">log</span>(mpg) <span class="sc">~</span> hp, <span class="at">data =</span> autompg, <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">cex =</span> <span class="fl">1.5</span>)</span>
-<span id="cb1203-6"><a href="transformations.html#cb1203-6" aria-hidden="true" tabindex="-1"></a>xplot <span class="ot">=</span> <span class="fu">seq</span>(<span class="fu">min</span>(autompg<span class="sc">$</span>hp), <span class="fu">max</span>(autompg<span class="sc">$</span>hp), <span class="at">by =</span> <span class="fl">0.1</span>)</span>
-<span id="cb1203-7"><a href="transformations.html#cb1203-7" aria-hidden="true" tabindex="-1"></a><span class="fu">lines</span>(xplot, <span class="fu">predict</span>(mpg_hp_log, <span class="at">newdata =</span> <span class="fu">data.frame</span>(<span class="at">hp =</span> xplot)),</span>
-<span id="cb1203-8"><a href="transformations.html#cb1203-8" aria-hidden="true" tabindex="-1"></a>      <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>, <span class="at">lty =</span> <span class="dv">1</span>)</span>
-<span id="cb1203-9"><a href="transformations.html#cb1203-9" aria-hidden="true" tabindex="-1"></a></span>
-<span id="cb1203-10"><a href="transformations.html#cb1203-10" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(<span class="fu">fitted</span>(mpg_hp_log), <span class="fu">resid</span>(mpg_hp_log), <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>,</span>
-<span id="cb1203-11"><a href="transformations.html#cb1203-11" aria-hidden="true" tabindex="-1"></a>     <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">cex =</span> <span class="fl">1.5</span>, <span class="at">xlab =</span> <span class="st">&quot;Fitted&quot;</span>, <span class="at">ylab =</span> <span class="st">&quot;Residuals&quot;</span>)</span>
-<span id="cb1203-12"><a href="transformations.html#cb1203-12" aria-hidden="true" tabindex="-1"></a><span class="fu">abline</span>(<span class="at">h =</span> <span class="dv">0</span>, <span class="at">lty =</span> <span class="dv">2</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1203"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1203-1"><a href="transformations.html#cb1203-1" tabindex="-1"></a>mpg_hp_log <span class="ot">=</span> <span class="fu">lm</span>(<span class="fu">log</span>(mpg) <span class="sc">~</span> hp <span class="sc">+</span> <span class="fu">I</span>(hp <span class="sc">^</span> <span class="dv">2</span>), <span class="at">data =</span> autompg)</span>
+<span id="cb1203-2"><a href="transformations.html#cb1203-2" tabindex="-1"></a></span>
+<span id="cb1203-3"><a href="transformations.html#cb1203-3" tabindex="-1"></a><span class="fu">par</span>(<span class="at">mfrow =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>))</span>
+<span id="cb1203-4"><a href="transformations.html#cb1203-4" tabindex="-1"></a></span>
+<span id="cb1203-5"><a href="transformations.html#cb1203-5" tabindex="-1"></a><span class="fu">plot</span>(<span class="fu">log</span>(mpg) <span class="sc">~</span> hp, <span class="at">data =</span> autompg, <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">cex =</span> <span class="fl">1.5</span>)</span>
+<span id="cb1203-6"><a href="transformations.html#cb1203-6" tabindex="-1"></a>xplot <span class="ot">=</span> <span class="fu">seq</span>(<span class="fu">min</span>(autompg<span class="sc">$</span>hp), <span class="fu">max</span>(autompg<span class="sc">$</span>hp), <span class="at">by =</span> <span class="fl">0.1</span>)</span>
+<span id="cb1203-7"><a href="transformations.html#cb1203-7" tabindex="-1"></a><span class="fu">lines</span>(xplot, <span class="fu">predict</span>(mpg_hp_log, <span class="at">newdata =</span> <span class="fu">data.frame</span>(<span class="at">hp =</span> xplot)),</span>
+<span id="cb1203-8"><a href="transformations.html#cb1203-8" tabindex="-1"></a>      <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>, <span class="at">lty =</span> <span class="dv">1</span>)</span>
+<span id="cb1203-9"><a href="transformations.html#cb1203-9" tabindex="-1"></a></span>
+<span id="cb1203-10"><a href="transformations.html#cb1203-10" tabindex="-1"></a><span class="fu">plot</span>(<span class="fu">fitted</span>(mpg_hp_log), <span class="fu">resid</span>(mpg_hp_log), <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>,</span>
+<span id="cb1203-11"><a href="transformations.html#cb1203-11" tabindex="-1"></a>     <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">cex =</span> <span class="fl">1.5</span>, <span class="at">xlab =</span> <span class="st">&quot;Fitted&quot;</span>, <span class="at">ylab =</span> <span class="st">&quot;Residuals&quot;</span>)</span>
+<span id="cb1203-12"><a href="transformations.html#cb1203-12" tabindex="-1"></a><span class="fu">abline</span>(<span class="at">h =</span> <span class="dv">0</span>, <span class="at">lty =</span> <span class="dv">2</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
 <p><img src="transformations_files/figure-html/unnamed-chunk-108-1.png" width="960" style="display: block; margin: auto;" /></p>
-<div class="sourceCode" id="cb1204"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1204-1"><a href="transformations.html#cb1204-1" aria-hidden="true" tabindex="-1"></a>mpg_hp_loglog <span class="ot">=</span> <span class="fu">lm</span>(<span class="fu">log</span>(mpg) <span class="sc">~</span> <span class="fu">log</span>(hp), <span class="at">data =</span> autompg)</span>
-<span id="cb1204-2"><a href="transformations.html#cb1204-2" aria-hidden="true" tabindex="-1"></a></span>
-<span id="cb1204-3"><a href="transformations.html#cb1204-3" aria-hidden="true" tabindex="-1"></a><span class="fu">par</span>(<span class="at">mfrow =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>))</span>
-<span id="cb1204-4"><a href="transformations.html#cb1204-4" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(<span class="fu">log</span>(mpg) <span class="sc">~</span> <span class="fu">log</span>(hp), <span class="at">data =</span> autompg, <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">cex =</span> <span class="fl">1.5</span>)</span>
-<span id="cb1204-5"><a href="transformations.html#cb1204-5" aria-hidden="true" tabindex="-1"></a><span class="fu">abline</span>(mpg_hp_loglog, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span>
-<span id="cb1204-6"><a href="transformations.html#cb1204-6" aria-hidden="true" tabindex="-1"></a></span>
-<span id="cb1204-7"><a href="transformations.html#cb1204-7" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(<span class="fu">fitted</span>(mpg_hp_loglog), <span class="fu">resid</span>(mpg_hp_loglog), <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>,</span>
-<span id="cb1204-8"><a href="transformations.html#cb1204-8" aria-hidden="true" tabindex="-1"></a>     <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">cex =</span> <span class="fl">1.5</span>, <span class="at">xlab =</span> <span class="st">&quot;Fitted&quot;</span>, <span class="at">ylab =</span> <span class="st">&quot;Residuals&quot;</span>)</span>
-<span id="cb1204-9"><a href="transformations.html#cb1204-9" aria-hidden="true" tabindex="-1"></a><span class="fu">abline</span>(<span class="at">h =</span> <span class="dv">0</span>, <span class="at">lty =</span> <span class="dv">2</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1204"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1204-1"><a href="transformations.html#cb1204-1" tabindex="-1"></a>mpg_hp_loglog <span class="ot">=</span> <span class="fu">lm</span>(<span class="fu">log</span>(mpg) <span class="sc">~</span> <span class="fu">log</span>(hp), <span class="at">data =</span> autompg)</span>
+<span id="cb1204-2"><a href="transformations.html#cb1204-2" tabindex="-1"></a></span>
+<span id="cb1204-3"><a href="transformations.html#cb1204-3" tabindex="-1"></a><span class="fu">par</span>(<span class="at">mfrow =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="dv">2</span>))</span>
+<span id="cb1204-4"><a href="transformations.html#cb1204-4" tabindex="-1"></a><span class="fu">plot</span>(<span class="fu">log</span>(mpg) <span class="sc">~</span> <span class="fu">log</span>(hp), <span class="at">data =</span> autompg, <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">cex =</span> <span class="fl">1.5</span>)</span>
+<span id="cb1204-5"><a href="transformations.html#cb1204-5" tabindex="-1"></a><span class="fu">abline</span>(mpg_hp_loglog, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span>
+<span id="cb1204-6"><a href="transformations.html#cb1204-6" tabindex="-1"></a></span>
+<span id="cb1204-7"><a href="transformations.html#cb1204-7" tabindex="-1"></a><span class="fu">plot</span>(<span class="fu">fitted</span>(mpg_hp_loglog), <span class="fu">resid</span>(mpg_hp_loglog), <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>,</span>
+<span id="cb1204-8"><a href="transformations.html#cb1204-8" tabindex="-1"></a>     <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">cex =</span> <span class="fl">1.5</span>, <span class="at">xlab =</span> <span class="st">&quot;Fitted&quot;</span>, <span class="at">ylab =</span> <span class="st">&quot;Residuals&quot;</span>)</span>
+<span id="cb1204-9"><a href="transformations.html#cb1204-9" tabindex="-1"></a><span class="fu">abline</span>(<span class="at">h =</span> <span class="dv">0</span>, <span class="at">lty =</span> <span class="dv">2</span>, <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
 <p><img src="transformations_files/figure-html/unnamed-chunk-109-1.png" width="960" style="display: block; margin: auto;" /></p>
-<div class="sourceCode" id="cb1205"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1205-1"><a href="transformations.html#cb1205-1" aria-hidden="true" tabindex="-1"></a>big_model <span class="ot">=</span> <span class="fu">lm</span>(mpg <span class="sc">~</span> disp <span class="sc">*</span> hp <span class="sc">*</span> domestic, <span class="at">data =</span> autompg)</span></code></pre></div>
-<div class="sourceCode" id="cb1206"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1206-1"><a href="transformations.html#cb1206-1" aria-hidden="true" tabindex="-1"></a><span class="fu">qqnorm</span>(<span class="fu">resid</span>(big_model), <span class="at">col =</span> <span class="st">&quot;darkgrey&quot;</span>)</span>
-<span id="cb1206-2"><a href="transformations.html#cb1206-2" aria-hidden="true" tabindex="-1"></a><span class="fu">qqline</span>(<span class="fu">resid</span>(big_model), <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1205"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1205-1"><a href="transformations.html#cb1205-1" tabindex="-1"></a>big_model <span class="ot">=</span> <span class="fu">lm</span>(mpg <span class="sc">~</span> disp <span class="sc">*</span> hp <span class="sc">*</span> domestic, <span class="at">data =</span> autompg)</span></code></pre></div>
+<div class="sourceCode" id="cb1206"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1206-1"><a href="transformations.html#cb1206-1" tabindex="-1"></a><span class="fu">qqnorm</span>(<span class="fu">resid</span>(big_model), <span class="at">col =</span> <span class="st">&quot;darkgrey&quot;</span>)</span>
+<span id="cb1206-2"><a href="transformations.html#cb1206-2" tabindex="-1"></a><span class="fu">qqline</span>(<span class="fu">resid</span>(big_model), <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
 <p><img src="transformations_files/figure-html/unnamed-chunk-111-1.png" width="672" style="display: block; margin: auto;" /></p>
-<div class="sourceCode" id="cb1207"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1207-1"><a href="transformations.html#cb1207-1" aria-hidden="true" tabindex="-1"></a>bigger_model <span class="ot">=</span> <span class="fu">lm</span>(<span class="fu">log</span>(mpg) <span class="sc">~</span> disp <span class="sc">*</span> hp <span class="sc">*</span> domestic <span class="sc">+</span> </span>
-<span id="cb1207-2"><a href="transformations.html#cb1207-2" aria-hidden="true" tabindex="-1"></a>               <span class="fu">I</span>(disp <span class="sc">^</span> <span class="dv">2</span>) <span class="sc">+</span> <span class="fu">I</span>(hp <span class="sc">^</span> <span class="dv">2</span>), <span class="at">data =</span> autompg)</span>
-<span id="cb1207-3"><a href="transformations.html#cb1207-3" aria-hidden="true" tabindex="-1"></a><span class="fu">summary</span>(bigger_model)</span></code></pre></div>
+<div class="sourceCode" id="cb1207"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1207-1"><a href="transformations.html#cb1207-1" tabindex="-1"></a>bigger_model <span class="ot">=</span> <span class="fu">lm</span>(<span class="fu">log</span>(mpg) <span class="sc">~</span> disp <span class="sc">*</span> hp <span class="sc">*</span> domestic <span class="sc">+</span> </span>
+<span id="cb1207-2"><a href="transformations.html#cb1207-2" tabindex="-1"></a>               <span class="fu">I</span>(disp <span class="sc">^</span> <span class="dv">2</span>) <span class="sc">+</span> <span class="fu">I</span>(hp <span class="sc">^</span> <span class="dv">2</span>), <span class="at">data =</span> autompg)</span>
+<span id="cb1207-3"><a href="transformations.html#cb1207-3" tabindex="-1"></a><span class="fu">summary</span>(bigger_model)</span></code></pre></div>
 <pre><code>## 
 ## Call:
 ## lm(formula = log(mpg) ~ disp * hp * domestic + I(disp^2) + I(hp^2), 
@@ -1686,8 +1686,8 @@ <h3><span class="header-section-number">14.2.7</span> Data Example<a href="trans
 ## Residual standard error: 0.1507 on 373 degrees of freedom
 ## Multiple R-squared:  0.8107, Adjusted R-squared:  0.8062 
 ## F-statistic: 177.5 on 9 and 373 DF,  p-value: &lt; 2.2e-16</code></pre>
-<div class="sourceCode" id="cb1209"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1209-1"><a href="transformations.html#cb1209-1" aria-hidden="true" tabindex="-1"></a><span class="fu">qqnorm</span>(<span class="fu">resid</span>(bigger_model), <span class="at">col =</span> <span class="st">&quot;darkgrey&quot;</span>)</span>
-<span id="cb1209-2"><a href="transformations.html#cb1209-2" aria-hidden="true" tabindex="-1"></a><span class="fu">qqline</span>(<span class="fu">resid</span>(bigger_model), <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1209"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1209-1"><a href="transformations.html#cb1209-1" tabindex="-1"></a><span class="fu">qqnorm</span>(<span class="fu">resid</span>(bigger_model), <span class="at">col =</span> <span class="st">&quot;darkgrey&quot;</span>)</span>
+<span id="cb1209-2"><a href="transformations.html#cb1209-2" tabindex="-1"></a><span class="fu">qqline</span>(<span class="fu">resid</span>(bigger_model), <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>)</span></code></pre></div>
 <p><img src="transformations_files/figure-html/unnamed-chunk-113-1.png" width="672" style="display: block; margin: auto;" /></p>
 </div>
 </div>
@@ -1697,7 +1697,7 @@ <h2><span class="header-section-number">14.3</span> <code>R</code> Markdown<a hr
 <ul>
 <li><a href="transformations.Rmd" target="_blank"><code>transformations.Rmd</code></a></li>
 </ul>
-<p>The file was created using <code>R</code> version <code>4.3.2</code>.</p>
+<p>The file was created using <code>R</code> version <code>4.4.1</code>.</p>
 
 </div>
 </div>
diff --git a/transformations.md b/transformations.md
index 00086e9..ba6fc8d 100644
--- a/transformations.md
+++ b/transformations.md
@@ -23,12 +23,12 @@ Let's look at some (*fictional*) salary data from the (*fictional*) company *Ini
 
 
 
-```r
+``` r
 initech = read.csv("data/initech.csv")
 ```
 
 
-```r
+``` r
 plot(salary ~ years, data = initech, col = "grey", pch = 20, cex = 1.5,
      main = "Salaries at Initech, By Seniority")
 ```
@@ -40,7 +40,7 @@ plot(salary ~ years, data = initech, col = "grey", pch = 20, cex = 1.5,
 We first fit a simple linear model.
 
 
-```r
+``` r
 initech_fit = lm(salary ~ years, data = initech)
 summary(initech_fit)
 ```
@@ -69,7 +69,7 @@ summary(initech_fit)
 This model appears significant, but does it meet the model assumptions?
 
 
-```r
+``` r
 plot(salary ~ years, data = initech, col = "grey", pch = 20, cex = 1.5,
      main = "Salaries at Initech, By Seniority")
 abline(initech_fit, col = "darkorange", lwd = 2)
@@ -82,7 +82,7 @@ abline(initech_fit, col = "darkorange", lwd = 2)
 Adding the fitted line to the plot, we see that the linear relationship appears correct.
 
 
-```r
+``` r
 par(mfrow = c(1, 2))
 
 plot(fitted(initech_fit), resid(initech_fit), col = "grey", pch = 20,
@@ -152,14 +152,14 @@ which has the errors entering the model in a multiplicative fashion.
 Fitting this model in `R` requires only a minor modification to our formula specification.
 
 
-```r
+``` r
 initech_fit_log = lm(log(salary) ~ years, data = initech)
 ```
 
 Note that while `log(y)` is considered the new response variable, we do not actually create a new variable in `R`, but simply transform the variable inside the model formula.
 
 
-```r
+``` r
 plot(log(salary) ~ years, data = initech, col = "grey", pch = 20, cex = 1.5,
      main = "Salaries at Initech, By Seniority")
 abline(initech_fit_log, col = "darkorange", lwd = 2)
@@ -172,7 +172,7 @@ abline(initech_fit_log, col = "darkorange", lwd = 2)
 Plotting the data on the transformed log scale and adding the fitted line, the relationship again appears linear, and we can already see that the variation about the fitted line looks constant.
 
 
-```r
+``` r
 plot(salary ~ years, data = initech, col = "grey", pch = 20, cex = 1.5,
      main = "Salaries at Initech, By Seniority")
 curve(exp(initech_fit_log$coef[1] + initech_fit_log$coef[2] * x),
@@ -186,7 +186,7 @@ curve(exp(initech_fit_log$coef[1] + initech_fit_log$coef[2] * x),
 By plotting the data on the original scale, and adding the fitted regression, we see an exponential relationship. However, this is still a *linear* model, since the new transformed response, $\log(y)$, is still a *linear* combination of the predictors.
 
 
-```r
+``` r
 par(mfrow = c(1, 2))
 
 plot(fitted(initech_fit_log), resid(initech_fit_log), col = "grey", pch = 20,
@@ -206,7 +206,7 @@ The fitted versus residuals plot looks much better. It appears the constant vari
 Comparing the RMSE using the original and transformed response, we also see that the log transformed model simply fits better, with a smaller average squared error.
 
 
-```r
+``` r
 sqrt(mean(resid(initech_fit) ^ 2))
 ```
 
@@ -214,7 +214,7 @@ sqrt(mean(resid(initech_fit) ^ 2))
 ## [1] 27080.16
 ```
 
-```r
+``` r
 sqrt(mean(resid(initech_fit_log) ^ 2))
 ```
 
@@ -225,7 +225,7 @@ sqrt(mean(resid(initech_fit_log) ^ 2))
 But wait, that isn't fair, this difference is simply due to the different scales being used.
 
 
-```r
+``` r
 sqrt(mean((initech$salary - fitted(initech_fit)) ^ 2))
 ```
 
@@ -233,7 +233,7 @@ sqrt(mean((initech$salary - fitted(initech_fit)) ^ 2))
 ## [1] 27080.16
 ```
 
-```r
+``` r
 sqrt(mean((initech$salary - exp(fitted(initech_fit_log))) ^ 2))
 ```
 
@@ -244,7 +244,7 @@ sqrt(mean((initech$salary - exp(fitted(initech_fit_log))) ^ 2))
 Transforming the fitted values of the log model back to the data scale, we do indeed see that it fits better!
 
 
-```r
+``` r
 summary(initech_fit_log)
 ```
 
@@ -313,7 +313,7 @@ A $100(1 - \alpha)\%$ confidence interval for $\lambda$ is,
 which `R` will plot for us to help quickly select an appropriate $\lambda$ value. We often choose a "nice" value from within the confidence interval, instead of the value of $\lambda$ that truly maximizes the likelihood.
 
 
-```r
+``` r
 library(MASS)
 library(faraway)
 ```
@@ -323,14 +323,14 @@ Here we need the `MASS` package for the `boxcox()` function, and we will conside
 First we will use the `savings` dataset as an example of using the Box-Cox method to justify the use of no transformation. We fit an additive multiple regression model with `sr` as the response and each of the other variables as predictors.
 
 
-```r
+``` r
 savings_model = lm(sr ~ ., data = savings)
 ```
 
 We then use the `boxcox()` function to find the best transformation of the form considered by the Box-Cox method.
 
 
-```r
+``` r
 boxcox(savings_model, plotit = TRUE)
 ```
 
@@ -341,7 +341,7 @@ boxcox(savings_model, plotit = TRUE)
 `R` automatically plots the log-Likelihood as a function of possible $\lambda$ values. It indicates both the value that maximizes the log-likelihood, as well as a confidence interval for the $\lambda$ value that maximizes the log-likelihood.
 
 
-```r
+``` r
 boxcox(savings_model, plotit = TRUE, lambda = seq(0.5, 1.5, by = 0.1))
 ```
 
@@ -358,7 +358,7 @@ Note that we can specify a range of $\lambda$ values to consider and thus be plo
 This is essentially not a transformation. It would not change the variance or make the model fit better. By subtracting 1 from every value, we would only change the intercept of the model, and the resulting errors would be the same.
 
 
-```r
+``` r
 plot(fitted(savings_model), resid(savings_model), col = "dodgerblue",
      pch = 20, cex = 1.5, xlab = "Fitted", ylab = "Residuals")
 abline(h = 0, lty = 2, col = "darkorange", lwd = 2)
@@ -371,7 +371,7 @@ abline(h = 0, lty = 2, col = "darkorange", lwd = 2)
 Looking at a fitted versus residuals plot verifies that there likely are not any issues with the assumptions of this model, which Breusch-Pagan and Shapiro-Wilk tests verify.
 
 
-```r
+``` r
 library(lmtest)
 bptest(savings_model)
 ```
@@ -384,7 +384,7 @@ bptest(savings_model)
 ## BP = 4.9852, df = 4, p-value = 0.2888
 ```
 
-```r
+``` r
 shapiro.test(resid(savings_model))
 ```
 
@@ -399,12 +399,12 @@ shapiro.test(resid(savings_model))
 Now we will use the `gala` dataset as an example of using the Box-Cox method to justify a transformation other than $\log$. We fit an additive multiple regression model with `Species` as the response and most of the other variables as predictors.
 
 
-```r
+``` r
 gala_model = lm(Species ~ Area + Elevation + Nearest + Scruz + Adjacent, data = gala)
 ```
 
 
-```r
+``` r
 plot(fitted(gala_model), resid(gala_model), col = "dodgerblue",
      pch = 20, cex = 1.5, xlab = "Fitted", ylab = "Residuals")
 abline(h = 0, lty = 2, col = "darkorange", lwd = 2)
@@ -417,7 +417,7 @@ abline(h = 0, lty = 2, col = "darkorange", lwd = 2)
 Even though there is not a lot of data for large fitted values, it still seems very clear that the constant variance assumption is violated.
 
 
-```r
+``` r
 boxcox(gala_model, lambda = seq(-0.25, 0.75, by = 0.05), plotit = TRUE)
 ```
 
@@ -434,12 +434,12 @@ Using the Box-Cox method, we see that $\lambda = 0.3$ is both in the confidence
 We then fit a model with this transformation applied to the response.
 
 
-```r
+``` r
 gala_model_cox = lm((((Species ^ 0.3) - 1) / 0.3) ~ Area + Elevation + Nearest + Scruz + Adjacent, data = gala)
 ```
 
 
-```r
+``` r
 plot(fitted(gala_model_cox), resid(gala_model_cox), col = "dodgerblue",
      pch = 20, cex = 1.5, xlab = "Fitted", ylab = "Residuals")
 abline(h = 0, lty = 2, col = "darkorange", lwd = 2)
@@ -454,7 +454,7 @@ The resulting fitted versus residuals plot looks much better!
 Lastly, we return to the `initech` data, and the `initech_fit` model we had used earlier. Recall that this was the untransformed model, that we used a $\log$ transform to fix.
 
 
-```r
+``` r
 boxcox(initech_fit)
 ```
 
@@ -477,7 +477,7 @@ In addition to transformation of the response variable, we can also consider tra
 
 
 
-```r
+``` r
 str(autompg)
 ```
 
@@ -497,7 +497,7 @@ str(autompg)
 Recall the `autompg` dataset from the previous chapter. Here we will attempt to model `mpg` as a function of `hp`.
 
 
-```r
+``` r
 par(mfrow = c(1, 2))
 plot(mpg ~ hp, data = autompg, col = "dodgerblue", pch = 20, cex = 1.5)
 mpg_hp = lm(mpg ~ hp, data = autompg)
@@ -514,7 +514,7 @@ abline(h = 0, lty = 2, col = "darkorange", lwd = 2)
 We first attempt SLR, but we see a rather obvious pattern in the fitted versus residuals plot, which includes increasing variance, so we attempt a $\log$ transform of the response.
 
 
-```r
+``` r
 par(mfrow = c(1, 2))
 plot(log(mpg) ~ hp, data = autompg, col = "dodgerblue", pch = 20, cex = 1.5)
 mpg_hp_log = lm(log(mpg) ~ hp, data = autompg)
@@ -531,7 +531,7 @@ abline(h = 0, lty = 2, col = "darkorange", lwd = 2)
 After performing the $\log$ transform of the response, we still have some of the same issues with the fitted versus response. Now, we will try also $\log$ transforming the **predictor**.
 
 
-```r
+``` r
 par(mfrow = c(1, 2))
 plot(log(mpg) ~ log(hp), data = autompg, col = "dodgerblue", pch = 20, cex = 1.5)
 mpg_hp_loglog = lm(log(mpg) ~ log(hp), data = autompg)
@@ -556,12 +556,12 @@ It should come as no surprise that sales of a product are related to the adverti
 Consider monthly data for the sales of *Initech* widgets, $y$, as a function of *Initech*'s advertising expenditure for said widget, $x$, both in ten thousand dollars. The data can be found in [`marketing.csv`](data/marketing.csv).
 
 
-```r
+``` r
 marketing = read.csv("data/marketing.csv")
 ```
 
 
-```r
+``` r
 plot(sales ~ advert, data = marketing, 
      xlab = "Advert Spending (in $10,000)", ylab = "Sales (in $10,000)",
      pch = 20, cex = 2)
@@ -618,7 +618,7 @@ We also maintain the same distributional results
 \]
 
 
-```r
+``` r
 mark_mod = lm(sales ~ advert, data = marketing)
 summary(mark_mod)
 ```
@@ -647,7 +647,7 @@ summary(mark_mod)
 While the SLR model is significant, the fitted versus residuals plot would have a very clear pattern.
 
 
-```r
+``` r
 mark_mod_poly2 = lm(sales ~ advert + I(advert ^ 2), data = marketing)
 summary(mark_mod_poly2)
 ```
@@ -677,7 +677,7 @@ summary(mark_mod_poly2)
 To add the second order term we need to use the `I()` function in the model specification around our newly created predictor. We see that with the first order term in the model, the quadratic term is also significant.
 
 
-```r
+``` r
 n = length(marketing$advert)
 X = cbind(rep(1, n), marketing$advert, marketing$advert ^ 2)
 t(X) %*% X
@@ -690,7 +690,7 @@ t(X) %*% X
 ## [3,] 1107.95 12385.86 151369.12
 ```
 
-```r
+``` r
 solve(t(X) %*% X) %*% t(X) %*% marketing$sales
 ```
 
@@ -720,7 +720,7 @@ We could also add higher order terms, such as a third degree predictor. This is
 \]
 
 
-```r
+``` r
 mark_mod_poly3 = lm(sales ~ advert + I(advert ^ 2) + I(advert ^ 3), data = marketing)
 summary(mark_mod_poly3)
 ```
@@ -751,7 +751,7 @@ summary(mark_mod_poly3)
 Now we see that with the first and second order terms in the model, the third order term is also significant. But does this make sense practically? The following plot should gives hints as to why it doesn't. (The model with the third order term doesn't have diminishing returns!)
 
 
-```r
+``` r
 plot(sales ~ advert, data = marketing, 
      xlab = "Advert Spending (in $10,000)", ylab = "Sales (in $10,000)",
      pch = 20, cex = 2)
@@ -770,7 +770,7 @@ lines(xplot, predict(mark_mod_poly3, newdata = data.frame(advert = xplot)),
 The previous plot was made using base graphics in `R`. The next plot was made using the package [`ggplot2`](https://ggplot2.tidyverse.org/){target="_blank"}, an increasingly popular plotting method in `R`.
 
 
-```r
+``` r
 library(ggplot2)
 ggplot(data = marketing, aes(x = advert, y = sales)) +
   stat_smooth(method = "lm", se = FALSE, color = "green", formula = y ~ x) +
@@ -792,7 +792,7 @@ Note we could fit a polynomial of an arbitrary order,
 However, we should be careful about over-fitting, since with a polynomial of degree one less than the number of observations, it is sometimes possible to fit a model perfectly.
 
 
-```r
+``` r
 set.seed(1234)
 x = seq(0, 10)
 y = 3 + x + 4 * x ^ 2 + rnorm(11, 0, 20)
@@ -832,7 +832,7 @@ summary(fit_perf)
 ## F-statistic:   NaN on 10 and 0 DF,  p-value: NA
 ```
 
-```r
+``` r
 xplot = seq(0, 10, by = 0.1)
 lines(xplot, predict(fit, newdata = data.frame(x = xplot)),
       col = "dodgerblue", lwd = 2, lty = 1)
@@ -853,7 +853,7 @@ Suppose you work for an automobile manufacturer which makes a large luxury sedan
 Our goal then, is to fit a model to this data in order to be able to predict fuel efficiency when driving at certain speeds. The data from this example can be found in [`fuel_econ.csv`](data/fuel_econ.csv).
 
 
-```r
+``` r
 econ = read.csv("data/fuel_econ.csv")
 ```
 
@@ -862,7 +862,7 @@ In this example, we will be frequently looking at the fitted versus residuals pl
 We will also be adding fitted curves to scatterplots repeatedly, so smartly we will write a function to do so.
 
 
-```r
+``` r
 plot_econ_curve = function(model) {
   plot(mpg ~ mph, data = econ, xlab = "Speed (Miles per Hour)", 
        ylab = "Fuel Efficiency (Miles per Gallon)", col = "dodgerblue", 
@@ -876,12 +876,12 @@ plot_econ_curve = function(model) {
 So now we first fit a simple linear regression to this data.
 
 
-```r
+``` r
 fit1 = lm(mpg ~ mph, data = econ)
 ```
 
 
-```r
+``` r
 par(mfrow = c(1, 2))
 plot_econ_curve(fit1)
 plot(fitted(fit1), resid(fit1), xlab = "Fitted", ylab = "Residuals", 
@@ -898,7 +898,7 @@ Pretty clearly we can do better. Yes fuel efficiency does increase as speed incr
 We will now add polynomial terms until we fit a suitable fit.
 
 
-```r
+``` r
 fit2 = lm(mpg ~ mph + I(mph ^ 2), data = econ)
 summary(fit2)
 ```
@@ -926,7 +926,7 @@ summary(fit2)
 ```
 
 
-```r
+``` r
 par(mfrow = c(1, 2))
 plot_econ_curve(fit2)
 plot(fitted(fit2), resid(fit2), xlab = "Fitted", ylab = "Residuals", 
@@ -941,7 +941,7 @@ plot(fitted(fit2), resid(fit2), xlab = "Fitted", ylab = "Residuals",
 While this model clearly fits much better, and the second order term is significant, we still see a pattern in the fitted versus residuals plot which suggests higher order terms will help. Also, we would expect the curve to flatten as speed increases or decreases, not go sharply downward as we see here.
 
 
-```r
+``` r
 fit3 = lm(mpg ~ mph + I(mph ^ 2) + I(mph ^ 3), data = econ)
 summary(fit3)
 ```
@@ -970,7 +970,7 @@ summary(fit3)
 ```
 
 
-```r
+``` r
 par(mfrow = c(1, 2))
 plot_econ_curve(fit3)
 plot(fitted(fit3), resid(fit3), xlab = "Fitted", ylab = "Residuals", 
@@ -985,7 +985,7 @@ plot(fitted(fit3), resid(fit3), xlab = "Fitted", ylab = "Residuals",
 Adding the third order term doesn't seem to help at all. The fitted curve hardly changes. This makes sense, since what we would like is for the curve to flatten at the extremes. For this we will need an even degree polynomial term.
 
 
-```r
+``` r
 fit4 = lm(mpg ~ mph + I(mph ^ 2) + I(mph ^ 3) + I(mph ^ 4), data = econ)
 summary(fit4)
 ```
@@ -1015,7 +1015,7 @@ summary(fit4)
 ```
 
 
-```r
+``` r
 par(mfrow = c(1, 2))
 plot_econ_curve(fit4)
 plot(fitted(fit4), resid(fit4), xlab = "Fitted", ylab = "Residuals", 
@@ -1030,7 +1030,7 @@ plot(fitted(fit4), resid(fit4), xlab = "Fitted", ylab = "Residuals",
 Now we are making progress. The fourth order term is significant with the other terms in the model. Also we are starting to see what we expected for low and high speed. However, there still seems to be a bit of a pattern in the residuals, so we will again try more higher order terms. We will add the fifth and sixth together, since adding the fifth will be similar to adding the third.
 
 
-```r
+``` r
 fit6 = lm(mpg ~ mph + I(mph ^ 2) + I(mph ^ 3) + I(mph ^ 4) + I(mph ^ 5) + I(mph^6), data = econ)
 summary(fit6)
 ```
@@ -1063,7 +1063,7 @@ summary(fit6)
 ```
 
 
-```r
+``` r
 par(mfrow = c(1, 2))
 plot_econ_curve(fit6)
 plot(fitted(fit6), resid(fit6), xlab = "Fitted", ylab = "Residuals", 
@@ -1082,7 +1082,7 @@ H_0: \beta_5 = \beta_6 = 0.
 \]
 
 
-```r
+``` r
 anova(fit4, fit6)
 ```
 
@@ -1101,7 +1101,7 @@ anova(fit4, fit6)
 So, this test does not reject the null hypothesis at a level of significance of $\alpha = 0.05$, however the p-value is still rather small, and the fitted versus residuals plot is much better for the model with the sixth order term. This makes the sixth order model a good choice. We could repeat this process one more time.
 
 
-```r
+``` r
 fit8 = lm(mpg ~ mph + I(mph ^ 2) + I(mph ^ 3) + I(mph ^ 4) + I(mph ^ 5)
           + I(mph ^ 6) + I(mph ^ 7) + I(mph ^ 8), data = econ)
 summary(fit8)
@@ -1135,7 +1135,7 @@ summary(fit8)
 ```
 
 
-```r
+``` r
 par(mfrow = c(1, 2))
 plot_econ_curve(fit8)
 plot(fitted(fit8), resid(fit8), xlab = "Fitted", ylab = "Residuals", 
@@ -1148,7 +1148,7 @@ plot(fitted(fit8), resid(fit8), xlab = "Fitted", ylab = "Residuals",
 \begin{center}\includegraphics{transformations_files/figure-latex/unnamed-chunk-55-1} \end{center}
 
 
-```r
+``` r
 summary(fit8)
 ```
 
@@ -1179,7 +1179,7 @@ summary(fit8)
 ## F-statistic: 128.1 on 8 and 19 DF,  p-value: 7.074e-15
 ```
 
-```r
+``` r
 anova(fit6, fit8)
 ```
 
@@ -1199,7 +1199,7 @@ Here we would clearly stick with `fit6`. The eighth order term is not significan
 As an aside, be aware that there is a quicker way to specify a model with many higher order terms.
 
 
-```r
+``` r
 fit6_alt = lm(mpg ~ poly(mph, 6), data = econ)
 all.equal(fitted(fit6), fitted(fit6_alt))
 ```
@@ -1211,7 +1211,7 @@ all.equal(fitted(fit6), fitted(fit6_alt))
 We first verify that this method produces the same fitted values. However, the estimated coefficients are different.
 
 
-```r
+``` r
 coef(fit6)
 ```
 
@@ -1222,7 +1222,7 @@ coef(fit6)
 ##  3.585201e-06 -1.401995e-08
 ```
 
-```r
+``` r
 coef(fit6_alt)
 ```
 
@@ -1236,7 +1236,7 @@ coef(fit6_alt)
 This is because `poly()` uses *orthogonal polynomials*, which solves an issue we will discuss in the next chapter. 
 
 
-```r
+``` r
 summary(fit6)
 ```
 
@@ -1267,7 +1267,7 @@ summary(fit6)
 ## F-statistic:   186 on 6 and 21 DF,  p-value: < 2.2e-16
 ```
 
-```r
+``` r
 summary(fit6_alt)
 ```
 
@@ -1302,7 +1302,7 @@ Notice though that the p-value for testing the degree 6 term is the same. Becaus
 To use `poly()` to obtain the same results as using `I()` repeatedly, we would need to set `raw = TRUE`.
 
 
-```r
+``` r
 fit6_alt2 = lm(mpg ~ poly(mph, 6, raw = TRUE), data = econ)
 coef(fit6_alt2)
 ```
@@ -1323,12 +1323,12 @@ We've now seen how to transform predictor and response variables. In this chapte
 ## Response Transformations {-}
 
 
-```r
+``` r
 initech = read.csv("data/initech.csv")
 ```
 
 
-```r
+``` r
 plot(salary ~ years, data = initech, col = "grey", pch = 20, cex = 1.5,
      main = "Salaries at Initech, By Seniority")
 ```
@@ -1338,7 +1338,7 @@ plot(salary ~ years, data = initech, col = "grey", pch = 20, cex = 1.5,
 \begin{center}\includegraphics{transformations_files/figure-latex/unnamed-chunk-62-1} \end{center}
 
 
-```r
+``` r
 initech_fit = lm(salary ~ years, data = initech)
 summary(initech_fit)
 ```
@@ -1365,7 +1365,7 @@ summary(initech_fit)
 ```
 
 
-```r
+``` r
 plot(salary ~ years, data = initech, col = "grey", pch = 20, cex = 1.5,
      main = "Salaries at Initech, By Seniority")
 abline(initech_fit, col = "darkorange", lwd = 2)
@@ -1376,7 +1376,7 @@ abline(initech_fit, col = "darkorange", lwd = 2)
 \begin{center}\includegraphics{transformations_files/figure-latex/unnamed-chunk-64-1} \end{center}
 
 
-```r
+``` r
 par(mfrow = c(1, 2))
 
 plot(fitted(initech_fit), resid(initech_fit), col = "grey", pch = 20,
@@ -1392,7 +1392,7 @@ qqline(resid(initech_fit), col = "dodgerblue", lwd = 2)
 \begin{center}\includegraphics{transformations_files/figure-latex/unnamed-chunk-65-1} \end{center}
 
 
-```r
+``` r
 initech_fit_log = lm(log(salary) ~ years, data = initech)
 ```
 
@@ -1401,7 +1401,7 @@ $$
 $$
 
 
-```r
+``` r
 plot(log(salary) ~ years, data = initech, col = "grey", pch = 20, cex = 1.5,
      main = "Salaries at Initech, By Seniority")
 abline(initech_fit_log, col = "darkorange", lwd = 2)
@@ -1416,7 +1416,7 @@ Y_i = \exp(\beta_0 + \beta_1 x_i) \cdot \exp(\epsilon_i)
 $$
 
 
-```r
+``` r
 plot(salary ~ years, data = initech, col = "grey", pch = 20, cex = 1.5,
      main = "Salaries at Initech, By Seniority")
 curve(exp(initech_fit_log$coef[1] + initech_fit_log$coef[2] * x),
@@ -1428,7 +1428,7 @@ curve(exp(initech_fit_log$coef[1] + initech_fit_log$coef[2] * x),
 \begin{center}\includegraphics{transformations_files/figure-latex/unnamed-chunk-68-1} \end{center}
 
 
-```r
+``` r
 par(mfrow = c(1, 2))
 
 plot(fitted(initech_fit_log), resid(initech_fit_log), col = "grey", pch = 20,
@@ -1444,7 +1444,7 @@ qqline(resid(initech_fit_log), col = "dodgerblue", lwd = 2)
 \begin{center}\includegraphics{transformations_files/figure-latex/unnamed-chunk-69-1} \end{center}
 
 
-```r
+``` r
 sqrt(mean(resid(initech_fit) ^ 2))
 ```
 
@@ -1452,7 +1452,7 @@ sqrt(mean(resid(initech_fit) ^ 2))
 ## [1] 27080.16
 ```
 
-```r
+``` r
 sqrt(mean(resid(initech_fit_log) ^ 2))
 ```
 
@@ -1461,7 +1461,7 @@ sqrt(mean(resid(initech_fit_log) ^ 2))
 ```
 
 
-```r
+``` r
 sqrt(mean((initech$salary - fitted(initech_fit)) ^ 2))
 ```
 
@@ -1469,7 +1469,7 @@ sqrt(mean((initech$salary - fitted(initech_fit)) ^ 2))
 ## [1] 27080.16
 ```
 
-```r
+``` r
 sqrt(mean((initech$salary - exp(fitted(initech_fit_log))) ^ 2))
 ```
 
@@ -1482,7 +1482,7 @@ sqrt(mean((initech$salary - exp(fitted(initech_fit_log))) ^ 2))
 ### A Quadratic Model
 
 
-```r
+``` r
 sim_quad = function(sample_size = 500) {
   x = runif(n = sample_size) * 5
   y = 3 + 5 * x ^ 2 + rnorm(n = sample_size, mean = 0, sd = 5)
@@ -1491,13 +1491,13 @@ sim_quad = function(sample_size = 500) {
 ```
 
 
-```r
+``` r
 set.seed(314)
 quad_data = sim_quad(sample_size = 200)
 ```
 
 
-```r
+``` r
 lin_fit = lm(y ~ x, data = quad_data)
 summary(lin_fit)
 ```
@@ -1524,7 +1524,7 @@ summary(lin_fit)
 ```
 
 
-```r
+``` r
 plot(y ~ x, data = quad_data, col = "grey", pch = 20, cex = 1.5,
      main = "Simulated Quadratic Data")
 abline(lin_fit, col = "darkorange", lwd = 2)
@@ -1535,7 +1535,7 @@ abline(lin_fit, col = "darkorange", lwd = 2)
 \begin{center}\includegraphics{transformations_files/figure-latex/unnamed-chunk-75-1} \end{center}
 
 
-```r
+``` r
 par(mfrow = c(1, 2))
 
 plot(fitted(lin_fit), resid(lin_fit), col = "grey", pch = 20,
@@ -1555,7 +1555,7 @@ Y_i = \beta_0 + \beta_1 x_i + \beta_2 x_i^2 + \epsilon_i
 $$
 
 
-```r
+``` r
 quad_fit = lm(y ~ x + I(x^2), data = quad_data)
 summary(quad_fit)
 ```
@@ -1583,7 +1583,7 @@ summary(quad_fit)
 ```
 
 
-```r
+``` r
 plot(y ~ x, data = quad_data, col = "grey", pch = 20, cex = 1.5,
      main = "Simulated Quadratic Data")
 curve(quad_fit$coef[1] + quad_fit$coef[2] * x + quad_fit$coef[3] * x ^ 2,
@@ -1595,7 +1595,7 @@ curve(quad_fit$coef[1] + quad_fit$coef[2] * x + quad_fit$coef[3] * x ^ 2,
 \begin{center}\includegraphics{transformations_files/figure-latex/unnamed-chunk-78-1} \end{center}
 
 
-```r
+``` r
 par(mfrow = c(1, 2))
 
 plot(fitted(quad_fit), resid(quad_fit), col = "grey", pch = 20,
@@ -1613,7 +1613,7 @@ qqline(resid(quad_fit), col = "dodgerblue", lwd = 2)
 ### Overfitting and Extrapolation
 
 
-```r
+``` r
 sim_for_perf = function() {
   x = seq(0, 10)
   y = 3 + x - 4 * x ^ 2 + rnorm(n = 11, mean = 0, sd = 25)
@@ -1622,13 +1622,13 @@ sim_for_perf = function() {
 ```
 
 
-```r
+``` r
 set.seed(1234)
 data_for_perf = sim_for_perf()
 ```
 
 
-```r
+``` r
 fit_correct = lm(y ~ x + I(x ^ 2), data = data_for_perf)
 fit_perfect = lm(y ~ x + I(x ^ 2) + I(x ^ 3) + I(x ^ 4) + I(x ^ 5) + I(x ^ 6) + 
                  I(x ^ 7) + I(x ^ 8) + I(x ^ 9) + I(x ^ 10), 
@@ -1636,7 +1636,7 @@ fit_perfect = lm(y ~ x + I(x ^ 2) + I(x ^ 3) + I(x ^ 4) + I(x ^ 5) + I(x ^ 6) +
 ```
 
 
-```r
+``` r
 x_plot = seq(-5, 15, by = 0.1)
 plot(y ~ x, data = data_for_perf, ylim = c(-450, 100), cex = 2, pch = 20)
 lines(x_plot, predict(fit_correct, newdata = data.frame(x = x_plot)),
@@ -1652,7 +1652,7 @@ lines(x_plot, predict(fit_perfect, newdata = data.frame(x = x_plot)),
 ### Comparing Polynomial Models
 
 
-```r
+``` r
 sim_higher = function(sample_size = 250) {
   x = runif(n = sample_size, min = -1, max = 1) * 2
   y = 3 + -6 * x ^ 2 + 1 * x ^ 4 + rnorm(n = sample_size, mean = 0, sd = 3)
@@ -1673,13 +1673,13 @@ Y_i = \beta_0 + \beta_1 x_i + \beta_2 x_i^2 + \beta_3 x_i^3 + \beta_4 x_i^4 + \b
 $$
 
 
-```r
+``` r
 set.seed(42)
 data_higher = sim_higher()
 ```
 
 
-```r
+``` r
 plot(y ~ x, data = data_higher, col = "grey", pch = 20, cex = 1.5,
      main = "Simulated Quartic Data")
 ```
@@ -1689,13 +1689,13 @@ plot(y ~ x, data = data_higher, col = "grey", pch = 20, cex = 1.5,
 \begin{center}\includegraphics{transformations_files/figure-latex/unnamed-chunk-86-1} \end{center}
 
 
-```r
+``` r
 fit_2 = lm(y ~ poly(x, 2), data = data_higher)
 fit_4 = lm(y ~ poly(x, 4), data = data_higher)
 ```
 
 
-```r
+``` r
 plot(y ~ x, data = data_higher, col = "grey", pch = 20, cex = 1.5,
      main = "Simulated Quartic Data")
 x_plot = seq(-5, 5, by = 0.05)
@@ -1710,7 +1710,7 @@ lines(x_plot, predict(fit_4, newdata = data.frame(x = x_plot)),
 \begin{center}\includegraphics{transformations_files/figure-latex/unnamed-chunk-88-1} \end{center}
 
 
-```r
+``` r
 par(mfrow = c(1, 2))
 
 plot(fitted(fit_2), resid(fit_2), col = "grey", pch = 20,
@@ -1726,7 +1726,7 @@ qqline(resid(fit_2), col = "dodgerblue", lwd = 2)
 \begin{center}\includegraphics{transformations_files/figure-latex/unnamed-chunk-89-1} \end{center}
 
 
-```r
+``` r
 par(mfrow = c(1, 2))
 
 plot(fitted(fit_4), resid(fit_4), col = "grey", pch = 20,
@@ -1742,7 +1742,7 @@ qqline(resid(fit_4), col = "dodgerblue", lwd = 2)
 \begin{center}\includegraphics{transformations_files/figure-latex/unnamed-chunk-90-1} \end{center}
 
 
-```r
+``` r
 anova(fit_2, fit_4)
 ```
 
@@ -1759,12 +1759,12 @@ anova(fit_2, fit_4)
 ```
 
 
-```r
+``` r
 fit_6 = lm(y ~ poly(x, 6), data = data_higher)
 ```
 
 
-```r
+``` r
 anova(fit_4, fit_6)
 ```
 
@@ -1785,14 +1785,14 @@ Y_i = \beta_0 + \beta_1 x_i + \beta_2 x_i^2 + \beta_3 x_i^3 + \beta_4 x_i^4 + \e
 $$
 
 
-```r
+``` r
 fit_4a = lm(y ~ poly(x, degree = 4), data = data_higher)
 fit_4b = lm(y ~ poly(x, degree = 4, raw = TRUE), data = data_higher)
 fit_4c = lm(y ~ x + I(x^2) + I(x^3) + I(x^4), data = data_higher)
 ```
 
 
-```r
+``` r
 coef(fit_4a)
 ```
 
@@ -1803,7 +1803,7 @@ coef(fit_4a)
 ##             0.669874            20.519759
 ```
 
-```r
+``` r
 coef(fit_4b)
 ```
 
@@ -1816,7 +1816,7 @@ coef(fit_4b)
 ##                        1.0282139
 ```
 
-```r
+``` r
 coef(fit_4c)
 ```
 
@@ -1826,7 +1826,7 @@ coef(fit_4c)
 ```
 
 
-```r
+``` r
 unname(coef(fit_4a))
 ```
 
@@ -1834,7 +1834,7 @@ unname(coef(fit_4a))
 ## [1]  -1.980036  -2.053929 -49.344752   0.669874  20.519759
 ```
 
-```r
+``` r
 unname(coef(fit_4b))
 ```
 
@@ -1842,7 +1842,7 @@ unname(coef(fit_4b))
 ## [1]  2.9996256 -0.3880250 -6.1511166  0.1269046  1.0282139
 ```
 
-```r
+``` r
 unname(coef(fit_4c))
 ```
 
@@ -1851,7 +1851,7 @@ unname(coef(fit_4c))
 ```
 
 
-```r
+``` r
 all.equal(fitted(fit_4a),
           fitted(fit_4b))
 ```
@@ -1861,7 +1861,7 @@ all.equal(fitted(fit_4a),
 ```
 
 
-```r
+``` r
 all.equal(resid(fit_4a),
           resid(fit_4b))
 ```
@@ -1871,7 +1871,7 @@ all.equal(resid(fit_4a),
 ```
 
 
-```r
+``` r
 summary(fit_4a)
 ```
 
@@ -1900,7 +1900,7 @@ summary(fit_4a)
 ```
 
 
-```r
+``` r
 summary(fit_4c)
 ```
 
@@ -1931,7 +1931,7 @@ summary(fit_4c)
 ### Inhibit Function
 
 
-```r
+``` r
 coef(lm(y ~ x + x ^ 2, data = quad_data))
 ```
 
@@ -1940,7 +1940,7 @@ coef(lm(y ~ x + x ^ 2, data = quad_data))
 ##   -18.32715    24.87163
 ```
 
-```r
+``` r
 coef(lm(y ~ x + I(x ^ 2), data = quad_data))
 ```
 
@@ -1950,7 +1950,7 @@ coef(lm(y ~ x + I(x ^ 2), data = quad_data))
 ```
 
 
-```r
+``` r
 coef(lm(y ~ x + x:x, data = quad_data))
 ```
 
@@ -1959,7 +1959,7 @@ coef(lm(y ~ x + x:x, data = quad_data))
 ##   -18.32715    24.87163
 ```
 
-```r
+``` r
 coef(lm(y ~ x * x, data = quad_data))
 ```
 
@@ -1968,7 +1968,7 @@ coef(lm(y ~ x * x, data = quad_data))
 ##   -18.32715    24.87163
 ```
 
-```r
+``` r
 coef(lm(y ~ x ^ 2, data = quad_data))
 ```
 
@@ -1977,7 +1977,7 @@ coef(lm(y ~ x ^ 2, data = quad_data))
 ##   -18.32715    24.87163
 ```
 
-```r
+``` r
 coef(lm(y ~ x + x ^ 2, data = quad_data))
 ```
 
@@ -1987,7 +1987,7 @@ coef(lm(y ~ x + x ^ 2, data = quad_data))
 ```
 
 
-```r
+``` r
 coef(lm(y ~ I(x + x), data = quad_data))
 ```
 
@@ -1996,7 +1996,7 @@ coef(lm(y ~ I(x + x), data = quad_data))
 ##   -18.32715    12.43582
 ```
 
-```r
+``` r
 coef(lm(y ~ x + x, data = quad_data))
 ```
 
@@ -2010,7 +2010,7 @@ coef(lm(y ~ x + x, data = quad_data))
 
 
 
-```r
+``` r
 pairs(autompg)
 ```
 
@@ -2019,7 +2019,7 @@ pairs(autompg)
 \begin{center}\includegraphics{transformations_files/figure-latex/unnamed-chunk-105-1} \end{center}
 
 
-```r
+``` r
 mpg_hp = lm(mpg ~ hp, data = autompg)
 
 par(mfrow = c(1, 2))
@@ -2037,7 +2037,7 @@ abline(h = 0, lty = 2, col = "darkorange", lwd = 2)
 \begin{center}\includegraphics{transformations_files/figure-latex/unnamed-chunk-106-1} \end{center}
 
 
-```r
+``` r
 mpg_hp_log = lm(mpg ~ hp + I(hp ^ 2), data = autompg)
 
 par(mfrow = c(1, 2))
@@ -2057,7 +2057,7 @@ abline(h = 0, lty = 2, col = "darkorange", lwd = 2)
 \begin{center}\includegraphics{transformations_files/figure-latex/unnamed-chunk-107-1} \end{center}
 
 
-```r
+``` r
 mpg_hp_log = lm(log(mpg) ~ hp + I(hp ^ 2), data = autompg)
 
 par(mfrow = c(1, 2))
@@ -2077,7 +2077,7 @@ abline(h = 0, lty = 2, col = "darkorange", lwd = 2)
 \begin{center}\includegraphics{transformations_files/figure-latex/unnamed-chunk-108-1} \end{center}
 
 
-```r
+``` r
 mpg_hp_loglog = lm(log(mpg) ~ log(hp), data = autompg)
 
 par(mfrow = c(1, 2))
@@ -2094,12 +2094,12 @@ abline(h = 0, lty = 2, col = "darkorange", lwd = 2)
 \begin{center}\includegraphics{transformations_files/figure-latex/unnamed-chunk-109-1} \end{center}
 
 
-```r
+``` r
 big_model = lm(mpg ~ disp * hp * domestic, data = autompg)
 ```
 
 
-```r
+``` r
 qqnorm(resid(big_model), col = "darkgrey")
 qqline(resid(big_model), col = "dodgerblue", lwd = 2)
 ```
@@ -2109,7 +2109,7 @@ qqline(resid(big_model), col = "dodgerblue", lwd = 2)
 \begin{center}\includegraphics{transformations_files/figure-latex/unnamed-chunk-111-1} \end{center}
 
 
-```r
+``` r
 bigger_model = lm(log(mpg) ~ disp * hp * domestic + 
                I(disp ^ 2) + I(hp ^ 2), data = autompg)
 summary(bigger_model)
@@ -2146,7 +2146,7 @@ summary(bigger_model)
 ```
 
 
-```r
+``` r
 qqnorm(resid(bigger_model), col = "darkgrey")
 qqline(resid(bigger_model), col = "dodgerblue", lwd = 2)
 ```
@@ -2161,4 +2161,4 @@ The `R` Markdown file for this chapter can be found here:
 
 - [`transformations.Rmd`](transformations.Rmd){target="_blank"}
 
-The file was created using `R` version `4.3.2`.
+The file was created using `R` version `4.4.1`.
diff --git a/variable-selection-and-model-building.html b/variable-selection-and-model-building.html
index 86f2bb9..2275b77 100644
--- a/variable-selection-and-model-building.html
+++ b/variable-selection-and-model-building.html
@@ -6,7 +6,7 @@
   <meta http-equiv="X-UA-Compatible" content="IE=edge" />
   <title>Chapter 16 Variable Selection and Model Building | Applied Statistics with R</title>
   <meta name="description" content="Chapter 16 Variable Selection and Model Building | Applied Statistics with R" />
-  <meta name="generator" content="bookdown 0.37 and GitBook 2.6.7" />
+  <meta name="generator" content="bookdown 0.40 and GitBook 2.6.7" />
 
   <meta property="og:title" content="Chapter 16 Variable Selection and Model Building | Applied Statistics with R" />
   <meta property="og:type" content="book" />
@@ -55,7 +55,7 @@
 
 <style type="text/css">
 pre > code.sourceCode { white-space: pre; position: relative; }
-pre > code.sourceCode > span { display: inline-block; line-height: 1.25; }
+pre > code.sourceCode > span { line-height: 1.25; }
 pre > code.sourceCode > span:empty { height: 1.2em; }
 .sourceCode { overflow: visible; }
 code.sourceCode > span { color: inherit; text-decoration: inherit; }
@@ -558,33 +558,33 @@ <h3><span class="header-section-number">16.1.3</span> Adjusted R-Squared<a href=
 <h3><span class="header-section-number">16.1.4</span> Cross-Validated RMSE<a href="variable-selection-and-model-building.html#cross-validated-rmse" class="anchor-section" aria-label="Anchor link to header"></a></h3>
 <p>Each of the previous three metrics explicitly used <span class="math inline">\(p\)</span>, the number of parameters, in their calculations. Thus, they all explicitly limit the size of models chosen when used to compare models.</p>
 <p>We’ll now briefly introduce <strong>overfitting</strong> and <strong>cross-validation</strong>.</p>
-<div class="sourceCode" id="cb1291"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1291-1"><a href="variable-selection-and-model-building.html#cb1291-1" aria-hidden="true" tabindex="-1"></a>make_poly_data <span class="ot">=</span> <span class="cf">function</span>(<span class="at">sample_size =</span> <span class="dv">11</span>) {</span>
-<span id="cb1291-2"><a href="variable-selection-and-model-building.html#cb1291-2" aria-hidden="true" tabindex="-1"></a>  x <span class="ot">=</span> <span class="fu">seq</span>(<span class="dv">0</span>, <span class="dv">10</span>)</span>
-<span id="cb1291-3"><a href="variable-selection-and-model-building.html#cb1291-3" aria-hidden="true" tabindex="-1"></a>  y <span class="ot">=</span> <span class="dv">3</span> <span class="sc">+</span> x <span class="sc">+</span> <span class="dv">4</span> <span class="sc">*</span> x <span class="sc">^</span> <span class="dv">2</span> <span class="sc">+</span> <span class="fu">rnorm</span>(<span class="at">n =</span> sample_size, <span class="at">mean =</span> <span class="dv">0</span>, <span class="at">sd =</span> <span class="dv">20</span>)</span>
-<span id="cb1291-4"><a href="variable-selection-and-model-building.html#cb1291-4" aria-hidden="true" tabindex="-1"></a>  <span class="fu">data.frame</span>(x, y)</span>
-<span id="cb1291-5"><a href="variable-selection-and-model-building.html#cb1291-5" aria-hidden="true" tabindex="-1"></a>}</span></code></pre></div>
-<div class="sourceCode" id="cb1292"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1292-1"><a href="variable-selection-and-model-building.html#cb1292-1" aria-hidden="true" tabindex="-1"></a><span class="fu">set.seed</span>(<span class="dv">1234</span>)</span>
-<span id="cb1292-2"><a href="variable-selection-and-model-building.html#cb1292-2" aria-hidden="true" tabindex="-1"></a>poly_data <span class="ot">=</span> <span class="fu">make_poly_data</span>()</span></code></pre></div>
+<div class="sourceCode" id="cb1291"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1291-1"><a href="variable-selection-and-model-building.html#cb1291-1" tabindex="-1"></a>make_poly_data <span class="ot">=</span> <span class="cf">function</span>(<span class="at">sample_size =</span> <span class="dv">11</span>) {</span>
+<span id="cb1291-2"><a href="variable-selection-and-model-building.html#cb1291-2" tabindex="-1"></a>  x <span class="ot">=</span> <span class="fu">seq</span>(<span class="dv">0</span>, <span class="dv">10</span>)</span>
+<span id="cb1291-3"><a href="variable-selection-and-model-building.html#cb1291-3" tabindex="-1"></a>  y <span class="ot">=</span> <span class="dv">3</span> <span class="sc">+</span> x <span class="sc">+</span> <span class="dv">4</span> <span class="sc">*</span> x <span class="sc">^</span> <span class="dv">2</span> <span class="sc">+</span> <span class="fu">rnorm</span>(<span class="at">n =</span> sample_size, <span class="at">mean =</span> <span class="dv">0</span>, <span class="at">sd =</span> <span class="dv">20</span>)</span>
+<span id="cb1291-4"><a href="variable-selection-and-model-building.html#cb1291-4" tabindex="-1"></a>  <span class="fu">data.frame</span>(x, y)</span>
+<span id="cb1291-5"><a href="variable-selection-and-model-building.html#cb1291-5" tabindex="-1"></a>}</span></code></pre></div>
+<div class="sourceCode" id="cb1292"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1292-1"><a href="variable-selection-and-model-building.html#cb1292-1" tabindex="-1"></a><span class="fu">set.seed</span>(<span class="dv">1234</span>)</span>
+<span id="cb1292-2"><a href="variable-selection-and-model-building.html#cb1292-2" tabindex="-1"></a>poly_data <span class="ot">=</span> <span class="fu">make_poly_data</span>()</span></code></pre></div>
 <p>Here we have generated data where the mean of <span class="math inline">\(Y\)</span> is a quadratic function of a single predictor <span class="math inline">\(x\)</span>, specifically,</p>
 <p><span class="math display">\[
 Y = 3 + x + 4 x ^ 2 + \epsilon.
 \]</span></p>
 <p>We’ll now fit two models to this data, one which has the correct form, quadratic, and one that is large, which includes terms up to and including an eighth degree.</p>
-<div class="sourceCode" id="cb1293"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1293-1"><a href="variable-selection-and-model-building.html#cb1293-1" aria-hidden="true" tabindex="-1"></a>fit_quad <span class="ot">=</span> <span class="fu">lm</span>(y <span class="sc">~</span> <span class="fu">poly</span>(x, <span class="at">degree =</span> <span class="dv">2</span>), <span class="at">data =</span> poly_data)</span>
-<span id="cb1293-2"><a href="variable-selection-and-model-building.html#cb1293-2" aria-hidden="true" tabindex="-1"></a>fit_big  <span class="ot">=</span> <span class="fu">lm</span>(y <span class="sc">~</span> <span class="fu">poly</span>(x, <span class="at">degree =</span> <span class="dv">8</span>), <span class="at">data =</span> poly_data)</span></code></pre></div>
+<div class="sourceCode" id="cb1293"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1293-1"><a href="variable-selection-and-model-building.html#cb1293-1" tabindex="-1"></a>fit_quad <span class="ot">=</span> <span class="fu">lm</span>(y <span class="sc">~</span> <span class="fu">poly</span>(x, <span class="at">degree =</span> <span class="dv">2</span>), <span class="at">data =</span> poly_data)</span>
+<span id="cb1293-2"><a href="variable-selection-and-model-building.html#cb1293-2" tabindex="-1"></a>fit_big  <span class="ot">=</span> <span class="fu">lm</span>(y <span class="sc">~</span> <span class="fu">poly</span>(x, <span class="at">degree =</span> <span class="dv">8</span>), <span class="at">data =</span> poly_data)</span></code></pre></div>
 <p>We then plot the data and the results of the two models.</p>
-<div class="sourceCode" id="cb1294"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1294-1"><a href="variable-selection-and-model-building.html#cb1294-1" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(y <span class="sc">~</span> x, <span class="at">data =</span> poly_data, <span class="at">ylim =</span> <span class="fu">c</span>(<span class="sc">-</span><span class="dv">100</span>, <span class="dv">400</span>), <span class="at">cex =</span> <span class="dv">2</span>, <span class="at">pch =</span> <span class="dv">20</span>)</span>
-<span id="cb1294-2"><a href="variable-selection-and-model-building.html#cb1294-2" aria-hidden="true" tabindex="-1"></a>xplot <span class="ot">=</span> <span class="fu">seq</span>(<span class="dv">0</span>, <span class="dv">10</span>, <span class="at">by =</span> <span class="fl">0.1</span>)</span>
-<span id="cb1294-3"><a href="variable-selection-and-model-building.html#cb1294-3" aria-hidden="true" tabindex="-1"></a><span class="fu">lines</span>(xplot, <span class="fu">predict</span>(fit_quad, <span class="at">newdata =</span> <span class="fu">data.frame</span>(<span class="at">x =</span> xplot)),</span>
-<span id="cb1294-4"><a href="variable-selection-and-model-building.html#cb1294-4" aria-hidden="true" tabindex="-1"></a>      <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>, <span class="at">lty =</span> <span class="dv">1</span>)</span>
-<span id="cb1294-5"><a href="variable-selection-and-model-building.html#cb1294-5" aria-hidden="true" tabindex="-1"></a><span class="fu">lines</span>(xplot, <span class="fu">predict</span>(fit_big, <span class="at">newdata =</span> <span class="fu">data.frame</span>(<span class="at">x =</span> xplot)),</span>
-<span id="cb1294-6"><a href="variable-selection-and-model-building.html#cb1294-6" aria-hidden="true" tabindex="-1"></a>      <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>, <span class="at">lty =</span> <span class="dv">2</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1294"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1294-1"><a href="variable-selection-and-model-building.html#cb1294-1" tabindex="-1"></a><span class="fu">plot</span>(y <span class="sc">~</span> x, <span class="at">data =</span> poly_data, <span class="at">ylim =</span> <span class="fu">c</span>(<span class="sc">-</span><span class="dv">100</span>, <span class="dv">400</span>), <span class="at">cex =</span> <span class="dv">2</span>, <span class="at">pch =</span> <span class="dv">20</span>)</span>
+<span id="cb1294-2"><a href="variable-selection-and-model-building.html#cb1294-2" tabindex="-1"></a>xplot <span class="ot">=</span> <span class="fu">seq</span>(<span class="dv">0</span>, <span class="dv">10</span>, <span class="at">by =</span> <span class="fl">0.1</span>)</span>
+<span id="cb1294-3"><a href="variable-selection-and-model-building.html#cb1294-3" tabindex="-1"></a><span class="fu">lines</span>(xplot, <span class="fu">predict</span>(fit_quad, <span class="at">newdata =</span> <span class="fu">data.frame</span>(<span class="at">x =</span> xplot)),</span>
+<span id="cb1294-4"><a href="variable-selection-and-model-building.html#cb1294-4" tabindex="-1"></a>      <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>, <span class="at">lty =</span> <span class="dv">1</span>)</span>
+<span id="cb1294-5"><a href="variable-selection-and-model-building.html#cb1294-5" tabindex="-1"></a><span class="fu">lines</span>(xplot, <span class="fu">predict</span>(fit_big, <span class="at">newdata =</span> <span class="fu">data.frame</span>(<span class="at">x =</span> xplot)),</span>
+<span id="cb1294-6"><a href="variable-selection-and-model-building.html#cb1294-6" tabindex="-1"></a>      <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>, <span class="at">lty =</span> <span class="dv">2</span>)</span></code></pre></div>
 <p><img src="selection_files/figure-html/unnamed-chunk-5-1.png" width="672" style="display: block; margin: auto;" /></p>
 <p>We can see that the solid blue curve models this data rather nicely. The dashed orange curve fits the points better, making smaller errors, however it is unlikely that it is correctly modeling the true relationship between <span class="math inline">\(x\)</span> and <span class="math inline">\(y\)</span>. It is fitting the random noise. This is an example of <strong>overfitting</strong>.</p>
 <p>We see that the larger model indeed has a lower <span class="math inline">\(\text{RMSE}\)</span>.</p>
-<div class="sourceCode" id="cb1295"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1295-1"><a href="variable-selection-and-model-building.html#cb1295-1" aria-hidden="true" tabindex="-1"></a><span class="fu">sqrt</span>(<span class="fu">mean</span>(<span class="fu">resid</span>(fit_quad) <span class="sc">^</span> <span class="dv">2</span>))</span></code></pre></div>
+<div class="sourceCode" id="cb1295"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1295-1"><a href="variable-selection-and-model-building.html#cb1295-1" tabindex="-1"></a><span class="fu">sqrt</span>(<span class="fu">mean</span>(<span class="fu">resid</span>(fit_quad) <span class="sc">^</span> <span class="dv">2</span>))</span></code></pre></div>
 <pre><code>## [1] 17.61812</code></pre>
-<div class="sourceCode" id="cb1297"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1297-1"><a href="variable-selection-and-model-building.html#cb1297-1" aria-hidden="true" tabindex="-1"></a><span class="fu">sqrt</span>(<span class="fu">mean</span>(<span class="fu">resid</span>(fit_big) <span class="sc">^</span> <span class="dv">2</span>))</span></code></pre></div>
+<div class="sourceCode" id="cb1297"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1297-1"><a href="variable-selection-and-model-building.html#cb1297-1" tabindex="-1"></a><span class="fu">sqrt</span>(<span class="fu">mean</span>(<span class="fu">resid</span>(fit_big) <span class="sc">^</span> <span class="dv">2</span>))</span></code></pre></div>
 <pre><code>## [1] 10.4197</code></pre>
 <p>To correct for this, we will introduce cross-validation. We define the leave-one-out cross-validated RMSE to be</p>
 <p><span class="math display">\[
@@ -605,24 +605,24 @@ <h3><span class="header-section-number">16.1.4</span> Cross-Validated RMSE<a hre
 \]</span></p>
 <p>where <span class="math inline">\(h_i\)</span> are the leverages and <span class="math inline">\(e_i\)</span> are the usual residuals. This is great, because now we can obtain the LOOCV <span class="math inline">\(\text{RMSE}\)</span> by fitting only one model! In practice 5 or 10 fold cross-validation are much more popular. For example, in 5-fold cross-validation, the model is fit 5 times, each time leaving out a fifth of the data, then predicting on those values. We’ll leave in-depth examination of cross-validation to a machine learning course, and simply use LOOCV here.</p>
 <p>Let’s calculate LOOCV <span class="math inline">\(\text{RMSE}\)</span> for both models, then discuss <em>why</em> we want to do so. We first write a function which calculates the LOOCV <span class="math inline">\(\text{RMSE}\)</span> as defined using the shortcut formula for linear models.</p>
-<div class="sourceCode" id="cb1299"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1299-1"><a href="variable-selection-and-model-building.html#cb1299-1" aria-hidden="true" tabindex="-1"></a>calc_loocv_rmse <span class="ot">=</span> <span class="cf">function</span>(model) {</span>
-<span id="cb1299-2"><a href="variable-selection-and-model-building.html#cb1299-2" aria-hidden="true" tabindex="-1"></a>  <span class="fu">sqrt</span>(<span class="fu">mean</span>((<span class="fu">resid</span>(model) <span class="sc">/</span> (<span class="dv">1</span> <span class="sc">-</span> <span class="fu">hatvalues</span>(model))) <span class="sc">^</span> <span class="dv">2</span>))</span>
-<span id="cb1299-3"><a href="variable-selection-and-model-building.html#cb1299-3" aria-hidden="true" tabindex="-1"></a>}</span></code></pre></div>
+<div class="sourceCode" id="cb1299"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1299-1"><a href="variable-selection-and-model-building.html#cb1299-1" tabindex="-1"></a>calc_loocv_rmse <span class="ot">=</span> <span class="cf">function</span>(model) {</span>
+<span id="cb1299-2"><a href="variable-selection-and-model-building.html#cb1299-2" tabindex="-1"></a>  <span class="fu">sqrt</span>(<span class="fu">mean</span>((<span class="fu">resid</span>(model) <span class="sc">/</span> (<span class="dv">1</span> <span class="sc">-</span> <span class="fu">hatvalues</span>(model))) <span class="sc">^</span> <span class="dv">2</span>))</span>
+<span id="cb1299-3"><a href="variable-selection-and-model-building.html#cb1299-3" tabindex="-1"></a>}</span></code></pre></div>
 <p>Then calculate the metric for both models.</p>
-<div class="sourceCode" id="cb1300"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1300-1"><a href="variable-selection-and-model-building.html#cb1300-1" aria-hidden="true" tabindex="-1"></a><span class="fu">calc_loocv_rmse</span>(fit_quad)</span></code></pre></div>
+<div class="sourceCode" id="cb1300"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1300-1"><a href="variable-selection-and-model-building.html#cb1300-1" tabindex="-1"></a><span class="fu">calc_loocv_rmse</span>(fit_quad)</span></code></pre></div>
 <pre><code>## [1] 23.57189</code></pre>
-<div class="sourceCode" id="cb1302"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1302-1"><a href="variable-selection-and-model-building.html#cb1302-1" aria-hidden="true" tabindex="-1"></a><span class="fu">calc_loocv_rmse</span>(fit_big)</span></code></pre></div>
+<div class="sourceCode" id="cb1302"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1302-1"><a href="variable-selection-and-model-building.html#cb1302-1" tabindex="-1"></a><span class="fu">calc_loocv_rmse</span>(fit_big)</span></code></pre></div>
 <pre><code>## [1] 1334.357</code></pre>
 <p>Now we see that the quadratic model has a much smaller LOOCV <span class="math inline">\(\text{RMSE}\)</span>, so we would prefer this quadratic model. This is because the large model has <em>severely</em> over-fit the data. By leaving a single data point out and fitting the large model, the resulting fit is much different than the fit using all of the data. For example, let’s leave out the third data point and fit both models, then plot the result.</p>
-<div class="sourceCode" id="cb1304"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1304-1"><a href="variable-selection-and-model-building.html#cb1304-1" aria-hidden="true" tabindex="-1"></a>fit_quad_removed <span class="ot">=</span> <span class="fu">lm</span>(y <span class="sc">~</span> <span class="fu">poly</span>(x, <span class="at">degree =</span> <span class="dv">2</span>), <span class="at">data =</span> poly_data[<span class="sc">-</span><span class="dv">3</span>, ])</span>
-<span id="cb1304-2"><a href="variable-selection-and-model-building.html#cb1304-2" aria-hidden="true" tabindex="-1"></a>fit_big_removed  <span class="ot">=</span> <span class="fu">lm</span>(y <span class="sc">~</span> <span class="fu">poly</span>(x, <span class="at">degree =</span> <span class="dv">8</span>), <span class="at">data =</span> poly_data[<span class="sc">-</span><span class="dv">3</span>, ])</span>
-<span id="cb1304-3"><a href="variable-selection-and-model-building.html#cb1304-3" aria-hidden="true" tabindex="-1"></a></span>
-<span id="cb1304-4"><a href="variable-selection-and-model-building.html#cb1304-4" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(y <span class="sc">~</span> x, <span class="at">data =</span> poly_data, <span class="at">ylim =</span> <span class="fu">c</span>(<span class="sc">-</span><span class="dv">100</span>, <span class="dv">400</span>), <span class="at">cex =</span> <span class="dv">2</span>, <span class="at">pch =</span> <span class="dv">20</span>)</span>
-<span id="cb1304-5"><a href="variable-selection-and-model-building.html#cb1304-5" aria-hidden="true" tabindex="-1"></a>xplot <span class="ot">=</span> <span class="fu">seq</span>(<span class="dv">0</span>, <span class="dv">10</span>, <span class="at">by =</span> <span class="fl">0.1</span>)</span>
-<span id="cb1304-6"><a href="variable-selection-and-model-building.html#cb1304-6" aria-hidden="true" tabindex="-1"></a><span class="fu">lines</span>(xplot, <span class="fu">predict</span>(fit_quad_removed, <span class="at">newdata =</span> <span class="fu">data.frame</span>(<span class="at">x =</span> xplot)),</span>
-<span id="cb1304-7"><a href="variable-selection-and-model-building.html#cb1304-7" aria-hidden="true" tabindex="-1"></a>      <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>, <span class="at">lty =</span> <span class="dv">1</span>)</span>
-<span id="cb1304-8"><a href="variable-selection-and-model-building.html#cb1304-8" aria-hidden="true" tabindex="-1"></a><span class="fu">lines</span>(xplot, <span class="fu">predict</span>(fit_big_removed, <span class="at">newdata =</span> <span class="fu">data.frame</span>(<span class="at">x =</span> xplot)),</span>
-<span id="cb1304-9"><a href="variable-selection-and-model-building.html#cb1304-9" aria-hidden="true" tabindex="-1"></a>      <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>, <span class="at">lty =</span> <span class="dv">2</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1304"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1304-1"><a href="variable-selection-and-model-building.html#cb1304-1" tabindex="-1"></a>fit_quad_removed <span class="ot">=</span> <span class="fu">lm</span>(y <span class="sc">~</span> <span class="fu">poly</span>(x, <span class="at">degree =</span> <span class="dv">2</span>), <span class="at">data =</span> poly_data[<span class="sc">-</span><span class="dv">3</span>, ])</span>
+<span id="cb1304-2"><a href="variable-selection-and-model-building.html#cb1304-2" tabindex="-1"></a>fit_big_removed  <span class="ot">=</span> <span class="fu">lm</span>(y <span class="sc">~</span> <span class="fu">poly</span>(x, <span class="at">degree =</span> <span class="dv">8</span>), <span class="at">data =</span> poly_data[<span class="sc">-</span><span class="dv">3</span>, ])</span>
+<span id="cb1304-3"><a href="variable-selection-and-model-building.html#cb1304-3" tabindex="-1"></a></span>
+<span id="cb1304-4"><a href="variable-selection-and-model-building.html#cb1304-4" tabindex="-1"></a><span class="fu">plot</span>(y <span class="sc">~</span> x, <span class="at">data =</span> poly_data, <span class="at">ylim =</span> <span class="fu">c</span>(<span class="sc">-</span><span class="dv">100</span>, <span class="dv">400</span>), <span class="at">cex =</span> <span class="dv">2</span>, <span class="at">pch =</span> <span class="dv">20</span>)</span>
+<span id="cb1304-5"><a href="variable-selection-and-model-building.html#cb1304-5" tabindex="-1"></a>xplot <span class="ot">=</span> <span class="fu">seq</span>(<span class="dv">0</span>, <span class="dv">10</span>, <span class="at">by =</span> <span class="fl">0.1</span>)</span>
+<span id="cb1304-6"><a href="variable-selection-and-model-building.html#cb1304-6" tabindex="-1"></a><span class="fu">lines</span>(xplot, <span class="fu">predict</span>(fit_quad_removed, <span class="at">newdata =</span> <span class="fu">data.frame</span>(<span class="at">x =</span> xplot)),</span>
+<span id="cb1304-7"><a href="variable-selection-and-model-building.html#cb1304-7" tabindex="-1"></a>      <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>, <span class="at">lty =</span> <span class="dv">1</span>)</span>
+<span id="cb1304-8"><a href="variable-selection-and-model-building.html#cb1304-8" tabindex="-1"></a><span class="fu">lines</span>(xplot, <span class="fu">predict</span>(fit_big_removed, <span class="at">newdata =</span> <span class="fu">data.frame</span>(<span class="at">x =</span> xplot)),</span>
+<span id="cb1304-9"><a href="variable-selection-and-model-building.html#cb1304-9" tabindex="-1"></a>      <span class="at">col =</span> <span class="st">&quot;darkorange&quot;</span>, <span class="at">lwd =</span> <span class="dv">2</span>, <span class="at">lty =</span> <span class="dv">2</span>)</span></code></pre></div>
 <p><img src="selection_files/figure-html/unnamed-chunk-9-1.png" width="672" style="display: block; margin: auto;" /></p>
 <p>We see that on average, the solid blue line for the quadratic model has similar errors as before. It has changed very slightly. However, the dashed orange line for the large model, has a huge error at the point that was removed and is much different than the previous fit.</p>
 <p>This is the purpose of cross-validation. By assessing how the model fits points that were not used to perform the regression, we get an idea of how well the model will work for future observations. It assesses how well the model works in general, not simply on the observed data.</p>
@@ -631,9 +631,9 @@ <h3><span class="header-section-number">16.1.4</span> Cross-Validated RMSE<a hre
 <div id="selection-procedures" class="section level2 hasAnchor" number="16.2">
 <h2><span class="header-section-number">16.2</span> Selection Procedures<a href="variable-selection-and-model-building.html#selection-procedures" class="anchor-section" aria-label="Anchor link to header"></a></h2>
 <p>We’ve now seen a number of model quality criteria, but now we need to address which models to consider. Model selection involves both a quality criterion, plus a search procedure.</p>
-<div class="sourceCode" id="cb1305"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1305-1"><a href="variable-selection-and-model-building.html#cb1305-1" aria-hidden="true" tabindex="-1"></a><span class="fu">library</span>(faraway)</span>
-<span id="cb1305-2"><a href="variable-selection-and-model-building.html#cb1305-2" aria-hidden="true" tabindex="-1"></a>hipcenter_mod <span class="ot">=</span> <span class="fu">lm</span>(hipcenter <span class="sc">~</span> ., <span class="at">data =</span> seatpos)</span>
-<span id="cb1305-3"><a href="variable-selection-and-model-building.html#cb1305-3" aria-hidden="true" tabindex="-1"></a><span class="fu">coef</span>(hipcenter_mod)</span></code></pre></div>
+<div class="sourceCode" id="cb1305"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1305-1"><a href="variable-selection-and-model-building.html#cb1305-1" tabindex="-1"></a><span class="fu">library</span>(faraway)</span>
+<span id="cb1305-2"><a href="variable-selection-and-model-building.html#cb1305-2" tabindex="-1"></a>hipcenter_mod <span class="ot">=</span> <span class="fu">lm</span>(hipcenter <span class="sc">~</span> ., <span class="at">data =</span> seatpos)</span>
+<span id="cb1305-3"><a href="variable-selection-and-model-building.html#cb1305-3" tabindex="-1"></a><span class="fu">coef</span>(hipcenter_mod)</span></code></pre></div>
 <pre><code>##  (Intercept)          Age       Weight      HtShoes           Ht       Seated 
 ## 436.43212823   0.77571620   0.02631308  -2.69240774   0.60134458   0.53375170 
 ##          Arm        Thigh          Leg 
@@ -647,7 +647,7 @@ <h2><span class="header-section-number">16.2</span> Selection Procedures<a href=
 <div id="backward-search" class="section level3 hasAnchor" number="16.2.1">
 <h3><span class="header-section-number">16.2.1</span> Backward Search<a href="variable-selection-and-model-building.html#backward-search" class="anchor-section" aria-label="Anchor link to header"></a></h3>
 <p>Backward selection procedures start with all possible predictors in the model, then consider how deleting a single predictor will affect a chosen metric. Let’s try this on the <code>seatpos</code> data. We will use the <code>step()</code> function in <code>R</code> which by default uses <span class="math inline">\(\text{AIC}\)</span> as its metric of choice.</p>
-<div class="sourceCode" id="cb1307"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1307-1"><a href="variable-selection-and-model-building.html#cb1307-1" aria-hidden="true" tabindex="-1"></a>hipcenter_mod_back_aic <span class="ot">=</span> <span class="fu">step</span>(hipcenter_mod, <span class="at">direction =</span> <span class="st">&quot;backward&quot;</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1307"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1307-1"><a href="variable-selection-and-model-building.html#cb1307-1" tabindex="-1"></a>hipcenter_mod_back_aic <span class="ot">=</span> <span class="fu">step</span>(hipcenter_mod, <span class="at">direction =</span> <span class="st">&quot;backward&quot;</span>)</span></code></pre></div>
 <pre><code>## Start:  AIC=283.62
 ## hipcenter ~ Age + Weight + HtShoes + Ht + Seated + Arm + Thigh + 
 ##     Leg
@@ -724,23 +724,23 @@ <h3><span class="header-section-number">16.2.1</span> Backward Search<a href="va
 \text{AIC} = n\log\left(\frac{\text{RSS}}{n}\right) + 2p,
 \]</span></p>
 <p>which we quickly verify.</p>
-<div class="sourceCode" id="cb1309"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1309-1"><a href="variable-selection-and-model-building.html#cb1309-1" aria-hidden="true" tabindex="-1"></a><span class="fu">extractAIC</span>(hipcenter_mod) <span class="co"># returns both p and AIC</span></span></code></pre></div>
+<div class="sourceCode" id="cb1309"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1309-1"><a href="variable-selection-and-model-building.html#cb1309-1" tabindex="-1"></a><span class="fu">extractAIC</span>(hipcenter_mod) <span class="co"># returns both p and AIC</span></span></code></pre></div>
 <pre><code>## [1]   9.000 283.624</code></pre>
-<div class="sourceCode" id="cb1311"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1311-1"><a href="variable-selection-and-model-building.html#cb1311-1" aria-hidden="true" tabindex="-1"></a>n <span class="ot">=</span> <span class="fu">length</span>(<span class="fu">resid</span>(hipcenter_mod))</span>
-<span id="cb1311-2"><a href="variable-selection-and-model-building.html#cb1311-2" aria-hidden="true" tabindex="-1"></a>(<span class="at">p =</span> <span class="fu">length</span>(<span class="fu">coef</span>(hipcenter_mod)))</span></code></pre></div>
+<div class="sourceCode" id="cb1311"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1311-1"><a href="variable-selection-and-model-building.html#cb1311-1" tabindex="-1"></a>n <span class="ot">=</span> <span class="fu">length</span>(<span class="fu">resid</span>(hipcenter_mod))</span>
+<span id="cb1311-2"><a href="variable-selection-and-model-building.html#cb1311-2" tabindex="-1"></a>(<span class="at">p =</span> <span class="fu">length</span>(<span class="fu">coef</span>(hipcenter_mod)))</span></code></pre></div>
 <pre><code>## [1] 9</code></pre>
-<div class="sourceCode" id="cb1313"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1313-1"><a href="variable-selection-and-model-building.html#cb1313-1" aria-hidden="true" tabindex="-1"></a>n <span class="sc">*</span> <span class="fu">log</span>(<span class="fu">mean</span>(<span class="fu">resid</span>(hipcenter_mod) <span class="sc">^</span> <span class="dv">2</span>)) <span class="sc">+</span> <span class="dv">2</span> <span class="sc">*</span> p</span></code></pre></div>
+<div class="sourceCode" id="cb1313"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1313-1"><a href="variable-selection-and-model-building.html#cb1313-1" tabindex="-1"></a>n <span class="sc">*</span> <span class="fu">log</span>(<span class="fu">mean</span>(<span class="fu">resid</span>(hipcenter_mod) <span class="sc">^</span> <span class="dv">2</span>)) <span class="sc">+</span> <span class="dv">2</span> <span class="sc">*</span> p</span></code></pre></div>
 <pre><code>## [1] 283.624</code></pre>
 <p>Returning to the first step, <code>R</code> then gives us a row which shows the effect of deleting each of the current predictors. The <code>-</code> signs at the beginning of each row indicate we are considering removing a predictor. There is also a row with <code>&lt;none&gt;</code> which is a row for keeping the current model. Notice that this row has the smallest <span class="math inline">\(\text{RSS}\)</span>, as it is the largest model.</p>
 <p>We see that every row above <code>&lt;none&gt;</code> has a smaller <span class="math inline">\(\text{AIC}\)</span> than the row for <code>&lt;none&gt;</code> with the one at the top, <code>Ht</code>, giving the lowest <span class="math inline">\(\text{AIC}\)</span>. Thus we remove <code>Ht</code> from the model, and continue the process.</p>
 <p>Notice, in the second step, we start with the model <code>hipcenter ~ Age + Weight + HtShoes + Seated + Arm + Thigh + Leg</code> and the variable <code>Ht</code> is no longer considered.</p>
 <p>We continue the process until we reach the model <code>hipcenter ~ Age + HtShoes + Leg</code>. At this step, the row for <code>&lt;none&gt;</code> tops the list, as removing any additional variable will not improve the <span class="math inline">\(\text{AIC}\)</span> This is the model which is stored in <code>hipcenter_mod_back_aic</code>.</p>
-<div class="sourceCode" id="cb1315"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1315-1"><a href="variable-selection-and-model-building.html#cb1315-1" aria-hidden="true" tabindex="-1"></a><span class="fu">coef</span>(hipcenter_mod_back_aic)</span></code></pre></div>
+<div class="sourceCode" id="cb1315"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1315-1"><a href="variable-selection-and-model-building.html#cb1315-1" tabindex="-1"></a><span class="fu">coef</span>(hipcenter_mod_back_aic)</span></code></pre></div>
 <pre><code>## (Intercept)         Age     HtShoes         Leg 
 ## 456.2136538   0.5998327  -2.3022555  -6.8297461</code></pre>
 <p>We could also search through the possible models in a backwards fashion using <span class="math inline">\(\text{BIC}\)</span>. To do so, we again use the <code>step()</code> function, but now specify <code>k = log(n)</code>, where <code>n</code> stores the number of observations in the data.</p>
-<div class="sourceCode" id="cb1317"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1317-1"><a href="variable-selection-and-model-building.html#cb1317-1" aria-hidden="true" tabindex="-1"></a>n <span class="ot">=</span> <span class="fu">length</span>(<span class="fu">resid</span>(hipcenter_mod))</span>
-<span id="cb1317-2"><a href="variable-selection-and-model-building.html#cb1317-2" aria-hidden="true" tabindex="-1"></a>hipcenter_mod_back_bic <span class="ot">=</span> <span class="fu">step</span>(hipcenter_mod, <span class="at">direction =</span> <span class="st">&quot;backward&quot;</span>, <span class="at">k =</span> <span class="fu">log</span>(n))</span></code></pre></div>
+<div class="sourceCode" id="cb1317"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1317-1"><a href="variable-selection-and-model-building.html#cb1317-1" tabindex="-1"></a>n <span class="ot">=</span> <span class="fu">length</span>(<span class="fu">resid</span>(hipcenter_mod))</span>
+<span id="cb1317-2"><a href="variable-selection-and-model-building.html#cb1317-2" tabindex="-1"></a>hipcenter_mod_back_bic <span class="ot">=</span> <span class="fu">step</span>(hipcenter_mod, <span class="at">direction =</span> <span class="st">&quot;backward&quot;</span>, <span class="at">k =</span> <span class="fu">log</span>(n))</span></code></pre></div>
 <pre><code>## Start:  AIC=298.36
 ## hipcenter ~ Age + Weight + HtShoes + Ht + Seated + Arm + Thigh + 
 ##     Leg
@@ -827,34 +827,34 @@ <h3><span class="header-section-number">16.2.1</span> Backward Search<a href="va
 ## - HtShoes  1     83534 131639 313.35</code></pre>
 <p>The procedure is exactly the same, except at each step we look to improve the <span class="math inline">\(\text{BIC}\)</span>, which <code>R</code> still labels <span class="math inline">\(\text{AIC}\)</span> in the output.</p>
 <p>The variable <code>hipcenter_mod_back_bic</code> stores the model chosen by this procedure.</p>
-<div class="sourceCode" id="cb1319"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1319-1"><a href="variable-selection-and-model-building.html#cb1319-1" aria-hidden="true" tabindex="-1"></a><span class="fu">coef</span>(hipcenter_mod_back_bic)</span></code></pre></div>
+<div class="sourceCode" id="cb1319"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1319-1"><a href="variable-selection-and-model-building.html#cb1319-1" tabindex="-1"></a><span class="fu">coef</span>(hipcenter_mod_back_bic)</span></code></pre></div>
 <pre><code>## (Intercept)     HtShoes 
 ##  565.592659   -4.262091</code></pre>
 <p>We note that this model is <em>smaller</em>, has fewer predictors, than the model chosen by <span class="math inline">\(\text{AIC}\)</span>, which is what we would expect. Also note that while both models are different, neither uses both <code>Ht</code> and <code>HtShoes</code> which are extremely correlated.</p>
 <p>We can use information from the <code>summary()</code> function to compare their Adjusted <span class="math inline">\(R^2\)</span> values. Note that either selected model performs better than the original full model.</p>
-<div class="sourceCode" id="cb1321"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1321-1"><a href="variable-selection-and-model-building.html#cb1321-1" aria-hidden="true" tabindex="-1"></a><span class="fu">summary</span>(hipcenter_mod)<span class="sc">$</span>adj.r.squared</span></code></pre></div>
+<div class="sourceCode" id="cb1321"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1321-1"><a href="variable-selection-and-model-building.html#cb1321-1" tabindex="-1"></a><span class="fu">summary</span>(hipcenter_mod)<span class="sc">$</span>adj.r.squared</span></code></pre></div>
 <pre><code>## [1] 0.6000855</code></pre>
-<div class="sourceCode" id="cb1323"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1323-1"><a href="variable-selection-and-model-building.html#cb1323-1" aria-hidden="true" tabindex="-1"></a><span class="fu">summary</span>(hipcenter_mod_back_aic)<span class="sc">$</span>adj.r.squared</span></code></pre></div>
+<div class="sourceCode" id="cb1323"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1323-1"><a href="variable-selection-and-model-building.html#cb1323-1" tabindex="-1"></a><span class="fu">summary</span>(hipcenter_mod_back_aic)<span class="sc">$</span>adj.r.squared</span></code></pre></div>
 <pre><code>## [1] 0.6531427</code></pre>
-<div class="sourceCode" id="cb1325"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1325-1"><a href="variable-selection-and-model-building.html#cb1325-1" aria-hidden="true" tabindex="-1"></a><span class="fu">summary</span>(hipcenter_mod_back_bic)<span class="sc">$</span>adj.r.squared</span></code></pre></div>
+<div class="sourceCode" id="cb1325"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1325-1"><a href="variable-selection-and-model-building.html#cb1325-1" tabindex="-1"></a><span class="fu">summary</span>(hipcenter_mod_back_bic)<span class="sc">$</span>adj.r.squared</span></code></pre></div>
 <pre><code>## [1] 0.6244149</code></pre>
 <p>We can also calculate the LOOCV <span class="math inline">\(\text{RMSE}\)</span> for both selected models, as well as the full model.</p>
-<div class="sourceCode" id="cb1327"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1327-1"><a href="variable-selection-and-model-building.html#cb1327-1" aria-hidden="true" tabindex="-1"></a><span class="fu">calc_loocv_rmse</span>(hipcenter_mod)</span></code></pre></div>
+<div class="sourceCode" id="cb1327"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1327-1"><a href="variable-selection-and-model-building.html#cb1327-1" tabindex="-1"></a><span class="fu">calc_loocv_rmse</span>(hipcenter_mod)</span></code></pre></div>
 <pre><code>## [1] 44.44564</code></pre>
-<div class="sourceCode" id="cb1329"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1329-1"><a href="variable-selection-and-model-building.html#cb1329-1" aria-hidden="true" tabindex="-1"></a><span class="fu">calc_loocv_rmse</span>(hipcenter_mod_back_aic)</span></code></pre></div>
+<div class="sourceCode" id="cb1329"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1329-1"><a href="variable-selection-and-model-building.html#cb1329-1" tabindex="-1"></a><span class="fu">calc_loocv_rmse</span>(hipcenter_mod_back_aic)</span></code></pre></div>
 <pre><code>## [1] 37.58473</code></pre>
-<div class="sourceCode" id="cb1331"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1331-1"><a href="variable-selection-and-model-building.html#cb1331-1" aria-hidden="true" tabindex="-1"></a><span class="fu">calc_loocv_rmse</span>(hipcenter_mod_back_bic)</span></code></pre></div>
+<div class="sourceCode" id="cb1331"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1331-1"><a href="variable-selection-and-model-building.html#cb1331-1" tabindex="-1"></a><span class="fu">calc_loocv_rmse</span>(hipcenter_mod_back_bic)</span></code></pre></div>
 <pre><code>## [1] 37.40564</code></pre>
 <p>We see that we would prefer the model chosen via <span class="math inline">\(\text{BIC}\)</span> if using LOOCV <span class="math inline">\(\text{RMSE}\)</span> as our metric.</p>
 </div>
 <div id="forward-search" class="section level3 hasAnchor" number="16.2.2">
 <h3><span class="header-section-number">16.2.2</span> Forward Search<a href="variable-selection-and-model-building.html#forward-search" class="anchor-section" aria-label="Anchor link to header"></a></h3>
 <p>Forward selection is the exact opposite of backwards selection. Here we tell <code>R</code> to start with a model using no predictors, that is <code>hipcenter ~ 1</code>, then at each step <code>R</code> will attempt to add a predictor until it finds a good model or reaches <code>hipcenter ~ Age + Weight + HtShoes + Ht + Seated + Arm + Thigh + Leg</code>.</p>
-<div class="sourceCode" id="cb1333"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1333-1"><a href="variable-selection-and-model-building.html#cb1333-1" aria-hidden="true" tabindex="-1"></a>hipcenter_mod_start <span class="ot">=</span> <span class="fu">lm</span>(hipcenter <span class="sc">~</span> <span class="dv">1</span>, <span class="at">data =</span> seatpos)</span>
-<span id="cb1333-2"><a href="variable-selection-and-model-building.html#cb1333-2" aria-hidden="true" tabindex="-1"></a>hipcenter_mod_forw_aic <span class="ot">=</span> <span class="fu">step</span>(</span>
-<span id="cb1333-3"><a href="variable-selection-and-model-building.html#cb1333-3" aria-hidden="true" tabindex="-1"></a>  hipcenter_mod_start, </span>
-<span id="cb1333-4"><a href="variable-selection-and-model-building.html#cb1333-4" aria-hidden="true" tabindex="-1"></a>  <span class="at">scope =</span> hipcenter <span class="sc">~</span> Age <span class="sc">+</span> Weight <span class="sc">+</span> HtShoes <span class="sc">+</span> Ht <span class="sc">+</span> Seated <span class="sc">+</span> Arm <span class="sc">+</span> Thigh <span class="sc">+</span> Leg, </span>
-<span id="cb1333-5"><a href="variable-selection-and-model-building.html#cb1333-5" aria-hidden="true" tabindex="-1"></a>  <span class="at">direction =</span> <span class="st">&quot;forward&quot;</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1333"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1333-1"><a href="variable-selection-and-model-building.html#cb1333-1" tabindex="-1"></a>hipcenter_mod_start <span class="ot">=</span> <span class="fu">lm</span>(hipcenter <span class="sc">~</span> <span class="dv">1</span>, <span class="at">data =</span> seatpos)</span>
+<span id="cb1333-2"><a href="variable-selection-and-model-building.html#cb1333-2" tabindex="-1"></a>hipcenter_mod_forw_aic <span class="ot">=</span> <span class="fu">step</span>(</span>
+<span id="cb1333-3"><a href="variable-selection-and-model-building.html#cb1333-3" tabindex="-1"></a>  hipcenter_mod_start, </span>
+<span id="cb1333-4"><a href="variable-selection-and-model-building.html#cb1333-4" tabindex="-1"></a>  <span class="at">scope =</span> hipcenter <span class="sc">~</span> Age <span class="sc">+</span> Weight <span class="sc">+</span> HtShoes <span class="sc">+</span> Ht <span class="sc">+</span> Seated <span class="sc">+</span> Arm <span class="sc">+</span> Thigh <span class="sc">+</span> Leg, </span>
+<span id="cb1333-5"><a href="variable-selection-and-model-building.html#cb1333-5" tabindex="-1"></a>  <span class="at">direction =</span> <span class="st">&quot;forward&quot;</span>)</span></code></pre></div>
 <pre><code>## Start:  AIC=311.71
 ## hipcenter ~ 1
 ## 
@@ -905,10 +905,10 @@ <h3><span class="header-section-number">16.2.2</span> Forward Search<a href="var
 ## + Weight   1     46.83 41891 276.20
 ## + HtShoes  1     13.38 41925 276.23</code></pre>
 <p>Again, by default <code>R</code> uses <span class="math inline">\(\text{AIC}\)</span> as its quality metric when using the <code>step()</code> function. Also note that now the rows begin with a <code>+</code> which indicates addition of predictors to the current model from any step.</p>
-<div class="sourceCode" id="cb1335"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1335-1"><a href="variable-selection-and-model-building.html#cb1335-1" aria-hidden="true" tabindex="-1"></a>hipcenter_mod_forw_bic <span class="ot">=</span> <span class="fu">step</span>(</span>
-<span id="cb1335-2"><a href="variable-selection-and-model-building.html#cb1335-2" aria-hidden="true" tabindex="-1"></a>  hipcenter_mod_start, </span>
-<span id="cb1335-3"><a href="variable-selection-and-model-building.html#cb1335-3" aria-hidden="true" tabindex="-1"></a>  <span class="at">scope =</span> hipcenter <span class="sc">~</span> Age <span class="sc">+</span> Weight <span class="sc">+</span> HtShoes <span class="sc">+</span> Ht <span class="sc">+</span> Seated <span class="sc">+</span> Arm <span class="sc">+</span> Thigh <span class="sc">+</span> Leg, </span>
-<span id="cb1335-4"><a href="variable-selection-and-model-building.html#cb1335-4" aria-hidden="true" tabindex="-1"></a>  <span class="at">direction =</span> <span class="st">&quot;forward&quot;</span>, <span class="at">k =</span> <span class="fu">log</span>(n))</span></code></pre></div>
+<div class="sourceCode" id="cb1335"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1335-1"><a href="variable-selection-and-model-building.html#cb1335-1" tabindex="-1"></a>hipcenter_mod_forw_bic <span class="ot">=</span> <span class="fu">step</span>(</span>
+<span id="cb1335-2"><a href="variable-selection-and-model-building.html#cb1335-2" tabindex="-1"></a>  hipcenter_mod_start, </span>
+<span id="cb1335-3"><a href="variable-selection-and-model-building.html#cb1335-3" tabindex="-1"></a>  <span class="at">scope =</span> hipcenter <span class="sc">~</span> Age <span class="sc">+</span> Weight <span class="sc">+</span> HtShoes <span class="sc">+</span> Ht <span class="sc">+</span> Seated <span class="sc">+</span> Arm <span class="sc">+</span> Thigh <span class="sc">+</span> Leg, </span>
+<span id="cb1335-4"><a href="variable-selection-and-model-building.html#cb1335-4" tabindex="-1"></a>  <span class="at">direction =</span> <span class="st">&quot;forward&quot;</span>, <span class="at">k =</span> <span class="fu">log</span>(n))</span></code></pre></div>
 <pre><code>## Start:  AIC=313.35
 ## hipcenter ~ 1
 ## 
@@ -936,28 +936,28 @@ <h3><span class="header-section-number">16.2.2</span> Forward Search<a href="var
 ## + HtShoes  1     25.76 47590 281.96
 ## + Thigh    1      4.63 47611 281.98</code></pre>
 <p>We can make the same modification as last time to instead use <span class="math inline">\(\text{BIC}\)</span> with forward selection.</p>
-<div class="sourceCode" id="cb1337"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1337-1"><a href="variable-selection-and-model-building.html#cb1337-1" aria-hidden="true" tabindex="-1"></a><span class="fu">summary</span>(hipcenter_mod)<span class="sc">$</span>adj.r.squared</span></code></pre></div>
+<div class="sourceCode" id="cb1337"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1337-1"><a href="variable-selection-and-model-building.html#cb1337-1" tabindex="-1"></a><span class="fu">summary</span>(hipcenter_mod)<span class="sc">$</span>adj.r.squared</span></code></pre></div>
 <pre><code>## [1] 0.6000855</code></pre>
-<div class="sourceCode" id="cb1339"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1339-1"><a href="variable-selection-and-model-building.html#cb1339-1" aria-hidden="true" tabindex="-1"></a><span class="fu">summary</span>(hipcenter_mod_forw_aic)<span class="sc">$</span>adj.r.squared</span></code></pre></div>
+<div class="sourceCode" id="cb1339"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1339-1"><a href="variable-selection-and-model-building.html#cb1339-1" tabindex="-1"></a><span class="fu">summary</span>(hipcenter_mod_forw_aic)<span class="sc">$</span>adj.r.squared</span></code></pre></div>
 <pre><code>## [1] 0.6533055</code></pre>
-<div class="sourceCode" id="cb1341"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1341-1"><a href="variable-selection-and-model-building.html#cb1341-1" aria-hidden="true" tabindex="-1"></a><span class="fu">summary</span>(hipcenter_mod_forw_bic)<span class="sc">$</span>adj.r.squared</span></code></pre></div>
+<div class="sourceCode" id="cb1341"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1341-1"><a href="variable-selection-and-model-building.html#cb1341-1" tabindex="-1"></a><span class="fu">summary</span>(hipcenter_mod_forw_bic)<span class="sc">$</span>adj.r.squared</span></code></pre></div>
 <pre><code>## [1] 0.6282374</code></pre>
 <p>We can compare the two selected models’ Adjusted <span class="math inline">\(R^2\)</span> as well as their LOOCV <span class="math inline">\(\text{RMSE}\)</span> The results are very similar to those using backwards selection, although the models are not exactly the same.</p>
-<div class="sourceCode" id="cb1343"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1343-1"><a href="variable-selection-and-model-building.html#cb1343-1" aria-hidden="true" tabindex="-1"></a><span class="fu">calc_loocv_rmse</span>(hipcenter_mod)</span></code></pre></div>
+<div class="sourceCode" id="cb1343"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1343-1"><a href="variable-selection-and-model-building.html#cb1343-1" tabindex="-1"></a><span class="fu">calc_loocv_rmse</span>(hipcenter_mod)</span></code></pre></div>
 <pre><code>## [1] 44.44564</code></pre>
-<div class="sourceCode" id="cb1345"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1345-1"><a href="variable-selection-and-model-building.html#cb1345-1" aria-hidden="true" tabindex="-1"></a><span class="fu">calc_loocv_rmse</span>(hipcenter_mod_forw_aic)</span></code></pre></div>
+<div class="sourceCode" id="cb1345"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1345-1"><a href="variable-selection-and-model-building.html#cb1345-1" tabindex="-1"></a><span class="fu">calc_loocv_rmse</span>(hipcenter_mod_forw_aic)</span></code></pre></div>
 <pre><code>## [1] 37.62516</code></pre>
-<div class="sourceCode" id="cb1347"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1347-1"><a href="variable-selection-and-model-building.html#cb1347-1" aria-hidden="true" tabindex="-1"></a><span class="fu">calc_loocv_rmse</span>(hipcenter_mod_forw_bic)</span></code></pre></div>
+<div class="sourceCode" id="cb1347"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1347-1"><a href="variable-selection-and-model-building.html#cb1347-1" tabindex="-1"></a><span class="fu">calc_loocv_rmse</span>(hipcenter_mod_forw_bic)</span></code></pre></div>
 <pre><code>## [1] 37.2511</code></pre>
 </div>
 <div id="stepwise-search" class="section level3 hasAnchor" number="16.2.3">
 <h3><span class="header-section-number">16.2.3</span> Stepwise Search<a href="variable-selection-and-model-building.html#stepwise-search" class="anchor-section" aria-label="Anchor link to header"></a></h3>
 <p>Stepwise search checks going both backwards and forwards at every step. It considers the addition of any variable not currently in the model, as well as the removal of any variable currently in the model.</p>
 <p>Here we perform stepwise search using <span class="math inline">\(\text{AIC}\)</span> as our metric. We start with the model <code>hipcenter ~ 1</code> and search up to <code>hipcenter ~ Age + Weight + HtShoes + Ht + Seated + Arm + Thigh + Leg</code>. Notice that at many of the steps, some row begin with <code>-</code>, while others begin with <code>+</code>.</p>
-<div class="sourceCode" id="cb1349"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1349-1"><a href="variable-selection-and-model-building.html#cb1349-1" aria-hidden="true" tabindex="-1"></a>hipcenter_mod_both_aic <span class="ot">=</span> <span class="fu">step</span>(</span>
-<span id="cb1349-2"><a href="variable-selection-and-model-building.html#cb1349-2" aria-hidden="true" tabindex="-1"></a>  hipcenter_mod_start, </span>
-<span id="cb1349-3"><a href="variable-selection-and-model-building.html#cb1349-3" aria-hidden="true" tabindex="-1"></a>  <span class="at">scope =</span> hipcenter <span class="sc">~</span> Age <span class="sc">+</span> Weight <span class="sc">+</span> HtShoes <span class="sc">+</span> Ht <span class="sc">+</span> Seated <span class="sc">+</span> Arm <span class="sc">+</span> Thigh <span class="sc">+</span> Leg, </span>
-<span id="cb1349-4"><a href="variable-selection-and-model-building.html#cb1349-4" aria-hidden="true" tabindex="-1"></a>  <span class="at">direction =</span> <span class="st">&quot;both&quot;</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1349"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1349-1"><a href="variable-selection-and-model-building.html#cb1349-1" tabindex="-1"></a>hipcenter_mod_both_aic <span class="ot">=</span> <span class="fu">step</span>(</span>
+<span id="cb1349-2"><a href="variable-selection-and-model-building.html#cb1349-2" tabindex="-1"></a>  hipcenter_mod_start, </span>
+<span id="cb1349-3"><a href="variable-selection-and-model-building.html#cb1349-3" tabindex="-1"></a>  <span class="at">scope =</span> hipcenter <span class="sc">~</span> Age <span class="sc">+</span> Weight <span class="sc">+</span> HtShoes <span class="sc">+</span> Ht <span class="sc">+</span> Seated <span class="sc">+</span> Arm <span class="sc">+</span> Thigh <span class="sc">+</span> Leg, </span>
+<span id="cb1349-4"><a href="variable-selection-and-model-building.html#cb1349-4" tabindex="-1"></a>  <span class="at">direction =</span> <span class="st">&quot;both&quot;</span>)</span></code></pre></div>
 <pre><code>## Start:  AIC=311.71
 ## hipcenter ~ 1
 ## 
@@ -1014,10 +1014,10 @@ <h3><span class="header-section-number">16.2.3</span> Stepwise Search<a href="va
 ## + Weight   1      46.8 41891 276.20
 ## + HtShoes  1      13.4 41925 276.23</code></pre>
 <p>We could again instead use <span class="math inline">\(\text{BIC}\)</span> as our metric.</p>
-<div class="sourceCode" id="cb1351"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1351-1"><a href="variable-selection-and-model-building.html#cb1351-1" aria-hidden="true" tabindex="-1"></a>hipcenter_mod_both_bic <span class="ot">=</span> <span class="fu">step</span>(</span>
-<span id="cb1351-2"><a href="variable-selection-and-model-building.html#cb1351-2" aria-hidden="true" tabindex="-1"></a>  hipcenter_mod_start, </span>
-<span id="cb1351-3"><a href="variable-selection-and-model-building.html#cb1351-3" aria-hidden="true" tabindex="-1"></a>  <span class="at">scope =</span> hipcenter <span class="sc">~</span> Age <span class="sc">+</span> Weight <span class="sc">+</span> HtShoes <span class="sc">+</span> Ht <span class="sc">+</span> Seated <span class="sc">+</span> Arm <span class="sc">+</span> Thigh <span class="sc">+</span> Leg, </span>
-<span id="cb1351-4"><a href="variable-selection-and-model-building.html#cb1351-4" aria-hidden="true" tabindex="-1"></a>  <span class="at">direction =</span> <span class="st">&quot;both&quot;</span>, <span class="at">k =</span> <span class="fu">log</span>(n))</span></code></pre></div>
+<div class="sourceCode" id="cb1351"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1351-1"><a href="variable-selection-and-model-building.html#cb1351-1" tabindex="-1"></a>hipcenter_mod_both_bic <span class="ot">=</span> <span class="fu">step</span>(</span>
+<span id="cb1351-2"><a href="variable-selection-and-model-building.html#cb1351-2" tabindex="-1"></a>  hipcenter_mod_start, </span>
+<span id="cb1351-3"><a href="variable-selection-and-model-building.html#cb1351-3" tabindex="-1"></a>  <span class="at">scope =</span> hipcenter <span class="sc">~</span> Age <span class="sc">+</span> Weight <span class="sc">+</span> HtShoes <span class="sc">+</span> Ht <span class="sc">+</span> Seated <span class="sc">+</span> Arm <span class="sc">+</span> Thigh <span class="sc">+</span> Leg, </span>
+<span id="cb1351-4"><a href="variable-selection-and-model-building.html#cb1351-4" tabindex="-1"></a>  <span class="at">direction =</span> <span class="st">&quot;both&quot;</span>, <span class="at">k =</span> <span class="fu">log</span>(n))</span></code></pre></div>
 <pre><code>## Start:  AIC=313.35
 ## hipcenter ~ 1
 ## 
@@ -1046,28 +1046,28 @@ <h3><span class="header-section-number">16.2.3</span> Stepwise Search<a href="va
 ## + Thigh    1         5  47611 281.98
 ## - Ht       1     84023 131639 313.35</code></pre>
 <p>Adjusted <span class="math inline">\(R^2\)</span> and LOOCV <span class="math inline">\(\text{RMSE}\)</span> comparisons are similar to backwards and forwards, which is not at all surprising, as some of the models selected are the same as before.</p>
-<div class="sourceCode" id="cb1353"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1353-1"><a href="variable-selection-and-model-building.html#cb1353-1" aria-hidden="true" tabindex="-1"></a><span class="fu">summary</span>(hipcenter_mod)<span class="sc">$</span>adj.r.squared</span></code></pre></div>
+<div class="sourceCode" id="cb1353"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1353-1"><a href="variable-selection-and-model-building.html#cb1353-1" tabindex="-1"></a><span class="fu">summary</span>(hipcenter_mod)<span class="sc">$</span>adj.r.squared</span></code></pre></div>
 <pre><code>## [1] 0.6000855</code></pre>
-<div class="sourceCode" id="cb1355"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1355-1"><a href="variable-selection-and-model-building.html#cb1355-1" aria-hidden="true" tabindex="-1"></a><span class="fu">summary</span>(hipcenter_mod_both_aic)<span class="sc">$</span>adj.r.squared</span></code></pre></div>
+<div class="sourceCode" id="cb1355"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1355-1"><a href="variable-selection-and-model-building.html#cb1355-1" tabindex="-1"></a><span class="fu">summary</span>(hipcenter_mod_both_aic)<span class="sc">$</span>adj.r.squared</span></code></pre></div>
 <pre><code>## [1] 0.6533055</code></pre>
-<div class="sourceCode" id="cb1357"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1357-1"><a href="variable-selection-and-model-building.html#cb1357-1" aria-hidden="true" tabindex="-1"></a><span class="fu">summary</span>(hipcenter_mod_both_bic)<span class="sc">$</span>adj.r.squared</span></code></pre></div>
+<div class="sourceCode" id="cb1357"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1357-1"><a href="variable-selection-and-model-building.html#cb1357-1" tabindex="-1"></a><span class="fu">summary</span>(hipcenter_mod_both_bic)<span class="sc">$</span>adj.r.squared</span></code></pre></div>
 <pre><code>## [1] 0.6282374</code></pre>
-<div class="sourceCode" id="cb1359"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1359-1"><a href="variable-selection-and-model-building.html#cb1359-1" aria-hidden="true" tabindex="-1"></a><span class="fu">calc_loocv_rmse</span>(hipcenter_mod)</span></code></pre></div>
+<div class="sourceCode" id="cb1359"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1359-1"><a href="variable-selection-and-model-building.html#cb1359-1" tabindex="-1"></a><span class="fu">calc_loocv_rmse</span>(hipcenter_mod)</span></code></pre></div>
 <pre><code>## [1] 44.44564</code></pre>
-<div class="sourceCode" id="cb1361"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1361-1"><a href="variable-selection-and-model-building.html#cb1361-1" aria-hidden="true" tabindex="-1"></a><span class="fu">calc_loocv_rmse</span>(hipcenter_mod_both_aic)</span></code></pre></div>
+<div class="sourceCode" id="cb1361"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1361-1"><a href="variable-selection-and-model-building.html#cb1361-1" tabindex="-1"></a><span class="fu">calc_loocv_rmse</span>(hipcenter_mod_both_aic)</span></code></pre></div>
 <pre><code>## [1] 37.62516</code></pre>
-<div class="sourceCode" id="cb1363"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1363-1"><a href="variable-selection-and-model-building.html#cb1363-1" aria-hidden="true" tabindex="-1"></a><span class="fu">calc_loocv_rmse</span>(hipcenter_mod_both_bic)</span></code></pre></div>
+<div class="sourceCode" id="cb1363"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1363-1"><a href="variable-selection-and-model-building.html#cb1363-1" tabindex="-1"></a><span class="fu">calc_loocv_rmse</span>(hipcenter_mod_both_bic)</span></code></pre></div>
 <pre><code>## [1] 37.2511</code></pre>
 </div>
 <div id="exhaustive-search" class="section level3 hasAnchor" number="16.2.4">
 <h3><span class="header-section-number">16.2.4</span> Exhaustive Search<a href="variable-selection-and-model-building.html#exhaustive-search" class="anchor-section" aria-label="Anchor link to header"></a></h3>
 <p>Backward, forward, and stepwise search are all useful, but do have an obvious issue. By not checking every possible model, sometimes they will miss the best possible model. With an extremely large number of predictors, sometimes this is necessary since checking every possible model would be rather time consuming, even with current computers.</p>
 <p>However, with a reasonably sized dataset, it isn’t too difficult to check all possible models. To do so, we will use the <code>regsubsets()</code> function in the <code>R</code> package <code>leaps</code>.</p>
-<div class="sourceCode" id="cb1365"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1365-1"><a href="variable-selection-and-model-building.html#cb1365-1" aria-hidden="true" tabindex="-1"></a><span class="fu">library</span>(leaps)</span>
-<span id="cb1365-2"><a href="variable-selection-and-model-building.html#cb1365-2" aria-hidden="true" tabindex="-1"></a>all_hipcenter_mod <span class="ot">=</span> <span class="fu">summary</span>(<span class="fu">regsubsets</span>(hipcenter <span class="sc">~</span> ., <span class="at">data =</span> seatpos))</span></code></pre></div>
+<div class="sourceCode" id="cb1365"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1365-1"><a href="variable-selection-and-model-building.html#cb1365-1" tabindex="-1"></a><span class="fu">library</span>(leaps)</span>
+<span id="cb1365-2"><a href="variable-selection-and-model-building.html#cb1365-2" tabindex="-1"></a>all_hipcenter_mod <span class="ot">=</span> <span class="fu">summary</span>(<span class="fu">regsubsets</span>(hipcenter <span class="sc">~</span> ., <span class="at">data =</span> seatpos))</span></code></pre></div>
 <p>A few points about this line of code. First, note that we immediately use <code>summary()</code> and store those results. That is simply the intended use of <code>regsubsets()</code>. Second, inside of <code>regsubsets()</code> we specify the model <code>hipcenter ~ .</code>. This will be the largest model considered, that is the model using all first-order predictors, and <code>R</code> will check all possible subsets. Third, <code>regsubsets()</code> has an argument of <code>nvmax</code> to set the maximum size of subsets to examine with <span class="math inline">\(8\)</span> as default. You need to change it if you investigate the subsets with more than <span class="math inline">\(8\)</span> predictors, although increasing <code>nvmax</code> will significantly increase computation time.</p>
 <p>We’ll now look at the information stored in <code>all_hipcenter_mod</code>.</p>
-<div class="sourceCode" id="cb1366"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1366-1"><a href="variable-selection-and-model-building.html#cb1366-1" aria-hidden="true" tabindex="-1"></a>all_hipcenter_mod<span class="sc">$</span>which</span></code></pre></div>
+<div class="sourceCode" id="cb1366"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1366-1"><a href="variable-selection-and-model-building.html#cb1366-1" tabindex="-1"></a>all_hipcenter_mod<span class="sc">$</span>which</span></code></pre></div>
 <pre><code>##   (Intercept)   Age Weight HtShoes    Ht Seated   Arm Thigh   Leg
 ## 1        TRUE FALSE  FALSE   FALSE  TRUE  FALSE FALSE FALSE FALSE
 ## 2        TRUE FALSE  FALSE   FALSE  TRUE  FALSE FALSE FALSE  TRUE
@@ -1078,100 +1078,100 @@ <h3><span class="header-section-number">16.2.4</span> Exhaustive Search<a href="
 ## 7        TRUE  TRUE   TRUE    TRUE FALSE   TRUE  TRUE  TRUE  TRUE
 ## 8        TRUE  TRUE   TRUE    TRUE  TRUE   TRUE  TRUE  TRUE  TRUE</code></pre>
 <p>Using <code>$which</code> gives us the best model, according to <span class="math inline">\(\text{RSS}\)</span>, for a model of each possible size, in this case ranging from one to eight predictors. For example the best model with four predictors (<span class="math inline">\(p = 5\)</span>) would use <code>Age</code>, <code>HtShoes</code>, <code>Thigh</code>, and <code>Leg</code>.</p>
-<div class="sourceCode" id="cb1368"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1368-1"><a href="variable-selection-and-model-building.html#cb1368-1" aria-hidden="true" tabindex="-1"></a>all_hipcenter_mod<span class="sc">$</span>rss</span></code></pre></div>
+<div class="sourceCode" id="cb1368"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1368-1"><a href="variable-selection-and-model-building.html#cb1368-1" tabindex="-1"></a>all_hipcenter_mod<span class="sc">$</span>rss</span></code></pre></div>
 <pre><code>## [1] 47615.79 44834.69 41938.09 41485.01 41313.00 41277.90 41266.80 41261.78</code></pre>
 <p>We can obtain the <span class="math inline">\(\text{RSS}\)</span> for each of these models using <code>$rss</code>. Notice that these are decreasing since the models range from small to large.</p>
 <p>Now that we have the <span class="math inline">\(\text{RSS}\)</span> for each of these models, it is rather easy to obtain <span class="math inline">\(\text{AIC}\)</span>, <span class="math inline">\(\text{BIC}\)</span>, and Adjusted <span class="math inline">\(R^2\)</span> since they are all a function of <span class="math inline">\(\text{RSS}\)</span> Also, since we have the models with the best <span class="math inline">\(\text{RSS}\)</span> for each size, they will result in the models with the best <span class="math inline">\(\text{AIC}\)</span>, <span class="math inline">\(\text{BIC}\)</span>, and Adjusted <span class="math inline">\(R^2\)</span> for each size. Then by picking from those, we can find the overall best <span class="math inline">\(\text{AIC}\)</span>, <span class="math inline">\(\text{BIC}\)</span>, and Adjusted <span class="math inline">\(R^2\)</span>.</p>
 <p>Conveniently, Adjusted <span class="math inline">\(R^2\)</span> is automatically calculated.</p>
-<div class="sourceCode" id="cb1370"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1370-1"><a href="variable-selection-and-model-building.html#cb1370-1" aria-hidden="true" tabindex="-1"></a>all_hipcenter_mod<span class="sc">$</span>adjr2</span></code></pre></div>
+<div class="sourceCode" id="cb1370"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1370-1"><a href="variable-selection-and-model-building.html#cb1370-1" tabindex="-1"></a>all_hipcenter_mod<span class="sc">$</span>adjr2</span></code></pre></div>
 <pre><code>## [1] 0.6282374 0.6399496 0.6533055 0.6466586 0.6371276 0.6257403 0.6133690
 ## [8] 0.6000855</code></pre>
 <p>To find which model has the highest Adjusted <span class="math inline">\(R^2\)</span> we can use the <code>which.max()</code> function.</p>
-<div class="sourceCode" id="cb1372"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1372-1"><a href="variable-selection-and-model-building.html#cb1372-1" aria-hidden="true" tabindex="-1"></a>(<span class="at">best_r2_ind =</span> <span class="fu">which.max</span>(all_hipcenter_mod<span class="sc">$</span>adjr2))</span></code></pre></div>
+<div class="sourceCode" id="cb1372"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1372-1"><a href="variable-selection-and-model-building.html#cb1372-1" tabindex="-1"></a>(<span class="at">best_r2_ind =</span> <span class="fu">which.max</span>(all_hipcenter_mod<span class="sc">$</span>adjr2))</span></code></pre></div>
 <pre><code>## [1] 3</code></pre>
 <p>We can then extract the predictors of that model.</p>
-<div class="sourceCode" id="cb1374"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1374-1"><a href="variable-selection-and-model-building.html#cb1374-1" aria-hidden="true" tabindex="-1"></a>all_hipcenter_mod<span class="sc">$</span>which[best_r2_ind, ]</span></code></pre></div>
+<div class="sourceCode" id="cb1374"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1374-1"><a href="variable-selection-and-model-building.html#cb1374-1" tabindex="-1"></a>all_hipcenter_mod<span class="sc">$</span>which[best_r2_ind, ]</span></code></pre></div>
 <pre><code>## (Intercept)         Age      Weight     HtShoes          Ht      Seated 
 ##        TRUE        TRUE       FALSE       FALSE        TRUE       FALSE 
 ##         Arm       Thigh         Leg 
 ##       FALSE       FALSE        TRUE</code></pre>
 <p>We’ll now calculate <span class="math inline">\(\text{AIC}\)</span> and <span class="math inline">\(\text{BIC}\)</span> for each of the models with the best <span class="math inline">\(\text{RSS}\)</span>. To do so, we will need both <span class="math inline">\(n\)</span> and the <span class="math inline">\(p\)</span> for the largest possible model.</p>
-<div class="sourceCode" id="cb1376"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1376-1"><a href="variable-selection-and-model-building.html#cb1376-1" aria-hidden="true" tabindex="-1"></a>p <span class="ot">=</span> <span class="fu">length</span>(<span class="fu">coef</span>(hipcenter_mod))</span>
-<span id="cb1376-2"><a href="variable-selection-and-model-building.html#cb1376-2" aria-hidden="true" tabindex="-1"></a>n <span class="ot">=</span> <span class="fu">length</span>(<span class="fu">resid</span>(hipcenter_mod))</span></code></pre></div>
+<div class="sourceCode" id="cb1376"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1376-1"><a href="variable-selection-and-model-building.html#cb1376-1" tabindex="-1"></a>p <span class="ot">=</span> <span class="fu">length</span>(<span class="fu">coef</span>(hipcenter_mod))</span>
+<span id="cb1376-2"><a href="variable-selection-and-model-building.html#cb1376-2" tabindex="-1"></a>n <span class="ot">=</span> <span class="fu">length</span>(<span class="fu">resid</span>(hipcenter_mod))</span></code></pre></div>
 <p>We’ll use the form of <span class="math inline">\(\text{AIC}\)</span> which leaves out the constant term that is equal across all models.</p>
 <p><span class="math display">\[
 \text{AIC} = n\log\left(\frac{\text{RSS}}{n}\right) + 2p.
 \]</span></p>
 <p>Since we have the <span class="math inline">\(\text{RSS}\)</span> of each model stored, this is easy to calculate.</p>
-<div class="sourceCode" id="cb1377"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1377-1"><a href="variable-selection-and-model-building.html#cb1377-1" aria-hidden="true" tabindex="-1"></a>hipcenter_mod_aic <span class="ot">=</span> n <span class="sc">*</span> <span class="fu">log</span>(all_hipcenter_mod<span class="sc">$</span>rss <span class="sc">/</span> n) <span class="sc">+</span> <span class="dv">2</span> <span class="sc">*</span> (<span class="dv">2</span><span class="sc">:</span>p)</span></code></pre></div>
+<div class="sourceCode" id="cb1377"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1377-1"><a href="variable-selection-and-model-building.html#cb1377-1" tabindex="-1"></a>hipcenter_mod_aic <span class="ot">=</span> n <span class="sc">*</span> <span class="fu">log</span>(all_hipcenter_mod<span class="sc">$</span>rss <span class="sc">/</span> n) <span class="sc">+</span> <span class="dv">2</span> <span class="sc">*</span> (<span class="dv">2</span><span class="sc">:</span>p)</span></code></pre></div>
 <p>We can then extract the predictors of the model with the best <span class="math inline">\(\text{AIC}\)</span>.</p>
-<div class="sourceCode" id="cb1378"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1378-1"><a href="variable-selection-and-model-building.html#cb1378-1" aria-hidden="true" tabindex="-1"></a>best_aic_ind <span class="ot">=</span> <span class="fu">which.min</span>(hipcenter_mod_aic)</span>
-<span id="cb1378-2"><a href="variable-selection-and-model-building.html#cb1378-2" aria-hidden="true" tabindex="-1"></a>all_hipcenter_mod<span class="sc">$</span>which[best_aic_ind,]</span></code></pre></div>
+<div class="sourceCode" id="cb1378"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1378-1"><a href="variable-selection-and-model-building.html#cb1378-1" tabindex="-1"></a>best_aic_ind <span class="ot">=</span> <span class="fu">which.min</span>(hipcenter_mod_aic)</span>
+<span id="cb1378-2"><a href="variable-selection-and-model-building.html#cb1378-2" tabindex="-1"></a>all_hipcenter_mod<span class="sc">$</span>which[best_aic_ind,]</span></code></pre></div>
 <pre><code>## (Intercept)         Age      Weight     HtShoes          Ht      Seated 
 ##        TRUE        TRUE       FALSE       FALSE        TRUE       FALSE 
 ##         Arm       Thigh         Leg 
 ##       FALSE       FALSE        TRUE</code></pre>
 <p>Let’s fit this model so we can compare to our previously chosen models using <span class="math inline">\(\text{AIC}\)</span> and search procedures.</p>
-<div class="sourceCode" id="cb1380"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1380-1"><a href="variable-selection-and-model-building.html#cb1380-1" aria-hidden="true" tabindex="-1"></a>hipcenter_mod_best_aic <span class="ot">=</span> <span class="fu">lm</span>(hipcenter <span class="sc">~</span> Age <span class="sc">+</span> Ht <span class="sc">+</span> Leg, <span class="at">data =</span> seatpos)</span></code></pre></div>
+<div class="sourceCode" id="cb1380"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1380-1"><a href="variable-selection-and-model-building.html#cb1380-1" tabindex="-1"></a>hipcenter_mod_best_aic <span class="ot">=</span> <span class="fu">lm</span>(hipcenter <span class="sc">~</span> Age <span class="sc">+</span> Ht <span class="sc">+</span> Leg, <span class="at">data =</span> seatpos)</span></code></pre></div>
 <p>The <code>extractAIC()</code> function will calculate the <span class="math inline">\(\text{AIC}\)</span> defined above for a fitted model.</p>
-<div class="sourceCode" id="cb1381"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1381-1"><a href="variable-selection-and-model-building.html#cb1381-1" aria-hidden="true" tabindex="-1"></a><span class="fu">extractAIC</span>(hipcenter_mod_best_aic)</span></code></pre></div>
+<div class="sourceCode" id="cb1381"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1381-1"><a href="variable-selection-and-model-building.html#cb1381-1" tabindex="-1"></a><span class="fu">extractAIC</span>(hipcenter_mod_best_aic)</span></code></pre></div>
 <pre><code>## [1]   4.0000 274.2418</code></pre>
-<div class="sourceCode" id="cb1383"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1383-1"><a href="variable-selection-and-model-building.html#cb1383-1" aria-hidden="true" tabindex="-1"></a><span class="fu">extractAIC</span>(hipcenter_mod_back_aic)</span></code></pre></div>
+<div class="sourceCode" id="cb1383"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1383-1"><a href="variable-selection-and-model-building.html#cb1383-1" tabindex="-1"></a><span class="fu">extractAIC</span>(hipcenter_mod_back_aic)</span></code></pre></div>
 <pre><code>## [1]   4.0000 274.2597</code></pre>
-<div class="sourceCode" id="cb1385"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1385-1"><a href="variable-selection-and-model-building.html#cb1385-1" aria-hidden="true" tabindex="-1"></a><span class="fu">extractAIC</span>(hipcenter_mod_forw_aic)</span></code></pre></div>
+<div class="sourceCode" id="cb1385"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1385-1"><a href="variable-selection-and-model-building.html#cb1385-1" tabindex="-1"></a><span class="fu">extractAIC</span>(hipcenter_mod_forw_aic)</span></code></pre></div>
 <pre><code>## [1]   4.0000 274.2418</code></pre>
-<div class="sourceCode" id="cb1387"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1387-1"><a href="variable-selection-and-model-building.html#cb1387-1" aria-hidden="true" tabindex="-1"></a><span class="fu">extractAIC</span>(hipcenter_mod_both_aic)</span></code></pre></div>
+<div class="sourceCode" id="cb1387"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1387-1"><a href="variable-selection-and-model-building.html#cb1387-1" tabindex="-1"></a><span class="fu">extractAIC</span>(hipcenter_mod_both_aic)</span></code></pre></div>
 <pre><code>## [1]   4.0000 274.2418</code></pre>
 <p>We see that two of the models chosen by search procedures have the best possible <span class="math inline">\(\text{AIC}\)</span>, as they are the same model. This is however never guaranteed. We see that the model chosen using backwards selection does not achieve the smallest possible <span class="math inline">\(\text{AIC}\)</span>.</p>
-<div class="sourceCode" id="cb1389"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1389-1"><a href="variable-selection-and-model-building.html#cb1389-1" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(hipcenter_mod_aic <span class="sc">~</span> <span class="fu">I</span>(<span class="dv">2</span><span class="sc">:</span>p), <span class="at">ylab =</span> <span class="st">&quot;AIC&quot;</span>, <span class="at">xlab =</span> <span class="st">&quot;p, number of parameters&quot;</span>, </span>
-<span id="cb1389-2"><a href="variable-selection-and-model-building.html#cb1389-2" aria-hidden="true" tabindex="-1"></a>     <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">type =</span> <span class="st">&quot;b&quot;</span>, <span class="at">cex =</span> <span class="dv">2</span>,</span>
-<span id="cb1389-3"><a href="variable-selection-and-model-building.html#cb1389-3" aria-hidden="true" tabindex="-1"></a>     <span class="at">main =</span> <span class="st">&quot;AIC vs Model Complexity&quot;</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1389"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1389-1"><a href="variable-selection-and-model-building.html#cb1389-1" tabindex="-1"></a><span class="fu">plot</span>(hipcenter_mod_aic <span class="sc">~</span> <span class="fu">I</span>(<span class="dv">2</span><span class="sc">:</span>p), <span class="at">ylab =</span> <span class="st">&quot;AIC&quot;</span>, <span class="at">xlab =</span> <span class="st">&quot;p, number of parameters&quot;</span>, </span>
+<span id="cb1389-2"><a href="variable-selection-and-model-building.html#cb1389-2" tabindex="-1"></a>     <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">type =</span> <span class="st">&quot;b&quot;</span>, <span class="at">cex =</span> <span class="dv">2</span>,</span>
+<span id="cb1389-3"><a href="variable-selection-and-model-building.html#cb1389-3" tabindex="-1"></a>     <span class="at">main =</span> <span class="st">&quot;AIC vs Model Complexity&quot;</span>)</span></code></pre></div>
 <p><img src="selection_files/figure-html/unnamed-chunk-37-1.png" width="672" style="display: block; margin: auto;" /></p>
 <p>We could easily repeat this process for <span class="math inline">\(\text{BIC}\)</span>.</p>
 <p><span class="math display">\[
 \text{BIC} = n\log\left(\frac{\text{RSS}}{n}\right) + \log(n)p.
 \]</span></p>
-<div class="sourceCode" id="cb1390"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1390-1"><a href="variable-selection-and-model-building.html#cb1390-1" aria-hidden="true" tabindex="-1"></a>hipcenter_mod_bic <span class="ot">=</span> n <span class="sc">*</span> <span class="fu">log</span>(all_hipcenter_mod<span class="sc">$</span>rss <span class="sc">/</span> n) <span class="sc">+</span> <span class="fu">log</span>(n) <span class="sc">*</span> (<span class="dv">2</span><span class="sc">:</span>p)</span></code></pre></div>
-<div class="sourceCode" id="cb1391"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1391-1"><a href="variable-selection-and-model-building.html#cb1391-1" aria-hidden="true" tabindex="-1"></a><span class="fu">which.min</span>(hipcenter_mod_bic)</span></code></pre></div>
+<div class="sourceCode" id="cb1390"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1390-1"><a href="variable-selection-and-model-building.html#cb1390-1" tabindex="-1"></a>hipcenter_mod_bic <span class="ot">=</span> n <span class="sc">*</span> <span class="fu">log</span>(all_hipcenter_mod<span class="sc">$</span>rss <span class="sc">/</span> n) <span class="sc">+</span> <span class="fu">log</span>(n) <span class="sc">*</span> (<span class="dv">2</span><span class="sc">:</span>p)</span></code></pre></div>
+<div class="sourceCode" id="cb1391"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1391-1"><a href="variable-selection-and-model-building.html#cb1391-1" tabindex="-1"></a><span class="fu">which.min</span>(hipcenter_mod_bic)</span></code></pre></div>
 <pre><code>## [1] 1</code></pre>
-<div class="sourceCode" id="cb1393"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1393-1"><a href="variable-selection-and-model-building.html#cb1393-1" aria-hidden="true" tabindex="-1"></a>all_hipcenter_mod<span class="sc">$</span>which[<span class="dv">1</span>,]</span></code></pre></div>
+<div class="sourceCode" id="cb1393"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1393-1"><a href="variable-selection-and-model-building.html#cb1393-1" tabindex="-1"></a>all_hipcenter_mod<span class="sc">$</span>which[<span class="dv">1</span>,]</span></code></pre></div>
 <pre><code>## (Intercept)         Age      Weight     HtShoes          Ht      Seated 
 ##        TRUE       FALSE       FALSE       FALSE        TRUE       FALSE 
 ##         Arm       Thigh         Leg 
 ##       FALSE       FALSE       FALSE</code></pre>
-<div class="sourceCode" id="cb1395"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1395-1"><a href="variable-selection-and-model-building.html#cb1395-1" aria-hidden="true" tabindex="-1"></a>hipcenter_mod_best_bic <span class="ot">=</span> <span class="fu">lm</span>(hipcenter <span class="sc">~</span> Ht, <span class="at">data =</span> seatpos)</span></code></pre></div>
-<div class="sourceCode" id="cb1396"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1396-1"><a href="variable-selection-and-model-building.html#cb1396-1" aria-hidden="true" tabindex="-1"></a><span class="fu">extractAIC</span>(hipcenter_mod_best_bic, <span class="at">k =</span> <span class="fu">log</span>(n))</span></code></pre></div>
+<div class="sourceCode" id="cb1395"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1395-1"><a href="variable-selection-and-model-building.html#cb1395-1" tabindex="-1"></a>hipcenter_mod_best_bic <span class="ot">=</span> <span class="fu">lm</span>(hipcenter <span class="sc">~</span> Ht, <span class="at">data =</span> seatpos)</span></code></pre></div>
+<div class="sourceCode" id="cb1396"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1396-1"><a href="variable-selection-and-model-building.html#cb1396-1" tabindex="-1"></a><span class="fu">extractAIC</span>(hipcenter_mod_best_bic, <span class="at">k =</span> <span class="fu">log</span>(n))</span></code></pre></div>
 <pre><code>## [1]   2.0000 278.3418</code></pre>
-<div class="sourceCode" id="cb1398"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1398-1"><a href="variable-selection-and-model-building.html#cb1398-1" aria-hidden="true" tabindex="-1"></a><span class="fu">extractAIC</span>(hipcenter_mod_back_bic, <span class="at">k =</span> <span class="fu">log</span>(n))</span></code></pre></div>
+<div class="sourceCode" id="cb1398"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1398-1"><a href="variable-selection-and-model-building.html#cb1398-1" tabindex="-1"></a><span class="fu">extractAIC</span>(hipcenter_mod_back_bic, <span class="at">k =</span> <span class="fu">log</span>(n))</span></code></pre></div>
 <pre><code>## [1]   2.0000 278.7306</code></pre>
-<div class="sourceCode" id="cb1400"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1400-1"><a href="variable-selection-and-model-building.html#cb1400-1" aria-hidden="true" tabindex="-1"></a><span class="fu">extractAIC</span>(hipcenter_mod_forw_bic, <span class="at">k =</span> <span class="fu">log</span>(n))</span></code></pre></div>
+<div class="sourceCode" id="cb1400"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1400-1"><a href="variable-selection-and-model-building.html#cb1400-1" tabindex="-1"></a><span class="fu">extractAIC</span>(hipcenter_mod_forw_bic, <span class="at">k =</span> <span class="fu">log</span>(n))</span></code></pre></div>
 <pre><code>## [1]   2.0000 278.3418</code></pre>
-<div class="sourceCode" id="cb1402"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1402-1"><a href="variable-selection-and-model-building.html#cb1402-1" aria-hidden="true" tabindex="-1"></a><span class="fu">extractAIC</span>(hipcenter_mod_both_bic, <span class="at">k =</span> <span class="fu">log</span>(n))</span></code></pre></div>
+<div class="sourceCode" id="cb1402"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1402-1"><a href="variable-selection-and-model-building.html#cb1402-1" tabindex="-1"></a><span class="fu">extractAIC</span>(hipcenter_mod_both_bic, <span class="at">k =</span> <span class="fu">log</span>(n))</span></code></pre></div>
 <pre><code>## [1]   2.0000 278.3418</code></pre>
 </div>
 </div>
 <div id="higher-order-terms" class="section level2 hasAnchor" number="16.3">
 <h2><span class="header-section-number">16.3</span> Higher Order Terms<a href="variable-selection-and-model-building.html#higher-order-terms" class="anchor-section" aria-label="Anchor link to header"></a></h2>
 <p>So far we have only allowed first-order terms in our models. Let’s return to the <code>autompg</code> dataset to explore higher-order terms.</p>
-<div class="sourceCode" id="cb1404"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1404-1"><a href="variable-selection-and-model-building.html#cb1404-1" aria-hidden="true" tabindex="-1"></a>autompg <span class="ot">=</span> <span class="fu">read.table</span>(</span>
-<span id="cb1404-2"><a href="variable-selection-and-model-building.html#cb1404-2" aria-hidden="true" tabindex="-1"></a>  <span class="st">&quot;http://archive.ics.uci.edu/ml/machine-learning-databases/auto-mpg/auto-mpg.data&quot;</span>,</span>
-<span id="cb1404-3"><a href="variable-selection-and-model-building.html#cb1404-3" aria-hidden="true" tabindex="-1"></a>  <span class="at">quote =</span> <span class="st">&quot;</span><span class="sc">\&quot;</span><span class="st">&quot;</span>,</span>
-<span id="cb1404-4"><a href="variable-selection-and-model-building.html#cb1404-4" aria-hidden="true" tabindex="-1"></a>  <span class="at">comment.char =</span> <span class="st">&quot;&quot;</span>,</span>
-<span id="cb1404-5"><a href="variable-selection-and-model-building.html#cb1404-5" aria-hidden="true" tabindex="-1"></a>  <span class="at">stringsAsFactors =</span> <span class="cn">FALSE</span>)</span>
-<span id="cb1404-6"><a href="variable-selection-and-model-building.html#cb1404-6" aria-hidden="true" tabindex="-1"></a><span class="fu">colnames</span>(autompg) <span class="ot">=</span> <span class="fu">c</span>(<span class="st">&quot;mpg&quot;</span>, <span class="st">&quot;cyl&quot;</span>, <span class="st">&quot;disp&quot;</span>, <span class="st">&quot;hp&quot;</span>, <span class="st">&quot;wt&quot;</span>, <span class="st">&quot;acc&quot;</span>, </span>
-<span id="cb1404-7"><a href="variable-selection-and-model-building.html#cb1404-7" aria-hidden="true" tabindex="-1"></a>                      <span class="st">&quot;year&quot;</span>, <span class="st">&quot;origin&quot;</span>, <span class="st">&quot;name&quot;</span>)</span>
-<span id="cb1404-8"><a href="variable-selection-and-model-building.html#cb1404-8" aria-hidden="true" tabindex="-1"></a>autompg <span class="ot">=</span> <span class="fu">subset</span>(autompg, autompg<span class="sc">$</span>hp <span class="sc">!=</span> <span class="st">&quot;?&quot;</span>)</span>
-<span id="cb1404-9"><a href="variable-selection-and-model-building.html#cb1404-9" aria-hidden="true" tabindex="-1"></a>autompg <span class="ot">=</span> <span class="fu">subset</span>(autompg, autompg<span class="sc">$</span>name <span class="sc">!=</span> <span class="st">&quot;plymouth reliant&quot;</span>)</span>
-<span id="cb1404-10"><a href="variable-selection-and-model-building.html#cb1404-10" aria-hidden="true" tabindex="-1"></a><span class="fu">rownames</span>(autompg) <span class="ot">=</span> <span class="fu">paste</span>(autompg<span class="sc">$</span>cyl, <span class="st">&quot;cylinder&quot;</span>, autompg<span class="sc">$</span>year, autompg<span class="sc">$</span>name)</span>
-<span id="cb1404-11"><a href="variable-selection-and-model-building.html#cb1404-11" aria-hidden="true" tabindex="-1"></a>autompg<span class="sc">$</span>hp <span class="ot">=</span> <span class="fu">as.numeric</span>(autompg<span class="sc">$</span>hp)</span>
-<span id="cb1404-12"><a href="variable-selection-and-model-building.html#cb1404-12" aria-hidden="true" tabindex="-1"></a>autompg<span class="sc">$</span>domestic <span class="ot">=</span> <span class="fu">as.numeric</span>(autompg<span class="sc">$</span>origin <span class="sc">==</span> <span class="dv">1</span>)</span>
-<span id="cb1404-13"><a href="variable-selection-and-model-building.html#cb1404-13" aria-hidden="true" tabindex="-1"></a>autompg <span class="ot">=</span> autompg[autompg<span class="sc">$</span>cyl <span class="sc">!=</span> <span class="dv">5</span>,]</span>
-<span id="cb1404-14"><a href="variable-selection-and-model-building.html#cb1404-14" aria-hidden="true" tabindex="-1"></a>autompg <span class="ot">=</span> autompg[autompg<span class="sc">$</span>cyl <span class="sc">!=</span> <span class="dv">3</span>,]</span>
-<span id="cb1404-15"><a href="variable-selection-and-model-building.html#cb1404-15" aria-hidden="true" tabindex="-1"></a>autompg<span class="sc">$</span>cyl <span class="ot">=</span> <span class="fu">as.factor</span>(autompg<span class="sc">$</span>cyl)</span>
-<span id="cb1404-16"><a href="variable-selection-and-model-building.html#cb1404-16" aria-hidden="true" tabindex="-1"></a>autompg<span class="sc">$</span>domestic <span class="ot">=</span> <span class="fu">as.factor</span>(autompg<span class="sc">$</span>domestic)</span>
-<span id="cb1404-17"><a href="variable-selection-and-model-building.html#cb1404-17" aria-hidden="true" tabindex="-1"></a>autompg <span class="ot">=</span> <span class="fu">subset</span>(autompg, <span class="at">select =</span> <span class="fu">c</span>(<span class="st">&quot;mpg&quot;</span>, <span class="st">&quot;cyl&quot;</span>, <span class="st">&quot;disp&quot;</span>, <span class="st">&quot;hp&quot;</span>, </span>
-<span id="cb1404-18"><a href="variable-selection-and-model-building.html#cb1404-18" aria-hidden="true" tabindex="-1"></a>                                     <span class="st">&quot;wt&quot;</span>, <span class="st">&quot;acc&quot;</span>, <span class="st">&quot;year&quot;</span>, <span class="st">&quot;domestic&quot;</span>))</span></code></pre></div>
-<div class="sourceCode" id="cb1405"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1405-1"><a href="variable-selection-and-model-building.html#cb1405-1" aria-hidden="true" tabindex="-1"></a><span class="fu">str</span>(autompg)</span></code></pre></div>
+<div class="sourceCode" id="cb1404"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1404-1"><a href="variable-selection-and-model-building.html#cb1404-1" tabindex="-1"></a>autompg <span class="ot">=</span> <span class="fu">read.table</span>(</span>
+<span id="cb1404-2"><a href="variable-selection-and-model-building.html#cb1404-2" tabindex="-1"></a>  <span class="st">&quot;http://archive.ics.uci.edu/ml/machine-learning-databases/auto-mpg/auto-mpg.data&quot;</span>,</span>
+<span id="cb1404-3"><a href="variable-selection-and-model-building.html#cb1404-3" tabindex="-1"></a>  <span class="at">quote =</span> <span class="st">&quot;</span><span class="sc">\&quot;</span><span class="st">&quot;</span>,</span>
+<span id="cb1404-4"><a href="variable-selection-and-model-building.html#cb1404-4" tabindex="-1"></a>  <span class="at">comment.char =</span> <span class="st">&quot;&quot;</span>,</span>
+<span id="cb1404-5"><a href="variable-selection-and-model-building.html#cb1404-5" tabindex="-1"></a>  <span class="at">stringsAsFactors =</span> <span class="cn">FALSE</span>)</span>
+<span id="cb1404-6"><a href="variable-selection-and-model-building.html#cb1404-6" tabindex="-1"></a><span class="fu">colnames</span>(autompg) <span class="ot">=</span> <span class="fu">c</span>(<span class="st">&quot;mpg&quot;</span>, <span class="st">&quot;cyl&quot;</span>, <span class="st">&quot;disp&quot;</span>, <span class="st">&quot;hp&quot;</span>, <span class="st">&quot;wt&quot;</span>, <span class="st">&quot;acc&quot;</span>, </span>
+<span id="cb1404-7"><a href="variable-selection-and-model-building.html#cb1404-7" tabindex="-1"></a>                      <span class="st">&quot;year&quot;</span>, <span class="st">&quot;origin&quot;</span>, <span class="st">&quot;name&quot;</span>)</span>
+<span id="cb1404-8"><a href="variable-selection-and-model-building.html#cb1404-8" tabindex="-1"></a>autompg <span class="ot">=</span> <span class="fu">subset</span>(autompg, autompg<span class="sc">$</span>hp <span class="sc">!=</span> <span class="st">&quot;?&quot;</span>)</span>
+<span id="cb1404-9"><a href="variable-selection-and-model-building.html#cb1404-9" tabindex="-1"></a>autompg <span class="ot">=</span> <span class="fu">subset</span>(autompg, autompg<span class="sc">$</span>name <span class="sc">!=</span> <span class="st">&quot;plymouth reliant&quot;</span>)</span>
+<span id="cb1404-10"><a href="variable-selection-and-model-building.html#cb1404-10" tabindex="-1"></a><span class="fu">rownames</span>(autompg) <span class="ot">=</span> <span class="fu">paste</span>(autompg<span class="sc">$</span>cyl, <span class="st">&quot;cylinder&quot;</span>, autompg<span class="sc">$</span>year, autompg<span class="sc">$</span>name)</span>
+<span id="cb1404-11"><a href="variable-selection-and-model-building.html#cb1404-11" tabindex="-1"></a>autompg<span class="sc">$</span>hp <span class="ot">=</span> <span class="fu">as.numeric</span>(autompg<span class="sc">$</span>hp)</span>
+<span id="cb1404-12"><a href="variable-selection-and-model-building.html#cb1404-12" tabindex="-1"></a>autompg<span class="sc">$</span>domestic <span class="ot">=</span> <span class="fu">as.numeric</span>(autompg<span class="sc">$</span>origin <span class="sc">==</span> <span class="dv">1</span>)</span>
+<span id="cb1404-13"><a href="variable-selection-and-model-building.html#cb1404-13" tabindex="-1"></a>autompg <span class="ot">=</span> autompg[autompg<span class="sc">$</span>cyl <span class="sc">!=</span> <span class="dv">5</span>,]</span>
+<span id="cb1404-14"><a href="variable-selection-and-model-building.html#cb1404-14" tabindex="-1"></a>autompg <span class="ot">=</span> autompg[autompg<span class="sc">$</span>cyl <span class="sc">!=</span> <span class="dv">3</span>,]</span>
+<span id="cb1404-15"><a href="variable-selection-and-model-building.html#cb1404-15" tabindex="-1"></a>autompg<span class="sc">$</span>cyl <span class="ot">=</span> <span class="fu">as.factor</span>(autompg<span class="sc">$</span>cyl)</span>
+<span id="cb1404-16"><a href="variable-selection-and-model-building.html#cb1404-16" tabindex="-1"></a>autompg<span class="sc">$</span>domestic <span class="ot">=</span> <span class="fu">as.factor</span>(autompg<span class="sc">$</span>domestic)</span>
+<span id="cb1404-17"><a href="variable-selection-and-model-building.html#cb1404-17" tabindex="-1"></a>autompg <span class="ot">=</span> <span class="fu">subset</span>(autompg, <span class="at">select =</span> <span class="fu">c</span>(<span class="st">&quot;mpg&quot;</span>, <span class="st">&quot;cyl&quot;</span>, <span class="st">&quot;disp&quot;</span>, <span class="st">&quot;hp&quot;</span>, </span>
+<span id="cb1404-18"><a href="variable-selection-and-model-building.html#cb1404-18" tabindex="-1"></a>                                     <span class="st">&quot;wt&quot;</span>, <span class="st">&quot;acc&quot;</span>, <span class="st">&quot;year&quot;</span>, <span class="st">&quot;domestic&quot;</span>))</span></code></pre></div>
+<div class="sourceCode" id="cb1405"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1405-1"><a href="variable-selection-and-model-building.html#cb1405-1" tabindex="-1"></a><span class="fu">str</span>(autompg)</span></code></pre></div>
 <pre><code>## &#39;data.frame&#39;:    383 obs. of  8 variables:
 ##  $ mpg     : num  18 15 18 16 17 15 14 14 14 15 ...
 ##  $ cyl     : Factor w/ 3 levels &quot;4&quot;,&quot;6&quot;,&quot;8&quot;: 3 3 3 3 3 3 3 3 3 3 ...
@@ -1182,26 +1182,26 @@ <h2><span class="header-section-number">16.3</span> Higher Order Terms<a href="v
 ##  $ year    : int  70 70 70 70 70 70 70 70 70 70 ...
 ##  $ domestic: Factor w/ 2 levels &quot;0&quot;,&quot;1&quot;: 2 2 2 2 2 2 2 2 2 2 ...</code></pre>
 <p>Recall that we have two factor variables, <code>cyl</code> and <code>domestic</code>. The <code>cyl</code> variable has three levels, while the <code>domestic</code> variable has only two. Thus the <code>cyl</code> variable will be coded using two dummy variables, while the <code>domestic</code> variable will only need one. We will pay attention to this later.</p>
-<div class="sourceCode" id="cb1407"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1407-1"><a href="variable-selection-and-model-building.html#cb1407-1" aria-hidden="true" tabindex="-1"></a><span class="fu">pairs</span>(autompg, <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1407"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1407-1"><a href="variable-selection-and-model-building.html#cb1407-1" tabindex="-1"></a><span class="fu">pairs</span>(autompg, <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>)</span></code></pre></div>
 <p><img src="selection_files/figure-html/unnamed-chunk-44-1.png" width="768" style="display: block; margin: auto;" /></p>
 <p>We’ll use the <code>pairs()</code> plot to determine which variables may benefit from a quadratic relationship with the response. We’ll also consider all possible two-way interactions. We won’t consider any three-order or higher. For example, we won’t consider the interaction between first-order terms and the added quadratic terms.</p>
 <p>So now, we’ll fit this rather large model. We’ll use a log-transformed response. Notice that <code>log(mpg) ~ . ^ 2</code> will automatically consider all first-order terms, as well as all two-way interactions. We use <code>I(var_name ^ 2)</code> to add quadratic terms for some variables. This generally works better than using <code>poly()</code> when performing variable selection.</p>
-<div class="sourceCode" id="cb1408"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1408-1"><a href="variable-selection-and-model-building.html#cb1408-1" aria-hidden="true" tabindex="-1"></a>autompg_big_mod <span class="ot">=</span> <span class="fu">lm</span>(</span>
-<span id="cb1408-2"><a href="variable-selection-and-model-building.html#cb1408-2" aria-hidden="true" tabindex="-1"></a>  <span class="fu">log</span>(mpg) <span class="sc">~</span> . <span class="sc">^</span> <span class="dv">2</span> <span class="sc">+</span> <span class="fu">I</span>(disp <span class="sc">^</span> <span class="dv">2</span>) <span class="sc">+</span> <span class="fu">I</span>(hp <span class="sc">^</span> <span class="dv">2</span>) <span class="sc">+</span> <span class="fu">I</span>(wt <span class="sc">^</span> <span class="dv">2</span>) <span class="sc">+</span> <span class="fu">I</span>(acc <span class="sc">^</span> <span class="dv">2</span>), </span>
-<span id="cb1408-3"><a href="variable-selection-and-model-building.html#cb1408-3" aria-hidden="true" tabindex="-1"></a>  <span class="at">data =</span> autompg)</span></code></pre></div>
+<div class="sourceCode" id="cb1408"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1408-1"><a href="variable-selection-and-model-building.html#cb1408-1" tabindex="-1"></a>autompg_big_mod <span class="ot">=</span> <span class="fu">lm</span>(</span>
+<span id="cb1408-2"><a href="variable-selection-and-model-building.html#cb1408-2" tabindex="-1"></a>  <span class="fu">log</span>(mpg) <span class="sc">~</span> . <span class="sc">^</span> <span class="dv">2</span> <span class="sc">+</span> <span class="fu">I</span>(disp <span class="sc">^</span> <span class="dv">2</span>) <span class="sc">+</span> <span class="fu">I</span>(hp <span class="sc">^</span> <span class="dv">2</span>) <span class="sc">+</span> <span class="fu">I</span>(wt <span class="sc">^</span> <span class="dv">2</span>) <span class="sc">+</span> <span class="fu">I</span>(acc <span class="sc">^</span> <span class="dv">2</span>), </span>
+<span id="cb1408-3"><a href="variable-selection-and-model-building.html#cb1408-3" tabindex="-1"></a>  <span class="at">data =</span> autompg)</span></code></pre></div>
 <p>We think it is rather unlikely that we truly need all of these terms. There are quite a few!</p>
-<div class="sourceCode" id="cb1409"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1409-1"><a href="variable-selection-and-model-building.html#cb1409-1" aria-hidden="true" tabindex="-1"></a><span class="fu">length</span>(<span class="fu">coef</span>(autompg_big_mod))</span></code></pre></div>
+<div class="sourceCode" id="cb1409"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1409-1"><a href="variable-selection-and-model-building.html#cb1409-1" tabindex="-1"></a><span class="fu">length</span>(<span class="fu">coef</span>(autompg_big_mod))</span></code></pre></div>
 <pre><code>## [1] 40</code></pre>
 <p>We’ll try backwards search with both <span class="math inline">\(\text{AIC}\)</span> and <span class="math inline">\(\text{BIC}\)</span> to attempt to find a smaller, more reasonable model.</p>
-<div class="sourceCode" id="cb1411"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1411-1"><a href="variable-selection-and-model-building.html#cb1411-1" aria-hidden="true" tabindex="-1"></a>autompg_mod_back_aic <span class="ot">=</span> <span class="fu">step</span>(autompg_big_mod, <span class="at">direction =</span> <span class="st">&quot;backward&quot;</span>, <span class="at">trace =</span> <span class="dv">0</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1411"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1411-1"><a href="variable-selection-and-model-building.html#cb1411-1" tabindex="-1"></a>autompg_mod_back_aic <span class="ot">=</span> <span class="fu">step</span>(autompg_big_mod, <span class="at">direction =</span> <span class="st">&quot;backward&quot;</span>, <span class="at">trace =</span> <span class="dv">0</span>)</span></code></pre></div>
 <p>Notice that we used <code>trace = 0</code> in the function call. This suppresses the output for each step, and simply stores the chosen model. This is useful, as this code would otherwise create a large amount of output. If we had viewed the output, which you can try on your own by removing <code>trace = 0</code>, we would see that <code>R</code> only considers the <code>cyl</code> variable as a single variable, despite the fact that it is coded using two dummy variables. So removing <code>cyl</code> would actually remove two parameters from the resulting model.</p>
 <p>You should also notice that <code>R</code> respects hierarchy when attempting to remove variables. That is, for example, <code>R</code> will not consider removing <code>hp</code> if <code>hp:disp</code> or <code>I(hp ^ 2)</code> are currently in the model.</p>
 <p>We also use <span class="math inline">\(\text{BIC}\)</span>.</p>
-<div class="sourceCode" id="cb1412"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1412-1"><a href="variable-selection-and-model-building.html#cb1412-1" aria-hidden="true" tabindex="-1"></a>n <span class="ot">=</span> <span class="fu">length</span>(<span class="fu">resid</span>(autompg_big_mod))</span>
-<span id="cb1412-2"><a href="variable-selection-and-model-building.html#cb1412-2" aria-hidden="true" tabindex="-1"></a>autompg_mod_back_bic <span class="ot">=</span> <span class="fu">step</span>(autompg_big_mod, <span class="at">direction =</span> <span class="st">&quot;backward&quot;</span>, </span>
-<span id="cb1412-3"><a href="variable-selection-and-model-building.html#cb1412-3" aria-hidden="true" tabindex="-1"></a>                            <span class="at">k =</span> <span class="fu">log</span>(n), <span class="at">trace =</span> <span class="dv">0</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1412"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1412-1"><a href="variable-selection-and-model-building.html#cb1412-1" tabindex="-1"></a>n <span class="ot">=</span> <span class="fu">length</span>(<span class="fu">resid</span>(autompg_big_mod))</span>
+<span id="cb1412-2"><a href="variable-selection-and-model-building.html#cb1412-2" tabindex="-1"></a>autompg_mod_back_bic <span class="ot">=</span> <span class="fu">step</span>(autompg_big_mod, <span class="at">direction =</span> <span class="st">&quot;backward&quot;</span>, </span>
+<span id="cb1412-3"><a href="variable-selection-and-model-building.html#cb1412-3" tabindex="-1"></a>                            <span class="at">k =</span> <span class="fu">log</span>(n), <span class="at">trace =</span> <span class="dv">0</span>)</span></code></pre></div>
 <p>Looking at the coefficients of the two chosen models, we see they are still rather large.</p>
-<div class="sourceCode" id="cb1413"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1413-1"><a href="variable-selection-and-model-building.html#cb1413-1" aria-hidden="true" tabindex="-1"></a><span class="fu">coef</span>(autompg_mod_back_aic)</span></code></pre></div>
+<div class="sourceCode" id="cb1413"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1413-1"><a href="variable-selection-and-model-building.html#cb1413-1" tabindex="-1"></a><span class="fu">coef</span>(autompg_mod_back_aic)</span></code></pre></div>
 <pre><code>##    (Intercept)           cyl6           cyl8           disp             hp 
 ##   3.671884e+00  -1.602563e-01  -8.581644e-01  -9.371971e-03   2.293534e-02 
 ##             wt            acc           year      domestic1        I(hp^2) 
@@ -1210,7 +1210,7 @@ <h2><span class="header-section-number">16.3</span> Higher Order Terms<a href="v
 ##   7.220298e-03   5.041915e-02   5.797816e-07   9.493770e-05  -5.062295e-04 
 ##        hp:year       acc:year  acc:domestic1 year:domestic1 
 ##  -1.838985e-04   2.345625e-03  -2.372468e-02  -7.332725e-03</code></pre>
-<div class="sourceCode" id="cb1415"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1415-1"><a href="variable-selection-and-model-building.html#cb1415-1" aria-hidden="true" tabindex="-1"></a><span class="fu">coef</span>(autompg_mod_back_bic)</span></code></pre></div>
+<div class="sourceCode" id="cb1415"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1415-1"><a href="variable-selection-and-model-building.html#cb1415-1" tabindex="-1"></a><span class="fu">coef</span>(autompg_mod_back_bic)</span></code></pre></div>
 <pre><code>##   (Intercept)          cyl6          cyl8          disp            hp 
 ##  4.657847e+00 -1.086165e-01 -7.611631e-01 -1.609316e-03  2.621266e-03 
 ##            wt           acc          year     domestic1      cyl6:acc 
@@ -1218,25 +1218,25 @@ <h2><span class="header-section-number">16.3</span> Higher Order Terms<a href="v
 ##      cyl8:acc       disp:wt        hp:acc      acc:year acc:domestic1 
 ##  4.610095e-02  4.102804e-07 -3.386261e-04  2.500137e-03 -2.193294e-02</code></pre>
 <p>However, they are much smaller than the original full model. Also notice that the resulting models respect hierarchy.</p>
-<div class="sourceCode" id="cb1417"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1417-1"><a href="variable-selection-and-model-building.html#cb1417-1" aria-hidden="true" tabindex="-1"></a><span class="fu">length</span>(<span class="fu">coef</span>(autompg_big_mod))</span></code></pre></div>
+<div class="sourceCode" id="cb1417"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1417-1"><a href="variable-selection-and-model-building.html#cb1417-1" tabindex="-1"></a><span class="fu">length</span>(<span class="fu">coef</span>(autompg_big_mod))</span></code></pre></div>
 <pre><code>## [1] 40</code></pre>
-<div class="sourceCode" id="cb1419"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1419-1"><a href="variable-selection-and-model-building.html#cb1419-1" aria-hidden="true" tabindex="-1"></a><span class="fu">length</span>(<span class="fu">coef</span>(autompg_mod_back_aic))</span></code></pre></div>
+<div class="sourceCode" id="cb1419"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1419-1"><a href="variable-selection-and-model-building.html#cb1419-1" tabindex="-1"></a><span class="fu">length</span>(<span class="fu">coef</span>(autompg_mod_back_aic))</span></code></pre></div>
 <pre><code>## [1] 19</code></pre>
-<div class="sourceCode" id="cb1421"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1421-1"><a href="variable-selection-and-model-building.html#cb1421-1" aria-hidden="true" tabindex="-1"></a><span class="fu">length</span>(<span class="fu">coef</span>(autompg_mod_back_bic))</span></code></pre></div>
+<div class="sourceCode" id="cb1421"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1421-1"><a href="variable-selection-and-model-building.html#cb1421-1" tabindex="-1"></a><span class="fu">length</span>(<span class="fu">coef</span>(autompg_mod_back_bic))</span></code></pre></div>
 <pre><code>## [1] 15</code></pre>
 <p>Calculating the LOOCV <span class="math inline">\(\text{RMSE}\)</span> for each, we see that the model chosen using <span class="math inline">\(\text{BIC}\)</span> performs the best. That means that it is both the best model for prediction, since it achieves the best LOOCV <span class="math inline">\(\text{RMSE}\)</span>, but also the best model for explanation, as it is also the smallest.</p>
-<div class="sourceCode" id="cb1423"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1423-1"><a href="variable-selection-and-model-building.html#cb1423-1" aria-hidden="true" tabindex="-1"></a><span class="fu">calc_loocv_rmse</span>(autompg_big_mod)</span></code></pre></div>
+<div class="sourceCode" id="cb1423"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1423-1"><a href="variable-selection-and-model-building.html#cb1423-1" tabindex="-1"></a><span class="fu">calc_loocv_rmse</span>(autompg_big_mod)</span></code></pre></div>
 <pre><code>## [1] 0.1112024</code></pre>
-<div class="sourceCode" id="cb1425"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1425-1"><a href="variable-selection-and-model-building.html#cb1425-1" aria-hidden="true" tabindex="-1"></a><span class="fu">calc_loocv_rmse</span>(autompg_mod_back_aic)</span></code></pre></div>
+<div class="sourceCode" id="cb1425"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1425-1"><a href="variable-selection-and-model-building.html#cb1425-1" tabindex="-1"></a><span class="fu">calc_loocv_rmse</span>(autompg_mod_back_aic)</span></code></pre></div>
 <pre><code>## [1] 0.1032888</code></pre>
-<div class="sourceCode" id="cb1427"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1427-1"><a href="variable-selection-and-model-building.html#cb1427-1" aria-hidden="true" tabindex="-1"></a><span class="fu">calc_loocv_rmse</span>(autompg_mod_back_bic)</span></code></pre></div>
+<div class="sourceCode" id="cb1427"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1427-1"><a href="variable-selection-and-model-building.html#cb1427-1" tabindex="-1"></a><span class="fu">calc_loocv_rmse</span>(autompg_mod_back_bic)</span></code></pre></div>
 <pre><code>## [1] 0.103134</code></pre>
 </div>
 <div id="explanation-versus-prediction-1" class="section level2 hasAnchor" number="16.4">
 <h2><span class="header-section-number">16.4</span> Explanation versus Prediction<a href="variable-selection-and-model-building.html#explanation-versus-prediction-1" class="anchor-section" aria-label="Anchor link to header"></a></h2>
 <p>Throughout this chapter, we have attempted to find reasonably “small” models, which are good at <strong>explaining</strong> the relationship between the response and the predictors, that also have small errors which are thus good for making <strong>predictions</strong>.</p>
 <p>We’ll further discuss the model <code>autompg_mod_back_bic</code> to better explain the difference between using models for <em>explaining</em> and <em>predicting</em>. This is the model fit to the <code>autompg</code> data that was chosen using Backwards Search and <span class="math inline">\(\text{BIC}\)</span>, which obtained the lowest LOOCV <span class="math inline">\(\text{RMSE}\)</span> of the models we considered.</p>
-<div class="sourceCode" id="cb1429"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1429-1"><a href="variable-selection-and-model-building.html#cb1429-1" aria-hidden="true" tabindex="-1"></a>autompg_mod_back_bic</span></code></pre></div>
+<div class="sourceCode" id="cb1429"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1429-1"><a href="variable-selection-and-model-building.html#cb1429-1" tabindex="-1"></a>autompg_mod_back_bic</span></code></pre></div>
 <pre><code>## 
 ## Call:
 ## lm(formula = log(mpg) ~ cyl + disp + hp + wt + acc + year + domestic + 
@@ -1260,7 +1260,7 @@ <h3><span class="header-section-number">16.4.1</span> Explanation<a href="variab
 <h4><span class="header-section-number">16.4.1.1</span> Correlation and Causation<a href="variable-selection-and-model-building.html#correlation-and-causation-1" class="anchor-section" aria-label="Anchor link to header"></a></h4>
 <p>A word of caution when using a model to <em>explain</em> a relationship. There are two terms often used to describe a relationship between two variables: <em>causation</em> and <em>correlation</em>. <a href="https://xkcd.com/552/" target="_blank">Correlation</a> is often also referred to as association.</p>
 <p>Just because two variables are correlated does not necessarily mean that one causes the other. For example, considering modeling <code>mpg</code> as only a function of <code>hp</code>.</p>
-<div class="sourceCode" id="cb1431"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1431-1"><a href="variable-selection-and-model-building.html#cb1431-1" aria-hidden="true" tabindex="-1"></a><span class="fu">plot</span>(mpg <span class="sc">~</span> hp, <span class="at">data =</span> autompg, <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">cex =</span> <span class="fl">1.5</span>)</span></code></pre></div>
+<div class="sourceCode" id="cb1431"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1431-1"><a href="variable-selection-and-model-building.html#cb1431-1" tabindex="-1"></a><span class="fu">plot</span>(mpg <span class="sc">~</span> hp, <span class="at">data =</span> autompg, <span class="at">col =</span> <span class="st">&quot;dodgerblue&quot;</span>, <span class="at">pch =</span> <span class="dv">20</span>, <span class="at">cex =</span> <span class="fl">1.5</span>)</span></code></pre></div>
 <p><img src="selection_files/figure-html/unnamed-chunk-53-1.png" width="672" style="display: block; margin: auto;" /></p>
 <p>Does an increase in horsepower cause a drop in fuel efficiency? Or, perhaps the causality is reversed and an increase in fuel efficiency causes a decrease in horsepower. Or, perhaps there is a third variable that explains both!</p>
 <p>The issue here is that we have <strong>observational</strong> data. With observational data, we can only detect associations. To speak with confidence about causality, we would need to run <strong>experiments</strong>.</p>
@@ -1282,7 +1282,7 @@ <h2><span class="header-section-number">16.5</span> <code>R</code> Markdown<a hr
 <ul>
 <li><a href="selection.Rmd" target="_blank"><code>selection.Rmd</code></a></li>
 </ul>
-<p>The file was created using <code>R</code> version <code>4.3.2</code>.</p>
+<p>The file was created using <code>R</code> version <code>4.4.1</code>.</p>
 
 </div>
 </div>