Sometimes the effect of one X variable on Y depends on the value of an other X variable. For example, in our clothing expenditure example, the impact of household size on clothing expenditure might depend on the household income. Increasing the household size by one more person might have a larger effect on clothing expenditure for richer households compared to poorer households.
To model such relationships we can use interaction terms. To interact two variables we can estimate the model:
\mathbb{E}\left[ Y_i|x_1,x_2 \right]=\beta_0 + \beta_1 x_1 + \beta_2 x_2 + \beta_3 x_1x_2
The 3rd term x_{i1} x_{i2} is called an interaction term. When we include this, the expected value of Y_i given x_{i1} now depends on the level of x_{i2} (and vice versa). To see this, let’s look at \mathbb{E}\left[ Y_i|x_{i1},x_{i2} \right] for different values of x_{i2}:
\begin{split}
\mathbb{E}\left[ Y_i|x_{i1},x_{i2}=0 \right] &=\beta_0 + \beta_1 x_{i1}\\
\mathbb{E}\left[ Y_i|x_{i1},x_{i2}=1 \right] &=\beta_0 + \beta_1 x_{i1} + \beta_2 + \beta_3 x_{i1}\\
\mathbb{E}\left[ Y_i|x_{i1},x_{i2}=2 \right] &=\beta_0 + \beta_1 x_{i1} + 2\beta_2 + 2\beta_3 x_{i1}\\
\end{split}
For each case, let’s increase x_{i1} by one unit:
\begin{split}
\mathbb{E}\left[ Y_i|x_{i1}+1,x_{i2}=0 \right] &=\beta_0 + \beta_1 x_{i1}+\beta_1\\
\mathbb{E}\left[ Y_i|x_{i1}+1,x_{i2}=1 \right] &=\beta_0 + \beta_1 x_{i1} + \beta_1 + \beta_2 + \beta_3 x_{i1} + \beta_3\\
\mathbb{E}\left[ Y_i|x_{i1}+1,x_{i2}=2 \right] &=\beta_0 + \beta_1 x_{i1} + \beta_1 + 2\beta_2 + 2\beta_3 x_{i1} + 2\beta_3\\
\end{split}
Taking the difference for each case:
\begin{split}
\mathbb{E}\left[ Y_i|x_{i1}+1,x_{i2}=0 \right]-\mathbb{E}\left[ Y_i|x_{i1},x_{i2}=0 \right] &=\beta_1\\
\mathbb{E}\left[ Y_i|x_{i1}+1,x_{i2}=1 \right]-\mathbb{E}\left[ Y_i|x_{i1},x_{i2}=1 \right] &= \beta_1 + \beta_3\\
\mathbb{E}\left[ Y_i|x_{i1}+1,x_{i2}=2 \right]-\mathbb{E}\left[ Y_i|x_{i1},x_{i2}=2 \right] &= \beta_1 + 2\beta_3\\
\end{split}
So:
When x_{i2}=0, a unit increase in x_{i1} increases Y_i by \beta_1 on average.
When x_{i2}=1, a unit increase in x_{i1} increases Y_i by \beta_1+\beta_3 on average.
When x_{i2}=2, a unit increase in x_{i1} increases Y_i by \beta_1+2\beta_3 on average.
When we include an interaction term, we therefore need to look at both \beta_1 and \beta_3 to learn about the impact of x_{i1} on Y_i.
23.2 Interaction Terms in R
Let’s try this out with the clothing expenditure data. We want to interact household income (X_{i1}) with household size (X_{i2}) and estimate the model:
\mathbb{E}\left[ Y_i|x_{i1},x_{i2} \right]=\beta_0 + \beta_1 x_{i1} + \beta_2 x_{i2} + \beta_3 x_{i1}x_{i2}
One way to do this is to create a new variable with the interaction and add it to the model. Let’s try this first:
Call:
lm(formula = clothing_exp ~ hh_inc + hh_size + hh_inc_hh_size,
data = df)
Residuals:
Min 1Q Median 3Q Max
-0.26738 -0.05882 -0.00592 0.05793 0.44451
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 0.0090748 0.0312002 0.291 0.771
hh_inc 0.0809904 0.0009976 81.189 <2e-16 ***
hh_size 0.0082387 0.0103454 0.796 0.426
hh_inc_hh_size 0.0003971 0.0003040 1.306 0.192
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Residual standard error: 0.1029 on 296 degrees of freedom
Multiple R-squared: 0.9922, Adjusted R-squared: 0.9921
F-statistic: 1.25e+04 on 3 and 296 DF, p-value: < 2.2e-16
But because interaction terms are so common in linear regression models, R has a shortcut to do this. All we have to do is include x1 * x2 in the formula and R will include the two level terms and the interaction term. So when we do this we don’t even need to add the individual x1 and x2 variables. Let’s try this out:
summary(lm(clothing_exp ~ hh_inc * hh_size, data = df))
Call:
lm(formula = clothing_exp ~ hh_inc * hh_size, data = df)
Residuals:
Min 1Q Median 3Q Max
-0.26738 -0.05882 -0.00592 0.05793 0.44451
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 0.0090748 0.0312002 0.291 0.771
hh_inc 0.0809904 0.0009976 81.189 <2e-16 ***
hh_size 0.0082387 0.0103454 0.796 0.426
hh_inc:hh_size 0.0003971 0.0003040 1.306 0.192
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Residual standard error: 0.1029 on 296 degrees of freedom
Multiple R-squared: 0.9922, Adjusted R-squared: 0.9921
F-statistic: 1.25e+04 on 3 and 296 DF, p-value: < 2.2e-16
We get the same as above! The term hh_inc:hh_size is the interaction term (Note: we can add an interaction term without the level terms to the model using x1:x2, but you should always include the level terms when doing an interaction).
Let’s interpret this. All terms, including the interaction term, are positive. With this model neither household size nor the interaction term are statistically significant. Ignoring statistical significance, we can interpret the parameter estimates as follows:
The larger the household size, the larger the effect of a unit increase in income on clothing expenditure.
This makes sense because if a large household gets more income they have more people they can buy clothes for.
The higher the household income, the larger the effect of a unit increase in household size on clothing expenditure.
This makes sense because if a richer household gets one more member in it, they have more money to buy clothes for the additional person.