MATH2010 Statistical Modelling I
SEMESTER 1 EXAMINATION 2021/22
1. [25 marks] Consider the following simple linear regression model without intercept:
Yi = βxi + ∈i , (1)
i = 1, . . . , n, where ∈i ~ N(0, σ2 ).
(a) [5 marks] Show that the least-squares estimator for β has the form.
Show thatβ(^) is unbiased, and compute its variance.
(b) [2 marks] Given explanatory variables x1 , . . . , xn and associated response variables y1 , . . . , yn, it is common to center the response and explanatory variables, i.e. to compute
Consider the alternative model:
YiI = βIxi(I) + ∈i. (2)
Derive the least-squares estimator for βI in terms of xi and Yi.
(c) [2 marks] Explain the difference between an estimator and an estimate. Write down the estimate corresponding to β' from model (2).
(d) [4 marks] Recall that under the simple linear regression model
Yi = β0 + β1xi + ∈i (3)
the estimate for β1 is given by
Show that the estimate obtained in (c) is equal to b(^)1 .
(e) The dataset in Table 1 gives the sales (yi; in £) and the advertising budget spent on newspaper advertisements (xi; in £) for a variety of products.
advertising.data = data.frame(
Newspaper=c(69.2, 45.1, 69.3, 58.5, 58.4),
Sales = c(22.1, 10.4, 9.3, 18.5, 12.9)
)
(i) [8 marks] Compute a 95% confidence interval for β in model (1), and β, in model (2). Test the null hypothesis that β = 0, and the null hypothesis that β, = 0, at the 95% level. You may find the following outputs from R helpful:
qt(0.95, 4)
## [1] 2.13
qt(0.95, 3)
## [1] 2.35
qt(0.975, 4)
## [1] 2.78
qt(0.975, 3)
## [1] 3.18
(ii) [4 marks] Consider the location x0 = 0, and let x = x0 - . Let
Y(^)0 = βx0 + ∈0 and Y(^)0, = β,x + ∈0 . Write down the conditional means
E(Y(^)0 ) and E(Y(^)0, ). Considering the computed conditional means, do you think
model (1) or model (2) is more appropriate for this data? Considering the
confidence intervals computed above, do you believe there is a significant relationship between budget spent on newspaper advertisements and sales?
2. [25 marks] Consider the usual multiple linear regression model Y = Xβ + ε ,
with Y an n × 1 response vector, X an n × (k + 1) design matrix, β a k + 1 vector of unknown parameters and ε ~ N (0, σ2In ). Assume least squares will be used to estimate β .
(a) [2 marks] Describe two diagnostic plots that are commonly used to check the assumptions that underpin a linear model, and discuss which assumptions they are intended to verify.
(b) [2 marks] Consider the following two plots. Explain what evidence of deviation from the model assumptions are shown in each plot.
(c) [8 marks] The leverage li of the ith observation is defined to be the derivative of the ith fitted value with-respect-to the ith observation,i.e.:
Show that li is given by
(d) [2 marks] A data point (xi , yi ) is said to be influential if the regression line
changes significantly when that point is excluded from the analysis. There can be more than one influential point for a regression model. Consider a scatter plot of leverages on the x-axis against standardised residuals on they-axis such as the figure below.
Which of the labelled points on the figure above do you think could be influential? Explain why.
(e) Consider the “leave-one-out” procedure, in which observation i is left out of the
regression analysis. Letb(^)(-i) denote the parameter vector obtained when
observation i is left out, and recall that we have the following formula for the change in the parameter vector:
where ri is the ith residual, ri = yi - ^(y)i. Consider the simple linear regression problem Yi = β0 + β1xi + ∈i. In this case, δ i is a vector of length 2,
δ i = [δi;0, δi;1]. The dataset in Table 2 gives 4 observations of the response yi , with explanatory variable xi, as well as the values of li and δi for all but the 4th observation.
The data can be read into R with the command:
data = data.frame(
x = c(-1, -0.9, 1, 5),
y = c(0.7569, 0.6948, 1.2881, 2.2798)
)
(i) [8 marks] Compute l4 , δ4;0 and δ4;1 . If you use R, or other software, to answer this question, please clearly state the commands you use.
(ii) [3 marks] Do you think the fourth point is more influential than the first three points? Explain why, or why not, particularly commenting on the values of l4 , δ4;0 and δ4;1 in your answer.
3. [25 marks] In a particular data set, n = 20 observations were made on a response variable (y) and three explanatory variables (x1, x2 and x3).
(a) [6 marks] Partial outputs from the summary command in R applied to lm objects regressing each of the explanatory variables on the response separately are given below. From this output, calculate AIC for each of the three models.
##
## Call:
## lm(formula = y ~ x1)
##
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 13.3004 0.5331 24.95 2.1e-15
## x1 0.0989 0.5857 0.17 0.87
##
## Residual standard error: 2.33 on 18 degrees of freedom
## F-statistic: 0.0285 on 1 and 18 DF, p-value: 0.868
##
## Call:
## lm(formula = y ~ x2)
##
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 13.334 0.262 50.96 < 2e-16
## x2 2.255 0.308 7.32 8.5e-07
##
## Residual standard error: 1.17 on 18 degrees of freedom
## F-statistic: 53.6 on 1 and 18 DF, p-value: 8.51e-07
##
## Call:
## lm(formula = y ~ x3)
##
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 13.302 0.250 53.18 < 2e-16
## x3 2.339 0.301 7.77 3.7e-07
##
## Residual standard error: 1.12 on 18 degrees of freedom
## F-statistic: 60.3 on 1 and 18 DF, p-value: 3.73e-07
(b) [2 marks] Given that AIC for the null model (containing no explanatory variables) is 31.783, which is the first variable that you would include when building a model using forward selection and AIC?
(c) [2 marks] A multiple linear regression model containing all three variables is now
fitted using lm, and partial output from the summary command is given below.
##
## Call:
## lm(formula = y ~ x1 + x2 + x3)
##
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 13.186 0.244 54.09 <2e-16
## x1 0.515 0.281 1.83 0.086
## x2 -1.192 3.171 -0.38 0.712
## x3 3.660 3.227 1.13 0.273
##
## Residual standard error: 1.06 on 16 degrees of freedom
Using t-tests, which variables can be individually dropped from the model without adversely affecting the quality of the fit?
(d) [5 marks] Give a possible explanation for the contradiction between your results from parts (b) and (c).
(e) [4 marks] Given that the sample variance of the response is sy(2) = 5.157, what is
the adjusted R2 value for the above multiple regression model? Is the model a good fit?
(f) [6 marks] Test the significance of the multiple regression model for these data; that is, compare it to the null model.
You may find the following quantities from R useful:
qf(0.95, 4, 16)
## [1] 3.01
qf(0.95, 3, 16)
## [1] 3.24
qf(0.95, 3, 20)
## [1] 3.1
4. [25 marks] Every year Forbes produces a data set of the top 2000 companies
world-wide, ranked by quantities such as Sales and Assets, across different market sectors, countries etc.
A linear model was fitted to the 2017 data, regressing log(Sales) against
log(Assets) (aquantitative variable) and Sector (a qualitative factor with 11 levels). The following analysis of variance table was obtained.
(a) [6 marks] Write down each of the nested model comparisons summarised in this table.
(b) [8 marks] Complete the ANOVA table. Give reasons for your choices for each entry in the “DF” column.
(c) [1 mark] What proportion of variation is explained by the model?
(d) [10 marks] Simplified output from the summary command from the fitted lm
object in R for this model is given on the next page. The units of each of Sales and Assets are billions of dollars; in the data set, Apple had sales of 217.5 and assets of 331.1, both in billions of dollars, larger than any other “Information
Technology” company.
What is the equation for the estimated mean response (log(Sales)) from a company in the “Information Technology” sector in terms of log(Assets)? What is it for the “Consumer Discretionary” Sector? Sketch a plot with these two equations. At what value of Assets is an information technology company predicted to have greater mean sales than a company in the consumer discretionary sector?
The company with largest sales in the data set is Wal-Mart (a consumer discretionary company), with sales of 485.3. According to the regression model, how large would Apple’s assets need to be to surpass Wal-Mart’s sales? What caveats do you have about this prediction?
Learning objectives:
LO1 Use the theory of linear models and matrix algebra to investigate standard and non- standard problems.
LO2 Interpret the output from an analysis including the meaning of interactions and terms based on qualitative factors.
LO3 Understand how to make a critical appraisal of a fitted model.
LO4 Carry out t-tests and calculate confidence intervals by hand and by computer. LO5 Using a variety of procedures for variable selection.
LO6 Fit multiple regression models using the adopted software package.
LO7 Carry out simple linear regression by computer.
LO6 and LO7 are assessed via coursework.