Further Maths Linear regression Practice Questions

Free Further Maths Linear regression practice questions with full step-by-step worked solutions. Covers least-squares, gradient, summary-statistics, intercept. Practise exam-style problems and check your method.

least-squaresgradientsummary-statisticsinterceptmean-pointresidual
Further Maths70 questionsStep-by-step solutions
Question 1
2 markseasy
A materials laboratory hangs a series of loads on a steel spring and measures the extension produced. For each observation, xx denotes the load (kg) and yy denotes the extension (mm). From the 55 paired observations it is found that Sxx=40S_{xx}=40 and Sxy=78S_{xy}=78. The least-squares regression line of yy on xx is the line y=a+bxy=a+bx that minimises the residual sum of squares ∑i=1n(yi−a−bxi)2\sum_{i=1}^{n}\left(y_i-a-bx_i\right)^2. Find the gradient bb of the least-squares regression line of yy on xx, giving your answer exactly.
Show worked solution

Worked solution

  1. Recall the least-squares gradient

    b=SxySxxb=\frac{S_{xy}}{S_{xx}}

    Setting the partial derivatives of the residual sum of squares to zero gives this value of bb.

  2. Substitute the two summary statistics

    b=SxySxx=7840b=\frac{S_{xy}}{S_{xx}}=\frac{78}{40}

    The gradient is the ratio of the two sums of squares, nothing else is needed.

  3. Evaluate the quotient

    b=7840=1.95b=\frac{78}{40}=1.95

    The sign of bb is the sign of SxyS_{xy}, since Sxx>0S_{xx}>0 always.

  4. State the gradient

    b=1.95b=1.95

    This is the least-squares estimate of the change in yy per unit increase in xx.

Answer
b=1.95b=1.95
Question 2
2 markseasy
A materials laboratory hangs a series of loads on a steel spring and measures the extension produced. For each observation, xx denotes the load (kg) and yy denotes the extension (mm). The least-squares regression line of yy on xx obtained from the 55 paired observations is y=20.5+1.95xy = 20.5 + 1.95x. An observation at x=8x=8 gave y=38y=38. What is the residual, and what does its sign tell you?
Show worked solution

Worked solution

  1. Recall the definition of a residual

    ei=yi−y^i=yi−(a+bxi)e_i=y_i-\hat{y}_i=y_i-\left(a+bx_i\right)

    A residual is the signed vertical distance from a data point to the fitted line.

  2. Find the fitted value

    y^=20.5+1.95×8=36.1\hat{y}=20.5+1.95\times8=36.1

    This is where the line says the point should be.

  3. Subtract

    e=38−36.1=1.9e=38-36.1=1.9

    Observed minus predicted, so the sign is meaningful.

  4. Interpret the sign

    e=1.9>0 ⇒ point above the linee=1.9>0\ \Rightarrow\ \text{point above the line}

    A positive residual means the model under-predicts this observation.

Answer
e=1.9, point above the linee=1.9,\ \text{point above the line}
Question 3
4 marksintermediate
A civil engineer crushes concrete cubes that have been cured for different numbers of days. For each observation, xx denotes the curing time (days) and yy denotes the compressive strength (newtons per square millimetre). The data are coded using p=x−64p=\frac{x-6}{4} and q=y−253q=\frac{y-25}{3}, and the regression line of qq on pp is found to have gradient bqp=1.6b_{qp}=1.6. What is the gradient of the regression line of yy on xx?
Show worked solution

Worked solution

  1. Write xx and yy in terms of pp and qq

    x=6+4p,y=25+3qx=6+4p,\qquad y=25+3q

    Inverting the coding is the quickest route to the effect on the gradient.

  2. Differentiate, or take the ratio of changes

    ΔyΔx=3 Δq4 Δp=34×ΔqΔp\frac{\Delta y}{\Delta x}=\frac{3\,\Delta q}{4\,\Delta p}=\frac{3}{4}\times\frac{\Delta q}{\Delta p}

    A unit change in pp is 44 units of xx, and a unit change in qq is 33 units of yy.

  3. Recall the effect of coding on a regression line

    p=x−hk, q=y−ml  ⇒  byx=lk bqpp=\frac{x-h}{k},\ q=\frac{y-m}{l}\;\Rightarrow\;b_{yx}=\frac{l}{k}\,b_{qp}

    Coding rescales the axes, so the gradient is multiplied by the ratio of the two scale factors.

  4. Substitute the coded gradient

    byx=34×1.6b_{yx}=\frac{3}{4}\times1.6

    The translations hh and mm do not affect the gradient at all; only the scale factors do.

  5. Evaluate

    byx=1.2b_{yx}=1.2

    Multiplying by kl\frac{k}{l} instead is the standard error here.

  6. Recall the least-squares summary statistics

    Sxx=∑x2−(∑x)2n,Sxy=∑xy−(∑x)(∑y)nS_{xx}=\sum x^2-\frac{\left(\sum x\right)^2}{n},\qquad S_{xy}=\sum xy-\frac{\left(\sum x\right)\left(\sum y\right)}{n}

    Everything in linear regression is built from these two sums of squares.

  7. State the gradient

    byx=1.2b_{yx}=1.2

    Coding is only a change of units; the underlying fit is unchanged.

Answer
byx=1.2b_{yx}=1.2
Question 4
6 markshard
A manufacturer records the annual maintenance cost of a machine against the number of hours it has been run. For each observation, xx denotes the running time (hundreds of hours) and yy denotes the maintenance cost (hundreds of pounds). The regression line of yy on xx has gradient byx=0.69b_{yx}=0.69 and the regression line of xx on yy has gradient bxy=1.38b_{xy}=1.38. Which of the following statements is correct?
Show worked solution

Worked solution

  1. Recall the link between the two regression lines

    byx bxy=SxySxx×SxySyy=Sxy2SxxSyy=r2b_{yx}\,b_{xy}=\frac{S_{xy}}{S_{xx}}\times\frac{S_{xy}}{S_{yy}}=\frac{S_{xy}^{2}}{S_{xx}S_{yy}}=r^{2}

    The product of the two gradients is the square of the product-moment correlation coefficient.

  2. Form the product of the two gradients

    byxbxy=(0.69)(1.38)=0.9522b_{yx}b_{xy}=\left(0.69\right)\left(1.38\right)=0.9522

    The two SxyS_{xy} factors multiply to give Sxy2S_{xy}^{2}.

  3. Identify the product

    Sxy2SxxSyy=r2\frac{S_{xy}^{2}}{S_{xx}S_{yy}}=r^{2}

    This is precisely the square of the product-moment correlation coefficient.

  4. Take the square root

    r=±0.9522≈±0.976r=\pm\sqrt{0.9522}\approx\pm0.976

    The square root leaves the sign undetermined.

  5. Fix the sign of rr from the sign of the gradients

    byx>0 ⇒ r>0b_{yx}>0\ \Rightarrow\ r>0

    rr, byxb_{yx}, bxyb_{xy} and SxyS_{xy} all carry the same sign.

  6. Check that r2≤1r^{2}\le1

    0.9522≤10.9522\le1

    The two gradients can therefore never both exceed 11 in modulus.

  7. Recall the proportion of variation explained

    r2=1−RSSSyyr^{2}=1-\frac{\text{RSS}}{S_{yy}}

    r2r^2 is the fraction of the variation in yy that the regression on xx accounts for.

  8. Recall the least-squares summary statistics

    Sxx=∑x2−(∑x)2n,Sxy=∑xy−(∑x)(∑y)nS_{xx}=\sum x^2-\frac{\left(\sum x\right)^2}{n},\qquad S_{xy}=\sum xy-\frac{\left(\sum x\right)\left(\sum y\right)}{n}

    Everything in linear regression is built from these two sums of squares.

  9. Recall the least-squares gradient

    b=SxySxxb=\frac{S_{xy}}{S_{xx}}

    Setting the partial derivatives of the residual sum of squares to zero gives this value of bb.

  10. Recall the least-squares intercept

    a=yˉ−bxˉa=\bar{y}-b\bar{x}

    The fitted line always passes through the mean point (xˉ,yˉ)\left(\bar{x},\bar{y}\right).

  11. State the relationship

    byxbxy=r2=0.9522,r≈0.976b_{yx}b_{xy}=r^{2}=0.9522,\qquad r\approx0.976

    The product of the gradients of the two regression lines is r2r^{2}, never rr itself.

Answer
byxbxy=r2=0.9522b_{yx}b_{xy}=r^{2}=0.9522
Question 5
9 markschallenging
A chemist measures the initial rate of a reaction at several catalyst concentrations. For each observation, xx denotes the catalyst concentration (mmol per litre) and yy denotes the initial rate (micromol per second). Both variables are subject to random measurement error, but in this later specimen it is yy that has been measured and xx that must be estimated. For the original 55 observations the regression line of yy on xx is y=19.25+2.15xy = 19.25 + 2.15x and the regression line of xx on yy is x=−7.9+0.43yx = -7.9 + 0.43y. A later specimen is found to have y=28y=28. Which line should be used to estimate the corresponding value of xx, and what estimate does it give?
Show worked solution

Worked solution

  1. Recall what the least-squares line minimises

    RSS=∑i=1n(yi−a−bxi)2\text{RSS}=\sum_{i=1}^{n}\left(y_i-a-bx_i\right)^2

    The vertical distances are squared, so points above and below the line both count against it.

  2. Recall the regression line of xx on yy

    x=a′+b′y,b′=SxySyy,a′=xˉ−b′yˉx=a'+b'y,\qquad b'=\frac{S_{xy}}{S_{yy}},\qquad a'=\bar{x}-b'\bar{y}

    This line minimises the squared HORIZONTAL distances, so it is the one to use when xx is the response.

  3. Decide which variable is the response here

    y is known,x is to be estimatedy\ \text{is known},\qquad x\ \text{is to be estimated}

    The quantity being estimated is the one whose deviations should be squared.

  4. Choose the line that minimises the deviations in xx

    minimise ∑(xi−a′−b′yi)2 ⇒ x on y\text{minimise}\ \sum\left(x_i-a'-b'y_i\right)^{2}\ \Rightarrow\ x\ \text{on}\ y

    That is the regression line of xx on yy.

  5. Write down the line of xx on yy

    x=−7.9+0.43yx = -7.9 + 0.43y

    Its gradient is b′=SxySyyb'=\frac{S_{xy}}{S_{yy}}, not 1b\frac{1}{b}.

  6. Substitute y=28y=28

    x^=−7.9+0.43×28=4.14\hat{x}=-7.9+0.43\times28=4.14

    This is the required estimate.

  7. See what rearranging the wrong line would have given

    x=28−19.252.15≈4.07x=\frac{28-19.25}{2.15}\approx4.07

    A genuinely different answer, because the two lines are different lines.

  8. Explain why the two disagree

    b b′=r2=0.9245<1b\,b'=r^{2}=0.9245<1

    The two lines coincide only when r2=1r^{2}=1; the weaker the correlation, the wider they open out.

  9. Confirm both lines meet at the mean point

    (xˉ,yˉ)=(5, 30)\left(\bar{x},\bar{y}\right)=\left(5,\ 30\right)

    They cross there and nowhere else.

  10. Recall the least-squares summary statistics

    Sxx=∑x2−(∑x)2n,Sxy=∑xy−(∑x)(∑y)nS_{xx}=\sum x^2-\frac{\left(\sum x\right)^2}{n},\qquad S_{xy}=\sum xy-\frac{\left(\sum x\right)\left(\sum y\right)}{n}

    Everything in linear regression is built from these two sums of squares.

  11. Recall the least-squares gradient

    b=SxySxxb=\frac{S_{xy}}{S_{xx}}

    Setting the partial derivatives of the residual sum of squares to zero gives this value of bb.

  12. Recall the least-squares intercept

    a=yˉ−bxˉa=\bar{y}-b\bar{x}

    The fitted line always passes through the mean point (xˉ,yˉ)\left(\bar{x},\bar{y}\right).

  13. Recall what the least-squares line minimises

    RSS=∑i=1n(yi−a−bxi)2\text{RSS}=\sum_{i=1}^{n}\left(y_i-a-bx_i\right)^2

    The vertical distances are squared, so points above and below the line both count against it.

  14. Recall the definition of a residual

    ei=yi−y^i=yi−(a+bxi)e_i=y_i-\hat{y}_i=y_i-\left(a+bx_i\right)

    A residual is the signed vertical distance from a data point to the fitted line.

  15. Recall the computational form of the residual sum of squares

    RSS=Syy−Sxy2Sxx\text{RSS}=S_{yy}-\frac{S_{xy}^{2}}{S_{xx}}

    Substituting the least-squares aa and bb into ∑(yi−a−bxi)2\sum\left(y_i-a-bx_i\right)^2 collapses it to this.

  16. State the line and the estimate

    x on y:x^=4.14x\ \text{on}\ y:\qquad \hat{x}=4.14

    Rearranging the line of yy on xx is the classic error and gives 4.074.07 instead.

Answer
x^=4.14\hat{x}=4.14

Unlock 65 more Linear regression questions

Create a free account to work through every Further Maths Linear regression question with instant step-by-step worked solutions, progress tracking and interactive lessons.

  • Full worked solutions for every question
  • Interactive lessons and instant feedback
  • Track your mastery across every topic
Create a Free Account

No card required · Free forever

More Linear regression practice

Hard QuestionsWorked Solutions

Related Statistics topics

Central limit theoremChi-squared testsCombinations of random variablesContinuous random variablesThe continuous uniform distributionDiscrete random variables