Learning Outcome Statement
Describe, interpret, and explain simple linear regression, including coefficient estimation using the least squares criterion.
On the exam: The question gives paired periodic observations in a table and asks you to estimate the least-squares intercept and slope, identify the fitted line, or interpret a coefficient in a fill-in, multiple-choice, or ordered-response format.
Simple linear regression uses a single independent variable to explain a dependent variable with a straight line. In pair $i$, $x_i$ is the independent observation and $y_i$ is the dependent observation. The line turns $x_i$ into a fitted value, $\hat{y}_i$, and the residual, $e_i$, is actual $y_i$ minus that fitted value. Squaring each residual stops positive and negative misses from canceling and gives larger misses more weight.
| Symbol | Definition | Unit |
|---|---|---|
| $x_i$ | independent observation | $x$ units |
| $y_i$ | dependent observation | $y$ units |
| $\hat{y}_i$ | fitted value from line | $y$ units |
| $e_i$ | actual minus fitted | $y$ units |
| $\bar{x}$ | sample mean of $x$ | $x$ units |
| $\bar{y}$ | sample mean of $y$ | $y$ units |
| $n$ | paired observations | observations |
| $\hat{\beta}_0$ | fitted intercept | $y$ units |
| $\hat{\beta}_1$ | fitted slope | $y$ units per $x$ unit |
The sample means provide the center of the observations, so centering separates the line’s tilt from its location. The coefficient units follow from the ratio in the slope and the direct level in the intercept.
Least squares chooses the intercept and slope that minimize the sum of squared residuals. With $\mathrm{SSE}$ naming that sum, the fitted line and coefficient estimates are:
$$ \begin{aligned} \hat{y}_i &= \hat{\beta}0+\hat{\beta}1x_i \ \mathrm{SSE} &= \sum{i=1}^{n}e_i^2=\sum{i=1}^{n}(y_i-\hat{y}i)^2 \ \hat{\beta}1 &= \frac{\sum{i=1}^{n}(x_i-\bar{x})(y_i-\bar{y})}{\sum{i=1}^{n}(x_i-\bar{x})^2} \ \hat{\beta}_0 &= \bar{y}-\hat{\beta}_1\bar{x} \end{aligned} $$
The numerator of the slope rewards pairs whose deviations from their means move in the same direction. Opposite deviations pull the slope down. The denominator records how widely $x$ moves, so more centered spread in $x$ makes the same co-movement produce a smaller slope. The intercept places the line so it passes through $(\bar{x},\bar{y})$, and it is fitted $y$ when $x$ equals zero.
Use this sequence when the table gives paired observations. Before calculating, keep the pairs intact: each $x_i$ belongs with its observed $y_i$. Find both sample means, then measure each observation relative to its own mean. Those centered deviations reveal whether the variables move together and how much independent-variable spread supports the fitted tilt. Once the slope is known, the mean point fixes the intercept and gives you a check on the line.
Loading diagram…
The geometry is simple: observed points sit away from a fitted line, and residual arrows measure the gaps. Each arrow is a vertical difference at a particular observed $x$, measured from actual $y$ to fitted $\hat{y}$. A point above the line has a positive residual; one below has a negative residual. Least squares squares these signed gaps before adding them, so neither direction cancels the other and a larger miss contributes more.
Worked example
You are handed paired monthly observations from a market excess-return series and a stock excess-return series. $x$ is the market excess return and $y$ is the stock excess return, both measured in percentage points per month. Treat the observations as sample data, keep the monthly frequency, and do not annualize the inputs. You need the least-squares intercept, slope, and fitted line.
| Given | Value |
|---|---|
| $n$ | $5$ paired observations |
| $x$ | $-2.0000, -1.0000, 1.0000, 2.0000, 3.0000$ percentage points per month |
| $y$ | $-1.0000, 0.0000, 2.0000, 1.0000, 4.0000$ percentage points per month |
| Frequency | Monthly |
| Data treatment | Sample data, no annualizing |
The entries in the $x$ and $y$ rows pair by position. The observed series is $y$, and the fitted series is $\hat{y}$ at the same $x$ values. Keep the units and frequency attached to both series throughout the calculation. The diagram gives a quick visual comparison: the first line traces the observed returns, while the second shows the candidate fitted values across the same ordered market returns. Vertical separation at any position is the residual for that pair. This makes the sign and size of each miss easier to inspect before working through the coefficient arithmetic.
Loading diagram…
Read the plotted series at each shared $x$ value rather than comparing points across different positions. The observed line rises and falls with the sample, while the fitted line follows the constant slope implied by the estimated coefficients. For example, the fitted value at the leftmost market return is below the observed return, and at the final return it is above. The following rows make those pairwise comparisons exact and let you check the fitted line numerically.
| $x$ | Observed $y$ | Fitted $\hat{y}$ |
|---|---|---|
| $-2.0000$ | $-1.0000$ | $-4.0000$ |
| $-1.0000$ | $0.0000$ | $-2.0000$ |
| $1.0000$ | $2.0000$ | $2.0000$ |
| $2.0000$ | $1.0000$ | $4.0000$ |
| $3.0000$ | $4.0000$ | $6.0000$ |
-
Average each column to anchor the line at the sample center. The totals are $3.0000$ and $6.0000$, so $\bar{x}=\frac{3.0000}{5}=0.6000$ and $\bar{y}=\frac{6.0000}{5}=1.2000$.
-
Subtract each mean before multiplying, because least squares uses deviations rather than raw levels. The centered cross-product sum is $14.4000$, and the centered $x$-deviation square sum is $7.2000$.
-
Divide centered co-movement by centered $x$ spread to get the slope. $\hat{\beta}_1=\frac{14.4000}{7.2000}=2.0000$.
-
Use the mean point to anchor the line and obtain the intercept. $\hat{\beta}_0=1.2000-2.0000\times0.6000=0.0000$.
-
Insert both coefficients to identify the fitted line. $\hat{y}=0.0000+2.0000x$.
The fitted line is $\hat{y}=0.0000+2.0000x$. For the decision, read the slope as a fitted change in $y$ for a one-unit change in $x$, and read the intercept as fitted $y$ when $x$ equals zero.
Changed condition
The same monthly setup changes only the last observed $y$. Before looking at the new coefficients, predict a higher slope. The changed observation sits at the largest positive $x$, so raising its $y$ strengthens centered co-movement while leaving the centered $x$ spread unchanged.
| Quantity | Before | After |
|---|---|---|
| Last $y$ | $4.0000$ | $5.0000$ |
| $\hat{\beta}_1$ | $2.0000$ | $2.3333$ |
| $\hat{\beta}_0$ | $0.0000$ | $-0.0000$ |
The prediction is correct. The slope rises, while the intercept stays at the sample-center anchor because the fitted line passes through the sample means.
Before moving on, redo the centered numerator and denominator yourself.
Common error
A candidate may calculate the slope from uncentered products or divide by the number of observations. It feels efficient because that resembles a covariance or variance shortcut built from the table’s raw entries. Both variables have nonzero means, so the deviations must be centered around their sample means before the products are summed, and the centered sums are not divided by the number of observations.
Calculator
Enter each paired observation in the data editor before asking the calculator for the slope. The intercept still comes from the mean-anchor formula.
| Keys | Display |
|---|---|
| 2ND CLR WORK | 0.0000 |
| 2ND DATA | 0.0000 |
| -2.0000 ENTER DOWN -1.0000 ENTER | -1.0000 |
| -1.0000 ENTER DOWN 0.0000 ENTER | 0.0000 |
| 1.0000 ENTER DOWN 2.0000 ENTER | 2.0000 |
| 2.0000 ENTER DOWN 1.0000 ENTER | 1.0000 |
| 3.0000 ENTER DOWN 4.0000 ENTER | 4.0000 |
| 2ND STAT LIN | 0.0000 |
| CPT b | 2.0000 |
Check yourself
Practice
Which interpretation matches the fitted slope of 2.0000 in the worked example?
Put the coefficient-estimation steps in order.
- Pair each x with its y
- Compute sample means
- Center deviations and form sums
- Divide centered sums for slope
- Use means for intercept
- Interpret fitted coefficients
Revision snapshot
Least squares chooses the intercept and slope that minimize the sum of squared residuals. Use centered observations:
$$ \hat{\beta}1=\frac{\sum{i=1}^{n}(x_i-\bar{x})(y_i-\bar{y})}{\sum_{i=1}^{n}(x_i-\bar{x})^2} \qquad \hat{\beta}_0=\bar{y}-\hat{\beta}_1\bar{x} $$
The slope is the fitted change in $y$ for a one-unit change in $x$. The intercept is fitted $y$ when $x$ equals zero. Raw products or division by the number of observations are the trap when both means are nonzero. On the exam, keep the paired periodic inputs in their stated frequency, estimate the line, and interpret the coefficient in its units.
Unlock the full lesson and continue reading
Signup to continue reading this lesson and unlimited access questions, flashcards, AI notes, and more