How it works
Simple linear regression fits a line y = a + bx. Least squares picks a and b to minimize the sum of squared residuals. The coefficient b tells how much y changes on average when x rises by one unit, and R² tells what share of y's variation the model explains. The name comes from Galton, who saw children of tall parents "regress toward the mean" in height.
Types and pitfalls
Multiple regression estimates effects while controlling for other factors; logistic regression handles yes/no outcomes. A significant coefficient doesn't establish causation; watch for omitted variables, multicollinearity, outliers, and extrapolation. Residual spread is summarized by the standard deviation. Regression is also a foundation of machine learning.
FAQ
- Is a high R² always good?
- Not necessarily—adding variables inflates it, so check adjusted R².
- Can I run it in Excel?
- Yes, with the Analysis ToolPak or LINEST.
Sources reviewed: 2026-10-26
References
- Galton, F. (1886). Regression towards mediocrity in hereditary stature. Journal of the Anthropological Institute, 15, 246–263.
- James, G., Witten, D., Hastie, T., & Tibshirani, R. (2021). An Introduction to Statistical Learning (2nd ed.). Springer.
- Wooldridge, J. M. (2019). Introductory Econometrics (7th ed.). Cengage.