Ordinary Least Squares

noun

say it: “OR-dih-nair-ee LEEST SKWAIRZ” /ˈɔrdəˌnɛri ˈlist ˈskwɛrz/

A method for fitting a straight line to data by making the sum of the squared differences between the data and the line as small as possible.

What is ordinary least squares?

Ordinary least squares (OLS) fits a line by choosing the one whose squared errors (each actual value minus the guess, squared) add up to the least. Squaring makes every miss count as positive and big misses count much more. scikit-learn's LinearRegression fits this way.

cups = [40, 65, 80]
line_a = [45, 60, 82]
line_b = [42, 62, 79]
print(sum((c - g) ** 2 for c, g in zip(cups, line_a)))   # 54
print(sum((c - g) ** 2 for c, g in zip(cups, line_b)))   # 14: OLS prefers this line

In a sentence

  • “Ordinary least squares chose the line with the smallest total of squared misses.”
  • “LinearRegression fits its line by ordinary least squares.”

Level: Journeyman · More AI Basics terms at this level

Updated