# Trees and Neighbors

Two more ways to classify, each easy to picture: a decision tree that plays twenty questions with your data, and nearest neighbors, which looks for the most similar examples it has seen and lets them vote. You'll read a tree's questions in plain words, measure distance between examples by hand, and see why the units of each feature matter. The running example is a fruit packing line.

Updated: November 19, 2026

## [Decision Trees](https://www.ask-codey.com/courses/machine-learning-python/topics/ml-decision-trees-topic)

1. [Decision Trees](https://www.ask-codey.com/lessons/ml-decision-trees) (Journeyman): A decision tree sorts examples into classes with yes-or-no questions about their features, picking each question so the groups it makes are as unmixed as possible (the Gini impurity), with max_depth to limit how many questions it asks and no need to rescale the features first.
2. [Reading a Decision Tree](https://www.ask-codey.com/lessons/ml-reading-a-decision-tree) (Journeyman): A decision tree's rules can be printed with export_text or drawn with plot_tree, one fruit's guess can be said as a sentence by following its path from the top question to a leaf, and the leaf's mix of training fruits is the probability the tree gives, but a deep tree's rules stop being readable and every rule describes the training data, not the world.

## [Nearest Neighbors](https://www.ask-codey.com/courses/machine-learning-python/topics/ml-nearest-neighbors-topic)

1. [Distance Between Examples](https://www.ask-codey.com/lessons/ml-distance-between-examples) (Apprentice): Two examples become points when each feature is a direction, the straight-line (Euclidean) distance between them comes from the Pythagorean theorem and math.dist works it out for any number of features, and a feature measured in big units, like grams, swamps one scored from 0 to 10 until the features are put on a common scale.
2. [Nearest Neighbors](https://www.ask-codey.com/lessons/ml-nearest-neighbors) (Journeyman): A nearest neighbors classifier guesses a new example's class by finding the k training examples closest to it and letting them vote, with KNeighborsClassifier(n_neighbors=5) storing the training rows instead of learning a rule, a small k following every quirk, a large k blurring the classes together, and a tied vote, which in the lesson's run went to the class listed first.

## [Exercise: Fruit Sorter](https://www.ask-codey.com/courses/machine-learning-python/topics/exercise-ml-fruit-sorter)

1. [Exercise: Fruit Sorter](https://www.ask-codey.com/lessons/exercise-ml-fruit-sorter) (Journeyman): Write four functions for Three Rivers Fruit Packers, a straight-line distance and a nearest-neighbors search in plain Python, then a decision tree and a nearest neighbors classifier with the settings you choose, and let pytest check the distances, the neighbors' order and the models' settings.

Source: https://www.ask-codey.com/courses/machine-learning-python/chapters/ml-trees-and-neighbors
