Picking Rows and Filling Gaps

Picking Rows and Filling Gaps: Filtering Rows with a Condition, Finding Missing Values: NaN and isna, Dropping or Filling Missing Values.

Updated

  1. Filtering Rows with a Condition (Apprentice): A comparison such as products["price"] < 100 gives a True or False answer for every row, called a mask; putting the mask in square brackets, products[mask], keeps only the rows where it is True; and sum() and mean() on the mask count the matching rows and give their share of the table.
  2. Finding Missing Values: NaN and isna (Apprentice): pandas marks a missing value as NaN, isna().sum() counts the missing values in every column, na_values in read_csv turns hidden blanks such as 'unknown' and '-' into NaN, and count and mean quietly leave missing values out.
  3. Dropping or Filling Missing Values (Apprentice): Once you know where a table's gaps are, dropna removes rows (all of them with any gap, or only those missing the columns you name in subset), fillna puts a value, a median or 'Unknown' in each gap, and a flag column records which values were filled so none is passed off as measured.