Pipes and Redirection

Hand the output of one command to the next, and send output and errors where you want them.

Updated

Plumbing

  1. Pipes: | (Apprentice): The pipe, |, hands everything one command prints to the next command as its input, so small commands that each do one job join into a pipeline that answers a question none of them can answer alone.
  2. Three Streams: Output, Errors and Input (Apprentice): Every command has three streams, output, errors and input, and redirection sends each where you choose, with 2> for errors, &> or > file 2>&1 for both together, /dev/null to throw them away and < to read a file in.

Counting and Cutting

  1. Count, Sort, Remove Duplicates: wc, sort and uniq (Apprentice): wc counts lines, words and characters, sort puts lines in order, as text or with -n as numbers, and uniq merges repeated lines that sit next to each other, so sort | uniq -c | sort -rn ranks how often each line appears.
  2. Columns and Characters: cut and tr (Apprentice): cut keeps the columns you name with -f, splitting each line at the delimiter you give with -d (or keeps character positions with -c), and tr changes characters one for one, such as capitals into lowercase, or deletes them with -d.
  3. Tee and Xargs: Splitting and Feeding (Journeyman): tee saves what flows through a pipe into a file and passes it on, so you can keep a report and read it at once, and xargs turns the lines it reads into arguments for commands such as mkdir and rm, which don't read a pipe.

Exercise: The Bakery's Orders

  1. Exercise: The Bakery's Orders (Apprentice): Answer a bakery owner's questions about a year of orders with pipelines, then rank the cities with tee, build a lowercase mailing list, keep separate logs of results and errors, and delete only the canceled receipts with xargs, while a check script tells you what's done.