Strict Mode: set -euo pipefail

set -euo pipefail makes a script stop at the first sign of trouble: -e stops at a command that fails, -u at a variable that was never set, and -o pipefail at a pipeline with a failing part, with a few surprises to know about: a grep that finds nothing, ((count++)) and a missing $1.

Freelancer · Unix · Linux · Updated

After this lesson you can

  • Turn on strict mode with set -euo pipefail, and say what -e, -u and -o pipefail each do
  • Work around strict mode's surprises in a script, such as a grep that finds nothing, ((count++)) and a missing $1

Key takeaways

  • -e stops the script at the first command that fails, instead of carrying on in the wrong folder or with the wrong data.
  • -u makes a variable that was never set an error, so a typo such as $photo_dri stops the script instead of becoming nothing.
  • -o pipefail makes a pipeline fail when any part of it fails, not only the last command.
  • The surprises: grep that finds nothing fails, ((count++)) fails when count is 0, and "$1" fails when there's no argument. Write || true, count=$((count + 1)) and "${1:-}".
  • Strict mode is for scripts. A failing test after if, or a command before || or &&, doesn't stop the script.

A script runs every line you give it, even after one has failed. That's like a cook who finds the oven broken and carries on, putting the cake mix in a cold oven and setting the timer anyway. If the failed line was a cd, every line after it works in the wrong folder; if it was a typo in a variable's name, the script works on nothing at all. Careful scripts start with one line that changes this, set -euo pipefail, often called strict mode: stop at the first sign of trouble. Administrators put it at the top of deployment and backup scripts, where carrying on after a failure does real damage.

The examples are small scripts in the mover practice folder from Errors a User Can Act On. Make each with nano, chmod +x it, and run it.

-e: stop at the first failure

tidy.sh is meant to tidy the inbox, but the folder's name is mistyped:

#!/usr/bin/env bash
cd inbx
echo "Tidying $(pwd)"

./tidy.sh
echo $?

./tidy.sh: line 2: cd: inbx: No such file or directory
Tidying /home/sam/terminal-practice/mover
0

cd failed, and the script carried on in the wrong folder, then reported success. If the next line deleted old files, it would delete them there. Add set -e (read "set, exit on error") as the second line, and run it again:

./tidy.sh: line 3: cd: inbx: No such file or directory
1

It stopped at the failure, and its exit code says so.

-u: a misspelled variable is an error

In where.sh, the variable is photo_dir but the ls line says photo_dri:

#!/usr/bin/env bash
photo_dir=photos
ls "$photo_dri/"

./where.sh

bin   dev  home  lib    lost+found  mnt  proc  run   snap  sys  usr
boot  etc  init  lib64  media       opt  root  sbin  srv   tmp  var

A variable that was never set counts as empty, so "$photo_dri/" became /, and ls listed the top of the whole computer. With rm in place of ls, that's how scripts have deleted far more than they meant to. Add set -u ("unset variables are errors") as the second line:

./where.sh: line 4: photo_dri: unbound variable

Unbound is bash's word for "never set".

-o pipefail: every part of a pipe counts

A pipeline's exit code is normally its last command's. In waiting.sh, with set -e already on, ls fails, but wc counts nothing happily:

#!/usr/bin/env bash
set -e
waiting=$(ls inbx | wc -l)
echo "$waiting photos are waiting."

./waiting.sh

ls: cannot access 'inbx': No such file or directory
0 photos are waiting.

Change the second line to set -eo pipefail (read "set option pipefail"), and the pipeline fails when any part of it fails: the script stops after ls's message, with exit code 2, ls's own. All three together are written set -euo pipefail, on the line after the comment at the top of a script.

The surprises

Strict mode is strict about some things that aren't mistakes:

  • A grep that finds nothing gives exit code 1, so pngs=$(ls inbox | grep -c png) stops the script silently when there are no PNG files. Add || true (read "or else, carry on"): pngs=$(ls inbox | grep -c png || true) prints 0 PNG files.
  • ((count++)), a shortcut for adding 1 that you'll see in other people's scripts, gives exit code 1 when count was 0, and the script stops silently after the first item. Write count=$((count + 1)), which always works.
  • "$1" with no arguments is an unset variable. Here's mover.sh with strict mode on line 3, run with nothing:
./mover.sh: line 11: $1: unbound variable

The usage message never got its chance. Write "${1:-}" on that line (read "the value of 1, or nothing if there isn't one"):

if [[ "${1:-}" == "--help" ]]; then

./mover.sh
echo $?

Usage: mover.sh FROM-FOLDER TO-FOLDER
2

Two things don't stop a strict script, on purpose: a failing test after if or while, and a command before || or &&. That's why [[ -d "$src" ]] || die "..." still works. And keep strict mode in scripts: typed at your prompt, set -e closes the terminal at your first typo.

Check yourself

2 questions to try on the lesson page.

So what

set -euo pipefail turns "carry on and hope" into "stop and say why", and knowing its three surprises means it never stops a script that was doing the right thing.

Key terms

Cheat sheets

Related