Tools of the Trade

I’ve recently made an introductory post on what knowing some bash can do for automating many system tasks. It showed some of what’s possible, but didn’t describe the imperative nature of having the proper ancillary tools in your environment that are crucial to writing quality bash. Beyond your text editor of choice (come to the dark side and use vim or neovim already if you haven’t), you’re going to want these tools.

Shellcheck

You’re going to want a code linter for parsing your scripts for poor choices. For this, shellcheck is the gold standard. It doesn’t just catch bugs, but also calls out poor stylistic code and code that may possibly lead to errors under certain use cases. It will call out many things that are easily missed, even by fairly well seasoned bash scripters. It also works with other shells, such as Korn shell, POSIX shell, and dash. Most linux distributions have shellcheck in their package repositories, so just grab it from there. To use it, just run shellcheck my_bash_script.sh. A particularly salient use case for shellcheck is in CI/CD pipelines to protect your code base from having low quality shell scripts getting pushed to it.

BATS

There are too many people who don’t see the importance of having a unit testing suite for bash. For this purpose, the Bash Automated Testing System (BATS) will serve us very well. We initialize a bats file, which is a bash script that has unique syntax for the purposes of defining test cases. In BATS, a test is a function with an associated description. A basic sample BATS test for a generic hello world function would look as follows:

#!/usr/bin/env bats

@test "Basic function should return 'Hello World'" {
   source $HOME/hello_world.sh 
   run hello
   [ "$output" == "Hello World!" ]
}

Here, hello is a function in the script hello_world.sh that echos “Hello World!”. To run a test, just execute bats test_file.bats in the terminal. If we want more verbose output in TAP format, we run bats with the --tap flag. The command run is used to run a function and a set of commands its given more broadly and then stores the exit status and the output in the variables result and output respectively. If we want to skip a particular test, we can insert the word skip along with an optional comment to let BATS know we want to ignore it. We can also push text to the TAP output via the reserved file descriptor &3. In my test function, I could write this:

echo 'This is some text' >&3

While the BATS core is powerful, albeit fairly limited in functionality, it also has a number of optional libraries that can be downloaded by the user. We’ll make a simple shell script to download and move them to the proper:

#!/bin/sh

mkdir -p "$HOME"/dl-dir

git clone https://github.com/bats-core/bats-support "$HOME"/dl-dir/bats-support
git clone https://github.com/bats-core/bats-assert "$HOME"/dl-dir/bats-assert
git clone https://github.com/bats-core/bats-file "$HOME"/dl-dir/bats-file

sudo cp -r "$HOME"/dl-dir/. /opt/bats-libs/

We can use these libraries in our test scripts by adding the follow lines at the top of the script

#!/usr/bin/env bats

load '/opt/bats-libs/bats-support/load.bash'
load '/opt/bats-libs/bats-assert/load.bash'
load '/opt/bats-libs/bats-file/load.bash'

A rudimentary example of our newly acquired tools like assert looks like this:

#!/usr/bin/env bats

load '/opt/bats-libs/bats-support/load.bash'
load '/opt/bats-libs/bats-assert/load.bash'

@test "An addition test for thee" {
    assert_equal $(echo 2+3 | bc) 5 # Give a command or function and then a space separated expected value
}

@test "Existential test" {
    mkdir -p ~/test
    rmdir ~/test
    refute [ -e '~/test' ] # This has a value of true as long as the directory doesn't exist
}

@test "Simple verification" {
    local a=2
    local b=3
    assert [ "$a" -lt "$b" ]
}

@test "Did the script execute without error?" {
    run test_func
    assert_success # Returns false if the exit code isn't 0
}

@test "Did the script fail?" {
    run test_func2
    assert_failure # Returns false if the exit code is 0
    assert_output -p "Function failed!" # We're giving an output message on failure 
}

Some Other Tools

While shellcheck and BATS can handle most stuff, there are a few other techniques and tools that can be used to debug bash scripts and ensure you’re writing good code. First, you can add set -x to any script to have a line by line print of every command as it’s being executed. You can combine this with the e flag as well to have that script halt when a line fails.