Unit Testing for Shell Scripts

Unit testing for shell scripts

UPDATE 2019-03-01: My preference is bats now. I have used it for a few years on small projects. I like the clean, concise syntax. I have not integrated it with CI/CD frameworks, but its exit status does reflect the overall success/failure of the suite, which is better than shunit2 as described below.

---

PREVIOUS ANSWER:

I'm using shunit2 for shell scripts related to a Java/Ruby web application in a Linux environment. It's been easy to use, and not a big departure from other xUnit frameworks.

I have not tried integrating with CruiseControl or Hudson/Jenkins, but in implementing continuous integration via other means I've encountered these issues:

• Exit status: When a test suite fails, shunit2 does not use a nonzero exit status to communicate the failure. So you either have to parse the shunit2 output to determine pass/fail of a suite, or change shunit2 to behave as some continuous integration frameworks expect, communicating pass/fail via exit status.
• XML logs: shunit2 does not produce a JUnit-style XML log of results.

Unit testing Bash scripts

I got the following answer from a discussion group:

it's possible to import (include,
whatever) a procedure (function,
whatever it's named) from an external
file. That's the key to writing a
testing script: you break up your
script into independent procedures
that can then be imported into both
your running script and your testing
script, and then you have your running
script be as simple as possible.

This method is like dependency injection for scripts and sounds reasonable. Avoiding Bash scripts and using more testable and less obscure language is preferable.

Shell Script unit testing: How to mockup a complex utility program

Cargill's quandary:

" Any design problem can be solved by adding an additional level of indirection, except for too many levels of indirection."

Why mock system commands ? After all if you are programming Bash, the system is your target goal and you should evaluate your script using the system.

Unit test, as the name suggests, will give you a confidence in a unitary part of the system you are designing. So you will have to define what is your unit in the case of a bash script. A function ? A script file ? A command ?

Given you want to define the unit as a function I would then suggest writing a list of well known errors as you listed above:

• Special characters in file or directory names
• Problems with quoting or encodings
• Missing ssh keys
• Wrong permissions and so on.

And write a test case for it. And try to not deviate from the system commands, since they are integral part of the system you are delivering.

Unit testing of shell commands called in a python script

Your question combines two interesting topics: a) Testing code generated from a code generator and b) testing shell code.

For testing code from a generator, you have in principle to do the following: i) test that the generator creates the code that is expected - which you have already done, ii) test that the code snippets / pieces which the generator glues together actually behave (independently and in combination) as it is intended (which in your case are the shell code pieces that in the end together will form a valid shell program) - this is the part about testing shell code that will be adressed below, and iii) test that the inputs that control the generator are correct.

It is comparable with a compiler: i) is the compiler code, ii) are assembly code snippets that the compiler combines to get the resulting assembly program, and iii) is the source code that is given to the compiler to get it compiled. Once i), ii) and iii) are tested, there is only seldom the need to also test the assembly code (that is, on assembly code level). In particular, the source code iii) is ideally tested by test frameworks in the same programming language.

In your case it is not so clear how part iii) looks and how it can be tested, though.

Regarding the testing of shell code / shell code snippets: Shell code is dominated by interactions with other executables or the operating system. The type of problems that lies in interactions in shell code goes in the direction of, am I calling the right executables in the right order with the arguments in the right order with properly formatted argument values, and are the outputs in the form I expect them to be etc. To test all this, you should not apply unit-testing, but integration testing instead.

In your case this means that the shell code snippets should be integration-tested on the different target operating systems (that is, not isolated from the operating system). And, the various ways in which the generator puts these snippets together should also be integration tested to see if they together operate nicely on the operating system.

However, there can be shell code that is suitable for unit-testing. This is, for example, code performing computations within the shell, or string manipulations. I would even consider shell code with calls to certain fundamental tools like basename as suitable for unit-testing (interpreting such tools as being part of the 'standard library' if you like). In your case as you describe the generated shell code

creates a filename from the function arguments.

This sounds like one example of a good candidate for 'unit-testing' shell code: This filename creation functionality could be put into a shell function and then be tested in isolation.

unit test bash script function which deletes files older than certain number of days

From my perspective, you are asking three separate questions:

1. Is my code any good?
2. How do I write test in general for BASH
3. How do I test this specific code?

As that sounds more like a request for code review, it might be better suited to https://codereview.stackexchange.com/ but I'll answer here anyway...

The command isn't really that complex. But even if it were, you'd be testing the side-effect of the code, not the code itself. So complexity in the code doesn't even really matter...

Anyway, a test would look something like this:

@test "deleteFilesOlderThan deletes files" {
  # Arrange
  touch -t 123412312345 ./test-files/test.txt

  # Act
  deleteFilesOlderThan 1000

  # Assert
  [ ! -f ./test-files/test.txt ]
}

You could add more tests, for instance checking the output using assert_output, and checking that newer files do not get deleted.

The code can be tested without being rewritten but there are some potential problems in the code:

• As state in the comments, the eval is not really needed. The find command can run fine as-is, without being wrapped in an eval.

• There are no checks. None. At all. You might want to at least check that $1 is actually provided. You could also check whether it is an integer or not.

• You could check that test-files actually exists

• The test-files directory is hard-coded. I would make that a parameter of the function. That way it can be provided with a different path for the test than that used for real.

These changes could look something like this:

function deleteFilesOlderThan() {
  local days="${1:?Two parameters required: <days> <path>}"
  local path="${2:?Two parameters required: <days> <path>}"
  
  if [[ -n ${days} && ${days} = *[!0123456789]* ]]; then
    echo "ERROR: Given days '${days}' is not an integer" >&2
  elif [[ ! -d "${path}" ]]; then
    echo "ERROR: Given path '${path}' is not a directory" >&2  
  else
    echo "Deleting files older than ${1} days in ${path}"

    find "${path}" -mtime "+${1}" -exec rm {} \;
  fi
}

Of course, now that there is more code, there should also be more tests. I'll leave that as an exercise for the reader.

---

If you are not already familiar with it, you might want to check out shellcheck. It will warn you if you write any code that might cause problems.

You might also want to look at shfmt (from the mvdan.cc/sh package) to formats shell script.

---

Related Topics