Why The Unit Test Frameworks in Fortran Rely on Ruby Instead of Fortran Itself

Why the unit test frameworks in Fortran rely on Ruby instead of Fortran itself?

Writing a unit testing framework with the complexity, capability and dynamic nature of a true XUnit clone would be utterly horrendous in Fortran which is really designed for numerical analysis.

Basic unit testing for Fortran in a locked-down environment

In my opinion the assert mechanisms are not the main concern for unit tests of Fortran. As mentioned in the answer you linked, there exist several unit test frameworks for Fortran, like funit and FRUIT.

However, I think, the main issue is the resolution of dependencies. You might have a huge project with many interdependent modules, and your test should cover one of the modules using many others. Thus, you need to find these dependencies and build the unit test accordingly. Everything boils down to compiling executables, and the advantages of assertions are very limited, as you anyway will need to define your tests and do the comparisons yourself.

We are building our Fortran applications with Waf, which comes with a unit testing utility itself. Now, I don't know if this would be possible for you to use, but the only requirement is Python, which should be available on almost any platform. One shortcoming is, that the tests rely on a return code, which is not easily obtained from Fortran, at least not in a portable way before Fortran 2008, which recommends to provide stop codes in the return code. So I modified the checking for success in our projects. Instead of checking the return code, I expect the test to write some string, and check for that in the output:

    def summary(bld):
            """
            Get the test results from last line of output::

                    Fortran applications can not return arbitrary return codes in
                    a standarized way, instead we use the last line of output to
                    decide the outcome of a test: It has to state "PASSED" to count
                    as a successful test.

                    Otherwise it is considered as a failed test. Non-Zero return codes
                    that might still happen are also considered as failures.

                    Display an execution summary:

                    def build(bld):
                            bld(features='cxx cxxprogram test', source='main.c', target='app')
                            from waflib.extras import utest_results
                            bld.add_post_fun(utest_results.summary)
            """
            from waflib import Logs
            import sys

            lst = getattr(bld, 'utest_results', [])

            # Check for the PASSED keyword in the last line of stdout, to
            # decide on the actual success/failure of the test.
            nlst = []
            for (f, code, out, err) in lst:
                    ncode = code
                    if not code:
                            if sys.version_info[0] > 2:
                                    lines = out.decode('ascii').splitlines()
                            else:
                                    lines = out.splitlines()
                            if lines:
                                    ncode = lines[-1].strip() != 'PASSED'
                            else:
                                    ncode = True
                    nlst.append([f, ncode, out, err])
            lst = nlst

Also I add tests by convention, in the build script just a directory has to be provided and all files within that directory ending with _test.f90 will be assumed to be unit tests and we will try to build and run them:

def utests(bld, use, path='utests'):
    """
    Define the unit tests from the programs found in the utests directory.
    """
    from waflib import Options
    for utest in bld.path.ant_glob(path + '/*_test.f90'):
            nprocs = search_procs_in_file(utest.abspath())
            if int(nprocs) > 0:
                    bld(
                        features = 'fc fcprogram test',
                        source = utest,
                        use = use,
                        ut_exec = [Options.options.mpicmd, '-n', nprocs,
                                   utest.change_ext('').abspath()],
                        target = utest.change_ext(''))
            else:
                    bld(
                        features = 'fc fcprogram test',
                        source = utest,
                        use = use,
                        target = utest.change_ext(''))

You can find unit tests defined like that in the Aotus library. Which are utilized in the wscript via:

from waflib.extras import utest_results
utest_results.utests(bld, 'aotus')

It is then also possible to build only subsets from the unit tests, for example by running

./waf build --target=aot_table_test

in Aotus. Our testcoverage is a little meagre, but I think this infrastructure fairs actually pretty well. A test can simply make use of all the modules in the project and it can be easily compiled without further ado.

Now I don't know wether this is suitable for you or not, but I would think more about the integration of your tests in your build environment, than about the assertion stuff. It definitely is a good idea to have a test routine in each module, which then can be easily called from a test program. I would try to aim for one executable per module you want to test, where each of these modules can of course contain several tests.

How to unit test functions whose interfaces are defined within submodules

I have little experience with submodules (so not sure if this is useful), but just to extend my comment above...

!! parent_mod.f90 (public things)
module parent_mod
    implicit none

    type myint_t
        integer :: n = 100
    contains
        procedure :: add, show
    endtype

    interface
        module subroutine add( x )
            class(myint_t) :: x
        end
        module subroutine show( x )
            class(myint_t) :: x
        end
    endinterface
end

!! parent_helper.f90 (private things to be used by parent_impl.f90
!! and possibly by unit tests)
module parent_helper
    use parent_mod, only: myint_t
    implicit none
contains
    subroutine debug_show( x )
        type(myint_t) :: x
        print *, "debug: x = ", x
    end
end

!! parent_impl.f90  (implementation)
submodule (parent_mod) parent_impl
    implicit none
contains
    module procedure add
        x% n = x% n + 1
    end
    module procedure show
        use parent_helper, only: debug_show
        call debug_show( x )
    end
end

!! main.f90
program main
    use parent_mod, only: myint_t
    implicit none
    type(myint_t) :: a

    call a% add()
    call a% show()  !! 101

    call a% add()
    call a% show()  !! 102

    block
      use parent_helper
      call debug_show( a )  !! 102
    endblock
end

!! build
$ gfortran-10 -fcheck=all -Wall -Wextra parent_mod.f90 parent_helper.f90 parent_impl.f90 main.f90

Does this possibly help avoid recompilation of parent_mod.f90 (even when parent_helper or parent_impl are modified)? (And I noticed that the module name "parent" has no meaning here... XD)

Why assertEquals(a, b) instead of assert(a == b)?

A simple assert will only throw AssertionError stating that the asserted conditation evaluated to false:

assert "foo".equals("boo")

java.lang.AssertionError: assertion failed

(not to mention assert string1 == string2 is incorrect due to reference comparison)

By passing both a and b the library can include them in the error message. Here: FEST assertions:

assertThat("foo").isEqualTo("boo");

//throws:
Exception in thread "main" org.junit.ComparisonFailure:
expected:<'[b]oo'> but was:<'[f]oo'>

Note that some languages are more powerful:

In Groovy (example from: Groovy 1.7 Power Assert):

a = 10
b = 9

assert 91 == a * b

yields:

Assertion failed: 

assert 91 == a * b
          |  | | |
          |  10| 9
          |    90
          false

    at ConsoleScript2.run(ConsoleScript2:4)

In Scala (ScalaTest) there is a special === operator:

assert(1 === 2)

yields 1 did not equal 2.

Why is IoC / DI not common in Python?

I don't actually think that DI/IoC are that uncommon in Python. What is uncommon, however, are DI/IoC frameworks/containers.

Think about it: what does a DI container do? It allows you to

1. wire together independent components into a complete application ...
2. ... at runtime.

We have names for "wiring together" and "at runtime":

1. scripting
2. dynamic

So, a DI container is nothing but an interpreter for a dynamic scripting language. Actually, let me rephrase that: a typical Java/.NET DI container is nothing but a crappy interpreter for a really bad dynamic scripting language with butt-ugly, sometimes XML-based, syntax.

When you program in Python, why would you want to use an ugly, bad scripting language when you have a beautiful, brilliant scripting language at your disposal? Actually, that's a more general question: when you program in pretty much any language, why would you want to use an ugly, bad scripting language when you have Jython and IronPython at your disposal?

So, to recap: the practice of DI/IoC is just as important in Python as it is in Java, for exactly the same reasons. The implementation of DI/IoC however, is built into the language and often so lightweight that it completely vanishes.

(Here's a brief aside for an analogy: in assembly, a subroutine call is a pretty major deal - you have to save your local variables and registers to memory, save your return address somewhere, change the instruction pointer to the subroutine you are calling, arrange for it to somehow jump back into your subroutine when it is finished, put the arguments somewhere where the callee can find them, and so on. IOW: in assembly, "subroutine call" is a Design Pattern, and before there were languages like Fortran which had subroutine calls built in, people were building their own "subroutine frameworks". Would you say that subroutine calls are "uncommon" in Python, just because you don't use subroutine frameworks?)

BTW: for an example of what it looks like to take DI to its logical conclusion, take a look at Gilad Bracha's Newspeak Programming Language and his writings on the subject:

• Constructors Considered Harmful
• Lethal Injection
• A Ban on Imports (continued)

---

Related Topics