Testing Biskit Modules

Biskit comes with a nifty little extension of the Python UnitTest framework. Most Biskit Modules contain test cases -- very important for keeping the package functional over time. The Biskit/test.py module automatically discovers all of these test cases and allows you to filter them by tags or package.

For the impatient

To run the most important test cases at once, go to the folder containing the Biskit packages and then:

cd biskit/Biskit
python test.py -e pvm old

This excludes test cases that require PVM (parallel virtual machine) and test cases labelled as "OLD". Call python test.py without any arguments to get a full list of available options and filtering methods.

In order to test a single Biskit module, simply execute it. For example:

python PDBModel.py

Read on if you want to know more...

Key features of Biskit.test

based on standard Python Unittest

tools that are developed for Unittest also work with the Biskit test clases

automatic discovery of test cases

there is no need for the registration of tests or any other house keeping

filtering of tests by tags

test cases can be labelled with freely chosen tags and then be included or excluded from test runs

filtering of tests by sub-package

tests can be run for the complete Biskit project or for any selection of sub-packages of Biskit

simple execution of modules triggers embedded tests

individual Biskit modules can be tested by simply executing them

interactive work with test variables

variables assigned to test classes are automatically pushed into the global name space for interactive work after the tests have finished or failed

 

Testing individual Biskit modules

The test cases associated with a single Biskit module can be run by simply "executing" the module with the python interpreter. This has the advantage that you receive detailed output from the test run and you can afterwards continue working with the variables created during the test. For example:

cd biskit/Biskit
python PDBModel.py

... will run all test cases embedded within the PDBModel module and will give verbose output from those tests.

Rescue test output files

By default, any temporary files created during the tests should be automatically deleted (we sometimes forget, please file bug reports!). However, if things go wrong, these files often are interesting evidence. You can suppress the cleanup by modifying the localTest() method in the main section of the (broken) Biskit module. By default, Biskit modules end like this:

if __name__ == '__main__':

    BT.localTest()

Simply change the last line into:

if __name__ == '__main__':

    BT.localTest(debug=True)

... and then check your /tmp or similar folder for the temporary output files.

Debugging Module tests

The Biskit testing "framework" automatically discovers that the module is not imported but is run stand-alone and kicks in the test suite. Moreover, variables that are created by the test cases are pushed into the namespace of the python interpreter so that you can inspect and work with them after the tests are done. For example, the test cases in PDBModel.py create several variables with test objects:

python -i PDBModel.py
   PDBModel.structureFit test ... center of mass deviation: 
   ...
   ----------------------------------------------------------------------
   Ran 12 tests in 29.002s
   OK
>>>
>>> dir()
[...'fout1', 'fout2', 'fout_pdb', 'local', 'log', 'm', 'm1', 'm2', 'm3', 'm4', 'm5',...]

The -i option tells python to remain in the interpreter after the program has been executed. dir() is a built-in python method to list all items in the current namespace of the interpreter.

Let's play around with variable m:

>>> m
[PDBModel 1BGS  7629 atoms, 2086 residues,  4 chains]
>>> m.source
LocalPath[ {/data/raik/py/biskit|$projectRoot}/Biskit/testdata/com/1BGS.pdb ]

This means m is a PDBModel object that has been originally read in from a PDB file in the Biskit testdata folder.

What's the advantage of all that?

Interactive programming!! For example, you fire up emacs to change a Biskit module, you create a python buffer within emacs (CTRL-C-!) and then hit CRTL-C-C to execute the modified code in this buffer. If something doesn't work as expected, you can play around in the python buffer to inspect the result of the tests, change your code, directly execute fragments of your code in the python buffer (CTRL-C-|) where they can interact with the variables generated by the test, then execute all again, and so on. This kind of interactive programming is a prime advantage of Python. Our little testing framework tries to facilitate this programming style whereas normal testing frameworks are more or less incompatible with it.

Testing many modules at once

The Biskit.test module also doubles as an executable program that discovers and collects test cases and executes them according to filtering options. The syntax of test.py is:

    test.py [-i |include tag1 tag2..| -e |exclude tag1 tag2..|
             -p |package1 package2..|
             -v |verbosity| -log |log-file| -nox ]

Please run test.py without any arguments to get a help screen with detailed explanations!

Writing Biskit Unit Tests

... is not difficult. (1) You embed a class derrived from Biskit.test.BiskitTest in your module, (2) write your tests like you would do it for the standard Python unittest and (3) put the method Biskit.test.testLocal() in the __main__ body of your module. The Biskit.test module contains a detailed description:
API documentation of Biskit.test