OU blog

Personal Blogs

New blog post

Visible to anyone in the world
Edited by Sam Marshall, Tuesday, 2 Nov 2010, 17:18

For the third Moodle Monday event here at the Open University, we (the VLE development team) decided to try fixing the unit tests in Moodle 2.

(If anyone's reading this who doesn't know what a unit test is, basically it is an automatic software test that checks part of the software is working as expected. The main purpose is to make sure that later changes don't break something which previously used to work. Moodle 2 has quite a few unit tests, although they probably cover only a small portion of the code base.)

The unit tests were really comprehensively broken. They ran a bit and then crashed with a fatal error. So it's not the case that tests just failed: they couldn't even complete.

Personally, I think it would be unacceptable to release software if the unit tests don't pass, even though in this case most of the failures are due to people not properly maintaining the unit tests rather than actual problems with the software being tested - so (at least in my assumption), this should prevent the release of Moodle 2.

I investigated it using MDL-24905 as the tracker bug, and filed (some during the day, as we got past other issues) 21 subtasks for each problem that was discovered. Plus Tim filed another one.

Of these, we fixed 13; 4 were duplicates of those; another 2 went away for unknown reasons (i.e. probably duplicates too). One more was partially fixed. So that will only leave two and a bit that we didn't deal with. There's a possibility we might handle some of these too, during the week.

After our changes the unit tests run to completion, though obviously they do not all pass. Here are the current results on our developer server:

114/119 test cases complete: 3781 passes, 0 fails and 59 exceptions.

Time taken: 31 mins 2 secs.

I think all the remaining breakage is in portfolio unit tests, which are generally in a mess.

Hopefully all Moodle developers can ensure that unit tests don't break in future. It would be nice if moodle.org had a system which automatically runs full unit tests each night and emails key people if they fail...

Permalink Add your comment
Share post


Tim at the Lone Pine koala sancutary

New comment

Actually, there is a hacked-together script that runs the unit tests every night: http://moodle.org/mod/cvsadmin/view.php?tab=unittest (that may only be visible to developers). However, it is currently showing blank, perhpas becuase of the fatal errors. It does not attempt to notify anyone of the failures. It relies on people bothering to look.