Personal Blogs

For the third Moodle Monday event here at the Open University, we (the VLE development team) decided to try fixing the unit tests in Moodle 2.

(If anyone's reading this who doesn't know what a unit test is, basically it is an automatic software test that checks part of the software is working as expected. The main purpose is to make sure that later changes don't break something which previously used to work. Moodle 2 has quite a few unit tests, although they probably cover only a small portion of the code base.)

The unit tests were really comprehensively broken. They ran a bit and then crashed with a fatal error. So it's not the case that tests just failed: they couldn't even complete.

Personally, I think it would be unacceptable to release software if the unit tests don't pass, even though in this case most of the failures are due to people not properly maintaining the unit tests rather than actual problems with the software being tested - so (at least in my assumption), this should prevent the release of Moodle 2.

I investigated it using MDL-24905 as the tracker bug, and filed (some during the day, as we got past other issues) 21 subtasks for each problem that was discovered. Plus Tim filed another one.

Of these, we fixed 13; 4 were duplicates of those; another 2 went away for unknown reasons (i.e. probably duplicates too). One more was partially fixed. So that will only leave two and a bit that we didn't deal with. There's a possibility we might handle some of these too, during the week.

After our changes the unit tests run to completion, though obviously they do not all pass. Here are the current results on our developer server:

114/119 test cases complete: 3781 passes, 0 fails and 59 exceptions.

Time taken: 31 mins 2 secs.

I think all the remaining breakage is in portfolio unit tests, which are generally in a mess.

Hopefully all Moodle developers can ensure that unit tests don't break in future. It would be nice if moodle.org had a system which automatically runs full unit tests each night and emails key people if they fail...

We had the second Moodle Monday event at the OU. This time we were attempting to fix bugs in Moodle 2, instead of just doing QA testing. Or in other words...

I'm not doing an individual score table this time (although depending on how you count it, Jenny is the winner but here's the outcome summary:

We looked at 27 bugs.

We potentially fixed (either attaching a patch to the bug, or actually committing) 12 of them. (Mostly these were just submitted as patches, so they might not necessarily all be accepted as correct fixes by the community.)

In 6 other cases we were able to determine that the bug isn't reproducible, is invalid, or is a duplicate.

For the other cases, either it was too difficult to fix or we just ran out of time. In some cases we were able to add comments to the bug describing our investigation, possibly helping whoever works on that bug next.

Hopefully this has been helpful to the Moodle project. I don't know yet if we're going to do Moodle Monday next week as well, let's see.