OU blog

Personal Blogs

Angelic, er, alternatives?

Visible to anyone in the world
Edited by Sam Marshall, Friday, 22 May 2009, 17:43

Last week I wrote a blog post about how crazy we must have been to make so many changes to the quiz system, causing it to become buggy because we couldn't reliably merge changes from new core Moodle versions.

This week I thought I'd write the opposite post explaining why we did this crazy thing... and why we might be doing more.

Basically it's quite simple - we're a huge institution and we have many existing procedures and policies when it comes to assessment. If we only used Moodle Quiz for formative quizzes, that would be fine. But we're trying to use it to replace the old paper-based multiple choice tests, too. Those count toward marks - so all the usual policies apply.

Data protection law is also a big deal. We have to make sure that the wrong people don't get to see student marks. For example, I just had to fix a panicky last-minute bug in our upcoming release. We have these custom quiz reports (which are part of the 'sort of in Moodle 2.0' work I mentioned last time). It turned out that one report didn't properly support groups, meaning that a tutor in one group could see the marks of students in other groups - not just their own. Oops! Apparently, that's a data protection issue.

(Personally, I get to see everyone's Moodle Quiz marks across the whole system, should I choose! Insert evil laughter here.)

In addition to the law, and university policies, there are also expectations involved with the fact that we've been doing things a certain way for a very long time. The Open University was a pioneer of computer-marked assessment. (And that's not an overstatement - I worked on a previous system that interacted with the assessment results from the current paper-based system; a student's results were, and quite possibly still are, represented as ASCII text files... with characters that have to be in specific positions in each line... because these files are exact representations of the punched cards used in a previous version of the system. Yes, really*.)

* ...well, probably. And it's a fun story.

When we've always done something one way, it's kind of hard to tell people 'nope, you have to do it differently now' - especially in the cases where 'differently' is actually 'worse'. Nobody's complaining that we got rid of punched cards, but pretty much anything else is open for argument.

The next looming challenge is double-marking of essay questions. Moodle quizzes can include human-marked questions. That's great! Unfortunately, University rules say that these questions have to be double-marked (marked independently by two different people who each can't see what the other gave it, and I guess with some procedures if the marks are significantly different). This isn't possible within Moodle. At present we solve this by... errr....

...printing out the answers and sending them to the two markers, who write down the marks (like, with a pen) and send them back (like, in the post) to the relevant department here, who work out the correct mark (like, with a calculator) and type it back into the Moodle gradebook (like, literally a person typing them in one by one).

As you can see that can't continue and drastically reduces the chances that we can use this kind of question at all. So there's a worrying prospect that we may have to make more, scarier custom changes to Moodle quiz in the future!

Key word, though: 'future'. Yep, I'm still waiting for Tim to get back. smile

Permalink 2 comments (latest comment by Sam Marshall, Wednesday, 3 June 2009, 13:38)
Share post

Devilish differences

Visible to anyone in the world
Edited by Sam Marshall, Friday, 15 May 2009, 17:50

(I thought I'd start with a long technical post just to scare off any non-geek readers! Well, maybe that wasn't the reason, but.)

My advice for any organisation that has developers working on customisations to Moodle code is to keep your changes to a minimum. Completely new code that plugs in - new modules, new admin reports, new blocks - is fine, go crazy. But try as hard as possible not to change existing Moodle core code, even if that means telling your users 'no we can't do that'.

Here's where it came up this week - our quiz module. Oh, dear gods, our quiz module. I'm not particularly enamoured of the fact that we've renamed it 'iCMA' (obviously somebody thinks that's a much snappier name than 'quiz') but hey, that was a local language file, so it didn't cause a difference from Moodle core code. Great!

Unfortunately the eighteen million other things we've changed in quiz did cause a difference from Moodle core code. We have custom navigation, custom statistics, custom reports, custom interface, custom question banks, custom grade calculation, custom ARGH. Some of this is kind of based on work which we were sort of hoping would go into Moodle 2.0 and become standard. But what's actually ended up in Moodle 2 is probably going to be a bit different, won't cover all of it, and is still at least a year away! (From becoming the OU's Moodle platform, that is. I'm not speculating on the public Moodle 2 release date. If you need more information on that, please see this definitive reference.)

The changes to our quiz module are so significant that - unlike every other area of code - we've given up merging bugfixes and other modifications from the Moodle 1.9 'stable' branch - there are so many changes it's impossible to merge without somebody completely expert in the system. While the rest of our platform is just moving to 1.9.4, I don't know where the quiz is, 1.9.1 with bits of 1.9.4? Nightmare.

In this particular case somebody was a little bit over-eager when merging in the 1.9.4 changes. They noticed that one file in quiz actually didn't have any OU custom changes! What an opportunity! So they merged the core changes for that file.

Unfortunately, that file turned out to call a function in another file - which had been newly added in 1.9.4, but hadn't been merged in, because the myriad OU changes already in that file made it too complicated.

For now I have fixed this by de-merging the changes and going back to the older version, which presumably means that we are again suffering from a bug which is fixed in standard Moodle. Yay.

So in summary, this is what we did. Don't do it.

Oh and the solution above might sound a bit short-term, but no worries - I have a long-term plan too. It is 'Wait for Tim to get back here and let him fix it'. smile

Share post

This blog might contain posts that are only visible to logged-in users, or where only logged-in users can comment. If you have an account on the system, please log in for full access.

Total visits to this blog: 247910