Personal Blogs

Last week I wrote a blog post about how crazy we must have been to make so many changes to the quiz system, causing it to become buggy because we couldn't reliably merge changes from new core Moodle versions.

This week I thought I'd write the opposite post explaining why we did this crazy thing... and why we might be doing more.

Basically it's quite simple - we're a huge institution and we have many existing procedures and policies when it comes to assessment. If we only used Moodle Quiz for formative quizzes, that would be fine. But we're trying to use it to replace the old paper-based multiple choice tests, too. Those count toward marks - so all the usual policies apply.

Data protection law is also a big deal. We have to make sure that the wrong people don't get to see student marks. For example, I just had to fix a panicky last-minute bug in our upcoming release. We have these custom quiz reports (which are part of the 'sort of in Moodle 2.0' work I mentioned last time). It turned out that one report didn't properly support groups, meaning that a tutor in one group could see the marks of students in other groups - not just their own. Oops! Apparently, that's a data protection issue.

(Personally, I get to see everyone's Moodle Quiz marks across the whole system, should I choose! Insert evil laughter here.)

In addition to the law, and university policies, there are also expectations involved with the fact that we've been doing things a certain way for a very long time. The Open University was a pioneer of computer-marked assessment. (And that's not an overstatement - I worked on a previous system that interacted with the assessment results from the current paper-based system; a student's results were, and quite possibly still are, represented as ASCII text files... with characters that have to be in specific positions in each line... because these files are exact representations of the punched cards used in a previous version of the system. Yes, really*.)

* ...well, probably. And it's a fun story.

When we've always done something one way, it's kind of hard to tell people 'nope, you have to do it differently now' - especially in the cases where 'differently' is actually 'worse'. Nobody's complaining that we got rid of punched cards, but pretty much anything else is open for argument.

The next looming challenge is double-marking of essay questions. Moodle quizzes can include human-marked questions. That's great! Unfortunately, University rules say that these questions have to be double-marked (marked independently by two different people who each can't see what the other gave it, and I guess with some procedures if the marks are significantly different). This isn't possible within Moodle. At present we solve this by... errr....

...printing out the answers and sending them to the two markers, who write down the marks (like, with a pen) and send them back (like, in the post) to the relevant department here, who work out the correct mark (like, with a calculator) and type it back into the Moodle gradebook (like, literally a person typing them in one by one).

As you can see that can't continue and drastically reduces the chances that we can use this kind of question at all. So there's a worrying prospect that we may have to make more, scarier custom changes to Moodle quiz in the future!

Key word, though: 'future'. Yep, I'm still waiting for Tim to get back.