Personal Blogs

Rant time! (Wait. That's every time... Well, this time I'm actually typing it into my blog.) This is all about computer programming so if you don't know anything about computer programming you are officially off the hook! (Enjoy your day!)

Coding style guidelines are usually a matter of opinion. But here's one of my opinions, and in my opinion, it's not an opinion, it's a fact.

Apart from normal start-of-line indent, trying to align lines of code horizontally is always wrong.

Here are three examples. For each one I'll give WRONG then RIGHT.

WRONG:

$this->something = $something;
$this->x         = $x;

Right:

$this->something = $something;
$this->x = $x;

WRONG:

$thing = array(
    'value'        => 1,
    'anothervalue' => 2,
    'more'         => 3);

Right:

$thing = array(
    'value' => 1,
    'anothervalue' => 2,
    'more' => 3);

WRONG:

$something->function1()->function2($variable, $anothervariable,
                                   $yetanothervariable) {

Right:

$something->function1()->function2($variable, $anothervariable,
        $yetanothervariable) {

etc.

The wrong examples are wrong because computer code is not carved in stone, nor even printed on paper. (When you're grave-carving or desktop-publishing, feel free to make things line up. When you're computer-programming, stop it! Now!) Unlike gravestones and newsprint, computer code changes.

Let's say I add a variable called $slightlylonger to the list of assignments. With the right code, this involves me adding one code line. With the wrong code, I have to add one code line, but now I have to modify 2 other code lines as well to make the stupid thing consistent.

Similarly if I change the name of one of the functions I would now have to go around changing all the following lines as well, where it wrapped to multiple lines. (The function example is also bad because horizontal space is usually at a premium, which is why you're having to wrap lines in the first place; when you wrap the line, you want to get a useful amount of increased space for adding more parameters in future. That doesn't happen if the wrap point is artificially shifted to way up near the end.)

Having to modify extra lines matters for two reasons: first, making the changes is annoying, and second, the more lines you touch then the more chance of a conflict in your version control system.

In addition, as I'm sure the newspaper designer would tell you, both these approaches are bad even from a layout and visual design perspective. They result in 'columns' that start every which way at random points. It's messy.

So, much better not to make it line up in the first place. If you disagree... well there's a word in bold in this post... let's leave it there

Rant over!

And no this wasn't directed at any particular developer, or even the Moodle project. Just, grr.

I'll try to write a blog post about something more specific to do with our Moodle developments next time. But this is better than nothing, right?

Sorry I've been neglecting this blog again. I'll do a proper post sometime (er I guess) but I just wanted to throw in a quick post that might be of interest to Moodle 2 developers.

Have you ever needed to test the 'restore' of a course (e.g. that includes your new module) and got annoyed because restoring involves clicking through about a million screens?

I've written a plugin that you can use to do temporary restore in one click (once you have set up the backup file you want to use). Then you can view the restored course in one click, delete it also in one click, restore it again (in, yes, one click), and repeat as much as you like while fixing bugs.

(I guess I said one click enough times that it's technically like three clicks now, but still. It's a lot better than doing the same thing manually.)

Plugin is called Broom (thanks to Tim for acronym help) because it's both fast and it also helps you keep your database clean by deleting all those courses.

https://github.com/sammarshallou/moodle-local_broom

A few things to update on with regard to the Moodle 2-related work I'm doing at the OU.

These are quite geeky but might be partially comprehensible by non-programmers this time? Maybe? I'm not sure. Oh well, not forcing anyone to read it.

1. No OU public repository. It's been discussed here and we have agreed that our Moodle 2-based source code repository won't be public after all, which is a bit disappointing. There were concerns about what happens if a developer commits code that contains a system password or something like that (this should be found in code review, but by that point, it's already in the repository).

However I still want to ensure that we release more things more reliably than last time (where we used manual processes). This should be easier because we are trying to keep core modifications as near to zero as possible, and to make our key customisations into standard, independent plugins wherever we can - so they're more practical for other people to use. The current plan is to put out all this stuff... in some as-yet-undetermined but automated way... from about September which is roughly when real live students start using the system, so it's the first time that really fully-tested code would be available.

So anyway, all our code that is suitable for use elsewhere will be public; that also means much more code will be public than under our Moodle 1.9 system; but you probably won't be able to point and laugh at all the core-Moodle customisations we might still be forced to make. Or to steal our passwords.

Before September I will probably do manual releases of a few key modules if anyone else is relying on them.

2. First outsourced project is complete

We've finished the testing process on a small admin report that was developed by Netspot. It works! Hopefully the whole outsourcing thing will actually be a success this time around.

I think the detailed test scripts I've been doing are pretty important to ensuring we actually get what we need (along with sensible selection of companies to do the work). Hopefully, anyhow, because it took ages to make them.

3. ForumNG nearly working in Moodle 2

ForumNG (our forum system) is more or less working in Moodle 2.

This has been a bit of an effort: according to git, ForumNG is about 28,700 lines of code. During the process I have added about 800 lines of code, deleted about 2800 lines of code, and modified about 25,700 lines of code. (If I modified the same line four times in separate commits, it counts for 4 in these stats, so I haven't really touched every line... yet.) Obviously some of these were search-and-replace modifications but I've done an awful lot of manual changes too.

It isn't finished yet. The big thing that doesn't work is the JavaScript, which needs converting to YUI3 (easy but tedious) and to support the new HTML editor and file manager (difficult and may possibly turn out to be a significant rewrite of certain areas). Apart from that I have decided that in order to check things I definitely need to write a comprehensive manual test script that covers all features. This is going to take quite a long time.

4. I got a fairly major change into Moodle 2...

As mentioned last blog post, I did a fairly big chunk of work for core Moodle 2 that will allow plugins (such as, for instance, ForumNG) to do more interesting things on the main course page (such as, for instance, add an 'Unread' indicator - not previously possible without hacks to core). It also improves a bunch of related areas of code, documentation, etc.

Anyhow, this work is now actually included in standard core Moodle 2. Yay - and many thanks to Petr (who came up with much of the overall concept, reviewed the design, reviewed the code, and fixed bugs) and Eloy and others at Moodle HQ.

5 ...but it kind of broke this week's release.

Sorry if you got this week's Moodle 2 release and hit one of the bits that were broken as a result of the change And, um, if you didn't get this week's release yet, here's a tip - wait for next week. We did actually test the change - I mean I tested it lots while I was developing it, then Moodle HQ tested it - but there have been just a couple of odd side effects in apparently unrelated areas.

Of course nobody runs random Moodle weekly releases on their live server straight away without any testing of their own, so this is okay, right? .... Right .....?

6. I proposed a presentation for the UK Moodle Moot.

We're going to the UK moot again, and I've proposed a short presentation about working together with Moodle HQ. Not counting chickens though: I think last time I proposed a presentation too, and it was going to be totally awesome*, but they rejected it.

* Safe to make that claim about a presentation that never existed, I think.

For those not familiar with Git - no I'm not being rude. (Perish the ****ing thought!) Actually, 'Git' is a popular version control system for organising software development.

And to be honest, if you didn't know that already, you should probably skip the rest of this post because it isn't going to make any sense. Really this is not a very exciting post at all. Sorry.

That said - as mentioned previously, developers here are getting ready to start work on developments for our Moodle 2-based course website system (aka 'VLE2').

We are using Git to host the code. That's good news because (a) it is probably the best (widely-used) version control system in existence, and (b) it's the same one that Moodle core developers are using. But there are three problems:

• For most (not all) of our developers, their only experience of version control systems has been the CVS-based process we have previously used here. Hardly any have any previous Git experience.
• Git is really complicated (=powerful) and includes loads of new concepts (the basic concept of any kind of distributed version control at all is probably the biggest and scariest of these).
• Git's GUI integration into developer tools (Eclipse) is frankly only about half written so far.

None of these problems are insoluble (we hope). The approach we're taking is to define a very specific way to use the software and the system that avoids as many of the complexities as possible, while retaining the key benefits we need. And to document that in a really clear manner.

The documentation is why I'm making this post. I know there are other institutions who are trying to switch to Git and I thought maybe the documents I wrote about this might be useful as a base to work on for your own plans. Understand these aren't generic documents in any way - they're specifically about our own system and our process and they can't directly be used by anyone else. But if you're trying to help a bunch of developers to switch to Git, you might want to use our documents (or possibly even our process) as a reference when putting together your own material.

Or to put it another way, if you really want a Word document that tells you how to install git on a Windows computer, with like a zillion screenshots, congratulations! This is your lucky day.

There are actually three documents - one about installing Git, configuring Eclipse to work with it, and setting up an account on our Redmine system; one about 'ordinary' developers doing work on the system; and one about lead developers merging in those changes after review.

Link to the three documents

These have been 'tested' to the extent that other people have tried going through these processes and it worked (of course, the documents were initially wrong, but have since been corrected), but no further.

Of course the most important question regarding all this is... does our process actually work? If anyone's interested in the answer to that, please get back to me in about six, seven months.

Just a quick update here about our Moodle 2 conversion plans (two people have asked already so I thought I'd post here and save effort later). If anyone's using any of the main OU custom modules and wants to know, it's good news! We will be converting these to Moodle 2. Specifically:

• ForumNG we'll be converting in-house soon. I'm hoping it will be finished around January. Same applies for the OU search block that provides live full-text search facilities to all these tools.
• OU blog and OU wiki we are currently trying to outsource - I wrote some specifications with really detailed test cases... We are hoping to get the code probably a bit later than ForumNG, maybe end January.

In both cases we are hoping to put these in with other code for final integration testing etc for an internal release at the start of March. First real usage (along with some other features) will be with our second-phase release in June, and it's likely students won't use it until July.

So basically, we should have the basic converted modules ready quite soon, but the more reliable 'same code we are using for live students' versions will be later.

I know Moodle 2 is theoretically about to be released very soon, so there might be some people out there thinking 'argh! but i'm about to upgrade and you're telling me I will lose my reliable ForumNG version for another seven months!' - but the version that's coming out now is the first one and in my opinion you'd be very unwise to use it on a live system. I'd recommend you wait at least another three, four months to upgrade live systems, even if you don't use any OU custom modules...

So anyway that was the good news - now for the... wait, it's more good news! Of a sort. We're planning to do a similar public release (with similar timescales) of our OU content module, which provides online content within Moodle based on XML document files, which are basically equivalent to textbooks but can include audio, video, Flash, Java applets, and some interactive thingies, etc. (By the way, this is also the same XML format we use to produce most of our actual printed textbooks, although the 'print' part is done in a separate system and won't be included in this release.) Example of current 1.9 version on OpenLearn

However, there might not be sufficient documentation etc. for anyone to actually use it as we have quite enough trouble supporting this system in-house without supporting other people too. It will definitely be an 'if you're stuck, better start looking at the code' type system. Still if you're interested, that's something to look forward to; I will post in this blog when it's available.

Also, I'm not quite sure this blog post can withstand yet more lashings of good news, but we're trying to make our entire Moodle 2 code repository public, which will mean that even if we don't officially 'release' and support something that is used in-house, anyone can come and get almost anything we do. Alternatively it will be a great resource for those who just want to point and laugh at our code quality, or count the number of customisations in core code, etc...

However, it's not me doing the public repository, and I don't know how it is going, so this part is a bit less sure than the other parts. Just thought I'd mention it since it seemed to fit in with the topic.

For the third Moodle Monday event here at the Open University, we (the VLE development team) decided to try fixing the unit tests in Moodle 2.

(If anyone's reading this who doesn't know what a unit test is, basically it is an automatic software test that checks part of the software is working as expected. The main purpose is to make sure that later changes don't break something which previously used to work. Moodle 2 has quite a few unit tests, although they probably cover only a small portion of the code base.)

The unit tests were really comprehensively broken. They ran a bit and then crashed with a fatal error. So it's not the case that tests just failed: they couldn't even complete.

Personally, I think it would be unacceptable to release software if the unit tests don't pass, even though in this case most of the failures are due to people not properly maintaining the unit tests rather than actual problems with the software being tested - so (at least in my assumption), this should prevent the release of Moodle 2.

I investigated it using MDL-24905 as the tracker bug, and filed (some during the day, as we got past other issues) 21 subtasks for each problem that was discovered. Plus Tim filed another one.

Of these, we fixed 13; 4 were duplicates of those; another 2 went away for unknown reasons (i.e. probably duplicates too). One more was partially fixed. So that will only leave two and a bit that we didn't deal with. There's a possibility we might handle some of these too, during the week.

After our changes the unit tests run to completion, though obviously they do not all pass. Here are the current results on our developer server:

114/119 test cases complete: 3781 passes, 0 fails and 59 exceptions.

Time taken: 31 mins 2 secs.

I think all the remaining breakage is in portfolio unit tests, which are generally in a mess.

Hopefully all Moodle developers can ensure that unit tests don't break in future. It would be nice if moodle.org had a system which automatically runs full unit tests each night and emails key people if they fail...

We had the second Moodle Monday event at the OU. This time we were attempting to fix bugs in Moodle 2, instead of just doing QA testing. Or in other words...

I'm not doing an individual score table this time (although depending on how you count it, Jenny is the winner but here's the outcome summary:

We looked at 27 bugs.

We potentially fixed (either attaching a patch to the bug, or actually committing) 12 of them. (Mostly these were just submitted as patches, so they might not necessarily all be accepted as correct fixes by the community.)

In 6 other cases we were able to determine that the bug isn't reproducible, is invalid, or is a duplicate.

For the other cases, either it was too difficult to fix or we just ran out of time. In some cases we were able to add comments to the bug describing our investigation, possibly helping whoever works on that bug next.

Hopefully this has been helpful to the Moodle project. I don't know yet if we're going to do Moodle Monday next week as well, let's see.

Just an update on my previous post. Public comments in OU blog are now working (I hope - I'm about to test it), so you can leave comments on this post even if you don't have an OU login.

If you have an OU blog like this one and you want to use the feature, see the previous post for information about how to turn it on for your own blog posts. You have to select it for each post, it won't happen by accident.

I got asked some questions about oublog (I mean, from somebody who is considering using it on their Moodle system), and I thought the answers might be useful to other people, so I'm reposting here.

What is the Personal blogs link on the front page?

The link on the home page takes you to the personal blog for the current user. I don't think there is a way to remove it, so please just hide it if you don't want the personal blog facility (if you are only using course blogs).

How can I set a course blog up for use as a journal where only the student involved, and a teacher on the course, can see that student's personal blog?

a - Set 'Individual blogs' option on blog settings to 'Separate individual blogs' (means students can't see each others' blogs)

b - Make sure your teacher role(s) has the 'View individual blogs' capability (I think they should have this by default but just to check)

c - Decide whether or not to use group mode:

i. If you only have one teacher there is no point.

ii. If you have lots of different teachers, each of which has their own group of students, then use group mode. Use visible groups if the teachers are allowed to see each others' students, or separate groups (and of course make sure the teachers belong to the groups) if they are only allowed to see their own students.

Who can edit comments?

You can delete comments (your own; or whoever posted the blog post can delete all comments; so can admins obviously), but not edit them. We didn't provide an edit facility for comments because they're supposed to be kind of trivial. [Also because, since we have to make sure everything's auditable, making them editable adds a lot of complexity.]

Could you add a grading option when upgrading the module to Moodle 2?

I will see what we can do but can't promise anything unless we manage to get some internal users to request it.

If we added grading, how should it work? Would you want the teacher to grade the student's participation (i.e. a single grade) or to rate each post like on forum (i.e. ratings per post with some kind of aggregation feature, like taking the average, to come up with an overall grade)?

My guess is that the former is more useful - but it's also something you can achieve already by manually adding a grade for the activity in the gradebook and having teachers use the gradebook interface to set the grade, so we'd only be making the interface a bit nicer.

I just ran browser stats on the last week of web logs (kind of wanted to do trends, but haven't found any data earlier than that yet).

This might be of interest to Moodle people in general (as well as us) given that Moodle 2 ditches support for Internet Explorer 6.

Anyhow, here are the details for OU course web sites, considering only external use i.e. all our students and tutors. (For campus-based staff there is a specific computer policy which obviously biases the results.) These are calculated using a program I wrote, based on the number of requests to 'interesting' pages (excluding images, javascript, etc). Percentages exclude things like search engines, feed readers, anything else identified that isn't a browser, plus user agents my system doesn't recognise (approx 2% of total). For all browsers except IE, I picked a version that covers the vast majority of users and only included that version and above; older versions are in 'other'.

• IE8: 39%
• Firefox 3+: 22%
• IE7: 19%
• IE6 and below: 7%
• Chrome 5+: 6%
• Safari 4+: 5%
• Mobile: 2%
• Other: 1%

(Adds up to 101% due to rounding, sorry.)

Anyway, the good news is that on our system, IE6 is down below 7%. A year ago, it was double that. Eventually, that monster's really going to die!

Good luck anyone waiting for IE9 to become current so we can all use HTML5, though...

Eclipse 'Helios' (3.6), which was released this month, includes the new version of PDT (PHP development tools). I tried this out for Moodle development at work today. In my workspace I've got about six copies of Moodle (various OU and core versions) plus the plugins CVS, plus a few Java projects kicking around. So it's a fairly large PHP workspace.

Good news:

1) The new DLTK indexing means that referencing PHP functions (i.e. ctrl-click to jump to definition) is lightning-quick.

2) There's a beta version of Git support as part of Helios - I wouldn't recommend using this until it hits 1.0 personally, and um I don't think it can do merge/pull yet which is a bit of a snag... but moving toward mainstream support is still great news.

Bad news:

1) It uses more RAM than 3.5; you definitely need to increase heap in eclipse.ini from Eclipse default (384MB) to at least 1GB. Note - on 32-bit windows, the max you can set for this is slightly over a gig even if you have 3+GB RAM, so use 64-bit if possible.

Sometimes I saw it peg the RAM at around the 1GB, dragging performance to a crawl; pretty sure this is a memory leak.

2) When editing PHP files I get frequent errors, mostly stack overflow somewhere in DLTK analysis.

3) Because the git stuff is beta, although it shouldn't affect these projects (they don't use git) I thought maybe it might, so I uninstalled that. Didn't make any difference.

Basically I am sure Eclipse Helios is a great release in general and for Java development but, not entirely surprisingly, the PDT folks (Zend I guess) have dropped the ball again; this will probably be better once they get it fixed, but right now it isn't ready for use. I've gone back to the previous version - it might not be able to to reliably find definitions of things, but at least it can usually keep up with my typing and doesn't show an error dialog every ten minutes.

Eclipse Helios SR1 (aka 3.6.1) is due on September 24th. If you're considering upgrading, that would seem like a good date to try.

Quick post to mention something I also posted in Moodle developer forums. I've been helping to investigate how much work it will be to move the OU from our current Moodle 1.9-based system to Moodle 2.

As part of that I converted the Structured Content module (oucontent), which is what displays textbook-like content within the OU VLE.

The non-techie version of this is 'it took about four or five days to convert this module'. If anyone's interested in the techie version, however, you can find it in this Word document which is my notes of the process.