OU blog

Personal Blogs

Moodle 4.2 cron

Visible to anyone in the world
Edited by Sam Marshall, Wednesday, 14 Feb 2024, 13:39

I'm writing a blog post about the cron system in Moodle 4.2 because I wanted to understand all the settings!

Cron overview

Normally, you set the cron script to be launched once per minute. These runs can happen in parallel (one keeps running while the next one starts), which is the only way that multiple tasks can run at once. So if you have just started the server, then only one task can run, but a minute later there will be another cron launch and now two tasks can run, and so on.

Processing within a single cron runner

1 If we are not already at the concurrency limit for scheduled tasks (task_scheduled_concurrency_limit - default 3):

a Get the next scheduled task and run it
b Repeat from (a) until there are no tasks due or we hit the scheduled task runner lifetime (task_scheduled_max_runtime - default 30 min)
 
2 If we are not already at the concurrency limit for adhoc tasks (task_adhoc_concurrency_limit - default 3):

a Get the next adhoc task and run it
b Repeat from (a) until there are no tasks due or we hit the adhoc task runner lifetime (task_adhoc_mask_runtime - default 30 min)
 
3 If we haven't already reached the keep alive limit (cron_keepalive - default 3 minutes), wait a second and repeat from (1).

Implications of default settings

No tasks

If there are no tasks to run, on default settings there will usually (after server startup) be 3 instances of the cron runner at a time, because of the 3 minute keep alive. (The one that started 3 minutes ago has just ended; the one that started 2 minutes ago and the one that started one minute ago are still running; a new one has just atarted.)

Each second, all 3 tasks will acquire the scheduled lock, check for scheduled tasks, release the lock, acquire and adhoc lock, check for adhoc tasks, release the lock.

Infinite tasks

If there sre infinite tasks of both types, on default settings there will usually be 6 instances of the cron runner (=total of concurrency limits). At any given time, three of the instances will be dedicated to scheduled tasks (because of the task_scheduled_maxruntime) while three will be dedicated to adhoc tasks (because of hitting the scheduled concurrency limit, then running until the task_adhoc_maxruntime).

Server startup with infinite tasks

On server startup, at default settings it will be 3 minutes before any adhoc tasks are processed; the first three runners will be dedicated to scheduled tasks, then the next three will be dedicated to adhoc tasks.

Impact of changing settings

task_scheduled_concurrency_limit, task_adhoc_concurrency_limit

These are the biggies. Be careful with these settings.

Low: If you turn down these settings, the cron server will require less server memory at busiest times, and there will be less database/storage impact rom task processing. This could be important because some tasks (GDPR, backup) can occupy quite a lot of both resources.

If you set it low you need to be careful you have also applied other limits to individual tasks (by configuring scheduled tasks to run only when needed, and by seting the limits so that adhoc tasks don't run too many at once, e.g. $CFG->task_concurrency_limit_default = 1 or something more specific). Otherwise it's very easy for a task to eat all the concurrency meaning no other tasks of that type can run. For example, if a GDPR data export task takes 12 hours to run, and you have concurrency limit of 2, then 2 of those would eat it.

High: You need to make sure the cron server has enough memory to cope, and the database can cope with the processing without reducing performance for student requests..

Overall: Depends on your cron server performance, especially memory, and also how heavyweight the tasks you normally run are.

task_scheduled_max_runtime, task_adhoc_max_runtime:

Low: If you set these settings lower it will stop a single task runner from being dedicated to a single type of task (scheduled for 30 mins if there is a slot available, then adhoc for 30 mins) in cases where there are lots of tasks to run. Instead the task runner will alternate between each type of task more quickly.

This will mean it keeps acquiring and releasing locks, so there might be a slight performance cost.

I'm not clear there is any particular benefit in doing this.

If you set these settings very low then it might be harder to reach the concurrency settings. For instance, if you set these to 1 minute each and cron_keepalive at default 3 minutes, and there are no long-running tasks, then the task runner will always exit after 4 minutes (1+1, 1+1), so there cannot be more than 4 tasks running in parallel. (In practice if you need many parallel tasks then you probably do have long-running tasks so this might not be a problem.)

High: If you set the settings very high then it will keep a single runner operating for a long time (up to the sum of both settings). This is probably a bad idea because you might as well exit the PHP process periodically (it could clear up any memory that got allocated, any static variables that got set, generally anything that might get encrusted). There's another one along in a minute! 

You don't need to set this high if a single task takes a long time, the task runner won't exit mid-task anyway.

Overall: The default is probably OK but a lower value like 10 minutes (= max runner lifetime approx. 20 minutes assuming there are no long-running tasks) might make more sense.

cron_keepalive:

Low: If you reduce this setting it will mean fewer task runners waiting when the system is idle. For example, if you set this to 60 seconds, there will usually be just one cron runner (if there are no tasks to run). This is both a sliight performance advantage (less locking etc) and also a disadvantage in terms of how quickly the system can scale up to run multiple tasks when busy, bearing in mind it can only add one parallel task per minute. 

For example, if the system is idle with one task runner, and then at least three scheduled tasks totalling at least three minutes of work become due at once, no ad-hoc tasks will run for the next three minutes as the existing idle runner and the next two task runners will be occupied with scheduled tasks.

If you set to less than a minute, then even if the system is idle, users will have to wait for the next minute for an adhoc task to run, rather than it happening immediately

High: If you increase the setting then more task runners will wait idle. I don't think there is likely to be very much of a case for increasing it beyond the default though..

Overall: You probably want it set to at least 60 seconds. If there are no big concerns over delay in parallel tasks starting, then 60 seconds is probably sufficient, otherwise maybe it needs to be higher.

Permalink Add your comment
Share post

Search changes in Moodle 3.4

Visible to anyone in the world
Edited by Sam Marshall, Friday, 17 Nov 2017, 12:12

It's a long time since I wrote a blog post but I thought I should make an effort to talk about the global search enhancements that I worked on for the Moodle 3.4 release.

Introduction

At the Open University, which has a large Moodle installation with many custom plugins, we are now beginning to use the Moodle global search facility (replacing internal alternatives). This is happening very gradually and we intend for students not to notice! (If you're an OU student or staff member, the first place you'll see it is on subject websites, probably fairly early in 2018. But it will look almost the same as our current search.)

This is good news for other Moodle users because the current global search doesn't seem to have been all that widely used at scale yet so people might be a bit hesitant about adopting it. We've found and fixed (or helped fix) several problems with it - in terms of how easy it is to manage, and how well it works. These bugfixes are included in Moodle 3.4, and although I'm not suggesting it's perfect now, if you are another institution thinking about using global search, Moodle 3.4 might be a good time. It's a pretty neat system!

The changes

Here are some specifics about the core changes I implemented which are relevant to institutions implementing Moodle global search. 

As usual, thanks go to the many HQ developers, and others, who have assisted significantly with these changes! There are also other search fixes in 3.4 (for example Matt Porritt at Catalyst has improved indexing to include more files that weren't previously indexed), which I'm not listing.

MDL-59039 Global search: Allow partial indexing (in scheduled task)


If you have a significant amount of content in your Moodle system, the initial index is going to take a while. The scheduled task now automatically divides the work into chunks (by default 10 minutes at a time) so that it can run normally in the background - you don't need to run any CLI scripts on your server, there are no long-running tasks that might crash halfway through, and students don't need to see the search interface before you finish indexing. All you need to do to set up search now is:
  1. Install and configure search as before (install Solr somewhere, configure Moodle to point to it, etc.) but do not enable global search yet.
  2. On Manage global search page, enable the new option to index in the background.
  3. If necessary, configure the time allowed per task run (on Manage global search page) and the frequency of the indexing task (on the Scheduled tasks page).
  4. After the task has run, look at the Search areas page. You'll see Not yet fully indexed against areas which haven't finished indexing, and by looking at the Newest document indexed column you can get an idea of how long it will take to index. (I.e. if the newest document indexed is back in 2010, you may be waiting a while for it to catch up.)
  5. Once indexing of all areas is complete (this could be days or weeks later), enable global search as before.

MDL-58957 Global search: Make it possible to search blocks

Any content in the HTML block is now searchable (provided the block is added to a course page). Also, developers can add support to search their own custom blocks - useful if there are cases where lots of content is provided in a custom block.

MDL-55356 Add restored content support to global search API

When you backup and restore a course, any content in the restored course probably has modified dates in the past. As a result, it was not previously indexed, because the search system only looked for content newer than its index. So if you needed to back up and restore courses (for example when creating a new copy of the course for each cohort) then the new ones wouldn't return any search results! We've added a rather complicated new mechanism so that everything in a restored site is indexed regardless of date. (This also means developers now have a way to re-index a specific context if necessary.)

MDL-59913 Global search: Allow search of non-enrolled courses

Unless you are a system administrator, the search system used to search only courses you are enrolled in. Now it also works on courses you can access for other reasons: because it has been configured to be open to everyone without login, or because your role has been overridden to have the moodle/course:view capability. This was really important for our systems as we have many courses that are open to all students. 

This is an option, which you need to turn on using the the Manage global search screen, because in some unusual cases it might make search results worse for your users.

MDL-60357 core_search: Future modified times cause serious problems

There are a few situations where content in Moodle might get a time in the future This content was previously indexed, but that would prevent any other content added between now and that future time from being indexed. We've changed it so that content dated in the future is not indexed until that date arrives.

MDL-60346 core_search: Solr connection ignores proxy settings

The Solr connection previously ignored proxy settings; if your servers are behind a firewall and need to use a proxy to access the Internet, and if the Solr server is hosted on somebody else's computer, outside your network, then it didn't work. This now works. 

However, you should be aware that during indexing, there will be a very large number of requests to the Solr server - it might still be better for your infrastructure if you can configure it to allow those requests through the firewall directly, and add your search server to the proxy bypass option.

What's next?

This project isn't over. We have plans to add more enhancements, some of which will make the interface better for students. With thanks to everyone who helped so far, I hope we can get some more changes in for Moodle 3.5. smile

Permalink 1 comment (latest comment by Estefano Ramirez, Friday, 14 Aug 2020, 11:45)
Share post

AMD JavaScript presentation

Visible to anyone in the world
Edited by Sam Marshall, Wednesday, 16 Sep 2015, 18:24

We are now developing against Moodle 2.9, which means the new 'AMD' method for including JavaScript in Moodle plugins is available to us. (I mentioned this in my last blog post.)

The good news is that it's a lot simpler than previous methods for including JavaScript. The bad news is that it's still a change that developers need to understand, and the Moodle documentation is a bit hard to follow.

So I gave a short introductory presentation for OU developers about how to use this new method. And In case anybody else would be interested, I've attached my slides here as a PDF.

A few details mention OU things but most of it would be applicable to all Moodle developers. (To be clear, this is an introduction and doesn't get into advanced details like performance or whatever.)


Permalink Add your comment
Share post

Moodle JavaScript (jQuery, AMD) for 2.8 developers

Visible to anyone in the world
Edited by Sam Marshall, Thursday, 11 Jun 2015, 12:13

This post is strictly for Moodle developers. If you're not a developer you can stop reading now as this is entirely technical - sorry about that. smile

In Moodle 2.9 the preferred way to write JavaScript is using jQuery and a module framework called AMD. This is documented in the Javascript Modules page of the Moodle developer documentation.

So what if you're writing a new plugin now, which currently has to run in 2.8, but you'd like to update it for 2.9 later with minimal work?

I did a hack to make this work (sort of). In case anyone else is interested, here's how...

1. Wrote my JavaScript using the 'new' approach, so that it begins:

define(['jquery'], function($) {

(For the purpose of this development, I can't currently specify any dependencies other than 'jquery' here; it's hard-coded.)

2. Added a simple wrapper that simulates the AMD module framework without implementing any of it. I put this in a file called define.js. Because this is all done within my plugin, I named the global variables according to my plugin's name. Here it is:

window.format_oustudyplan_modules = {};
window.format_oustudyplan_pending = [];
window.define = function(ignored, fn) {
    window.format_oustudyplan_pending.push(fn($));
};
window.format_oustudyplan_add_pending = function(module) {
    var next = window.format_oustudyplan_pending.shift();
    window.format_oustudyplan_modules[module] = next;
};
window.require = function(modules, fn) {
    var params = [];
    for (var i = 0; i < modules.length; i++) {
        params.push(window.format_oustudyplan_modules[modules[i]]);
    }
    fn.call(this, params);
};

3. Wrote a PHP function to 'load a module':

    public static function load($module) {
        global $PAGE;

        if (!self::$donefirst) {
            $PAGE->requires->jquery();
            $PAGE->requires->js(new \moodle_url('/course/format/oustudyplan/js/define.js'));
            self::$donefirst = true;
        }
        $PAGE->requires->js_init_code('format_oustudyplan_add_pending("' . $module . '");');
        $PAGE->requires->js(new \moodle_url('/course/format/oustudyplan/js/' . $module . '.js'));
    }

4. Wrote another one to 'call a module function':

    public static function call($module, $fn, array $params = array()) {
        global $PAGE;

        $jsparams = '';
        foreach ($params as $param) {
            if ($jsparams !== '') {
                $jsparams .= ',';
            }
            if (is_int($param)) {
                $jsparams .= $param;
            } else if (is_string($params)) {
                $jsparams .= '"' . addslashes_js($param) . '"';
            } else {
                throw new \coding_exception('Unexpected JS param');
            }
        }
        $PAGE->requires->js_init_code(
                'window.format_oustudyplan_modules.' . $module .'.' . $fn . '(' . $jsparams . ')');
    }

5 To include my JavaScript, I do this (the above two functions were in a 'js' class):

            js::load('sections');
            js::call('sections', 'init');

These PHP functions are not the same syntax as used in the 'real' function which is a single function js_call_amd, but are similar. The intention of this approach is that I should be able to move to 2.9 syntax without having to change my JavaScript code. What I will have to do:

  • Delete the PHP code above.
  • Move the JavaScript into the appropriately-named directory and update it using Grunt.
  • Change my loader code to use js_call_amd.

Hope this might be useful to somebody else smile

Permalink Add your comment
Share post

The problem with Behat

Visible to anyone in the world
Edited by Sam Marshall, Wednesday, 7 May 2014, 17:41

Moodle uses the Behat testing framework, which does automated functional testing (that's the type of automatic testing where it clicks on things in a browser window, in exactly the same way as if a user was doing it).

As a practical tool, this works great - yes, it's hard to install, only really works properly using Firefox at the moment, and is very slow, but these things are probably unavoidable.

I have two issues with it, so I'm going to write a rant. smile

1. Writing complex tests is very slow because there's no interactive mode.

Behat tests are written as a sequence of sentences like:

And I click on the 'span.someclass' 'css_element'

Each sentence is easy to write but there's lots of room for error. In this case, what if there are two elements on the page with that class? What if I got the CSS class wrong? What if I'm not actually on the right page that has that element yet, and I need to click another link first?

The answer to those 'what ifs' is that you have to figure out why it didn't work, change it, then run the whole test again. That's a familiar cycle for software developers, but in this case it's unfortunate because bearing in mind that Behat tests always begin from the home page of an empty Moodle site, there are usually a good few steps before you even get to the main part of your test (log in, go to the course page, create an activity, etc). As a result you probably have to wait at least thirty seconds before you can see if your change works - and if the problem is near the end of a complex test, it could be several minutes.

For anyone reading this who isn't a software developer: thirty seconds is a Long Time. You know when you just missed the bus and have to wait 25 minutes in the rain for the next one? That's about how long thirty seconds is to a software developer.

When writing a Behat test, what you're doing is taking an interactive activity (clicking on things in a web browser) and turning it into a script. That's fine, but it would be much more efficient if it were possible to actually do this interactively.

In other words, I could type my example sentence above into a Behat console; if it didn't work, rather than 'test failed, now you have to run it again from the start, loser' I would simply get an immediate 'nope' and be able to try a different variant of the command. And, with the web browser (that Behat uses for testing) conveniently open already at the right place, I could even use the web developer tools to check on the current page structure to see why it wasn't working.

Once you've got a test working, the current process is fine, but for actually developing it, an interactive console would be a huge advance. There's an issue about this here and a demo where somebody has made it here (link in demo doesn't work so I don't know how sketchy that is) but it doesn't seem to be a solved problem.

2. Somebody got all hung up on testing theory

There are theories about how you do this type of testing. It's strictly from the user's point of view. You don't write a test unless you can define what the benefit of that feature is for the user. Each scenario contains a single independent test. Yadda, yadda, yadda.

All of this is just so much meaningless junk and distracts from the really important thing which, in practice, Behat testing actually does. What it actually does is, in a repeatable and fully automated manner, carry out a sequence of steps and check that they have the expected result. So it's a shame it gets hidden behind Fully Agile Runtime Testing methodology, or whatever it's called, with everyone having to make up stupid prefixes at the top of the file like:

In order to take over the world
As a 37-year old vegetarian who hates dogs
I need to make sure the damn availability conditions page in Moodle actually works, okay?

That example could be included in a real test script in Moodle - I might have some effort getting it past code review, but other than that, it would work exactly as well as the real text in that test, which is to say, it would have no practical effect at all and be of no use even in documenting the test.

This is a waste of time. We already have a user interface feature and at this point, nobody should care why or who it's for. We want to know if it works or not. Maybe it needs testing from the perspective of different roles, maybe it doesn't; that's a detail that can be handled in the test. Somebody should have already decided it's useful (or if not it can be deleted) - that's a completely independent question from testing it. Do you need to pretend that your testing system is a part of your 'agile' requirements list? Nope. Completely independent things, much better kept separate.

You know what would be better? A documentation block at the top of a test that describes what is covered by the test. (Unless that's already obvious from the name.) Sort of like you have in other programming languages.

Just to hammer it home, imagine doing this in another programming language (and yes Behat scripts are a programming language), like PHP. Here's an example:

/**
 * In order to have users collaborate with each other 
 * As a teacher

 * I need to let Moodle APIs know the current version of the forum module.
 */

(That is, in 'agile' theory, the comment that should go at the top of mod/forum/version.php.)

The same type of issue applies to each sentence line, which starts with a word that's ignored, something like:

Given I follow "C1"
When I edit the section "1"
Then "Restrict access" "fieldset" should not exist

That first word (Given/When/Then) doesn't do anything, it's just part of the 'how you are supposed to express a test for no good reason because it fits somebody's theory' junk. And to make matters worse, Moodle have an actual coding standard requirement that you can't use it sensibly. Your scenario has to have exactly one 'Given', 'When', and 'Then' (you can have as many 'And' lines between them as you like). Back in reality we obviously want to check multiple things in a single test scenario because otherwise it would take a million years to do a Behat run instead of the mere 48,000 it takes at present, so this means we start every line with 'And'.

You know what would be better than these meaningless words? Bullet points.

- I follow "C1"
- I edit the section "1"
- "Restrict access" "fieldset" should not exist

Not sure that's allowed in Behat syntax, but it would be a nice improvement. Another nice improvement would be, you know, leaving out entirely the word that doesn't do anything.

-

Right, rant over. smile Behat does work pretty well, but the first issue makes it rather more painful to create tests than it should be, and the second just makes it annoying (at least if you're me).

Permalink Add your comment
Share post

Conditional availability improvements

Visible to anyone in the world
Edited by Sam Marshall, Monday, 24 Mar 2014, 22:54

Oh dear, it's been a very long time since my last blog post!

This is about a new enhancement I've been working on, hopefully for Moodle 2.7, to the conditional activity availability / 'restrict access' system. Basically it's a new API and quite a large change. The key improvements to Moodle out of the box are:

  • Better user interface.
  • Conditions can now be combined using Boolean OR and NOT as well as AND.

I made a screencast to demonstrate the user interface which I have attached below. For best results, make the video full-screen. Also, for some reason I don't understand, it won't start playing until it downloads the whole thing - sorry about that. And finally... drat! It won't play on this page unless you have an OU student/staff login. If you don't, then please right click or otherwise download the link in order to save it to your computer / device and watch there. Sorry about that. (I should report that to someone...)

Screencast (20MB .mp4, 8 minutes)Video player: screencast-availability.mp4

This isn't quite finished yet but it's very very close (a few days) and I wanted to get it out there. smile

It isn't shown in the screencast because I wanted to concentrate on the UI but another key advantage is that you can add plugins if you want to make availability conditional on something else.

Finally just to make clear, this does replicate absolutely all the functionality of the existing interface - you might have noticed there's no button for user profile conditions, but that's only because I didn't finish the user interface for it yet. It will be there. smile

Permalink Add your comment
Share post

Backup and restore shenanigans

Visible to anyone in the world
Edited by Sam Marshall, Wednesday, 23 Oct 2013, 11:21

This blog post is about the work that the Open University has funded (and I've coded) to improve backup and restore in Moodle 2.6.

I was going to talk about this in the Moodle developer meeting, but decided to skip my item because it isn't that critical for developers to know about, I had problems connecting to the technology, and the meeting would have run late. So I'm writing a blog post instead!

There were a number of problems that meant backup and restore was not reliable for large courses. These included:

  • Backup and restore didn't display any progress during the operation - you'd just see the browser waiting for a page to load. This wasn't a very reassuring user interface, and also comes with other problems:
    • If the backup or restore took longer than an hour it would fail due to PHP timeout.
    • On load-balanced/clustered systems with multiple webservers and a front-end server that shares load between them, the front-end server typically has a timeout of a few minutes if connections do not send any output; on these systems, even small backups would fail.
  • Before you get to the actual backup and restore, some of the interface pages can also take long time to load when you have a very big course. Again this is a bad user interface and causes problems for systems with front-end servers. In addition, because the default PHP time and memory limit for these pages was low, they would fail for courses with a large number of activities, even on a 'normal' Moodle installation with one server.
  • Backup files were limited to 4GB in size due to restrictions in PHP's support for the zip file format. (Some of our courses contain more than 4GB of files.) In addition, even where it created a larger file, it wasn't possible to actually download files larger than 2GB due to limitations of a function used in Moodle file download code.
  • If there are minor problems in backup or restore, the code may add a warning message in the backup/restore log. In Moodle 2.5 these logs are never shown to users, so this is not very helpful.
  • There wasn't a simple mechanism for developers to create a standardised large course for testing backup, so while we were seeing major problems in our real usage, it was difficult for developers to deal with the equivalent situation.

To solve these problems I made a number of changes, mostly listed under MDL-38189.

My basic approach was to start with the last problem - using the excellent 'generator' mechanism already in Moodle, I built a simple admin tool to create large courses for testing. The tool had options for different sizes ranging from XS to XXL (this last size I never tried; it may possibly be stupidly big!) with L designed to be similar to the largest courses on our system, and XL designed to be larger than those largest courses on our system.

After that I simply tried to backup, and then restore, courses at each size. The XS and S sizes worked immediately, but the M size failed. Once I got the M course to work I tried with the L course and finally the XL course.

The changes I made that correspond to the above problems are:

  • I added a progress reporting API inside backup and restore. This displays a progress bar and also a wibbler bar underneath it. The progress bar shows 'known' progress as a percentage (when the code knows this is step 47 out of 300) and the wibbler bar just wibbles (changes colour) to indicate that progress is happening when the code doesn't know how many steps there are going to be.
    • The PHP timeout is reset each time progress is displayed, so the backup/restore will never time out as long as something is happening and reporting progress.
    • Because HTML code is frequently output to the browser (usually once per second), any front-end server will not time out.
    • I had to change many areas of backup code (that can potentially take time) to use the new progress reporting API so that this works. I also had to change API in some other areas of code that the backup/restore uses (most notably the file packer) to add a mechanism for reporting progress in that area too.
    • You don't normally have to add progress reporting to individual plugins such as activity modules, if they handle backup/restore in the usual sort of way, because I already put progress reporting calls in the lower-level mechanisms that they generally use - for example in the restore, progress reporting happens automatically while it's reading the XML file with your plugin's data.
    • If you are writing custom code that calls backup and restore programmatically, you can add your own callback to get the progress information - you could use the standard 'progress bar + wibbler' display, or your own custom mechanism.
  • I added and used a similar progress reporting mechanism for the steps included in the backup/restore user interface pages before backup starts (e.g. preparing the large forms for selecting activities; unzipping the backup file at the start of restore). These too can display progress as the pages are prepared for display. To avoid unnecessary distraction, these progress bars don't appear unless it takes at least five seconds to prepare the page. (I also increased the memory limit for these pages so they don't run out of memory.)
  • I made a new backup format that supports files larger than 4GB - you have to turn this on via experimental settings. The files are still called .mbz, but if you turn on the new format, internally it uses the Unix standard .tar.gz compression format. (You can rename the file to .tar.gz if you want to unpack it using WinZip, or the Unix 'tar xf' command.) When restoring the system supports either type of file automatically, regardless of the experimental setting. I also changed Moodle file download so that it works to download large files. (In some cases this might require that the servers are running in 64-bit mode, although I'm not actually certain of that.)
  • When a backup or restore completes, if there are any messages in the log, these are displayed on the last screen of the backup/restore (the one which previously just had a 'continue to course' button). This gives the backup system and plugins a chance to report problems and may see more use in future now that the log actually displays to users.
  • As noted above, there is a 'Make test course' tool in the administration block within the Development folder for developers who want a large course to test against. Other people have enhanced this further since I started it, too!

What's next? Well, I'm not planning any more large changes. But first, it's still possible there might be regressions caused by some of this code which hopefully will be spotted before 2.6 release. And second, I expect there will be some areas of backup and restore which weren't exercised by my large test courses, and which can still potentially time out - so we might need to add progress reporting calls to more areas of the code that I didn't yet spot. The general principle is that if some operation might take longer than, say, a minute, it's probably necessary to find a way of calling into the progress API periodically during that operation.

Hopefully this is more than anyone wanted to know about the backup/restore changes. smile For most people I expect all they'll notice is that they'll get a progress bar when they do backup or restore.

Finally, thanks to all the developers who helped with peer reviews and integration reviews for the many changes related to this! And also to those who worked on (or are still working on) their own separate changes to improve it.

 

Permalink 3 comments (latest comment by Estefano Ramirez, Monday, 31 Aug 2020, 16:01)
Share post

OU release dates and what that means for our plugins

Visible to anyone in the world
Edited by Sam Marshall, Wednesday, 22 May 2013, 15:15

People frequently ask the question 'when will [an OU-developed Moodle plugin] be available for [a new or upcoming Moodle version]?' So I'm writing a blog post to cover it. Not very amusing I'm afraid...

In general, OU releases are about six months behind the community Moodle version (possibly seven months now the community schedule has been shifted). This is necessary to ensure reliability but also to give us time to update our plugins, while maintaining new feature development.

If you want to use a released, tested version of our plugins, you'll need to follow our release schedule. Or if you want to live life on the edge, you should probably find that the untested 'master' branch of our plugins will have been updated to work on the next Moodle version by about two months before our live release, i.e. about four months behind moodle.org.

This was probably hard to understand, so let me do headings and bullet points.

If you want a Moodle 2.4 version of our plugins...

  • (Moodle 2.4 came out on 3 December 2012.)
  • An untested version has been available in the 'master' branch of each plugin in GitHub since approximately the start of April 2013.
  • The tested version will be available in the MOODLE_24_STABLE branch of each plugin in GitHub at some point early in June 2013. (You can pester me online if I have forgotten to update it!)

If you want a Moodle 2.5 version of our plugins...

  • (Moodle 2.5 came out on 14 May 2013.)
  • An untested version will be available in the 'master' branch of each plugin in GitHub by approximately the start of October 2013.
  • The tested version will be available in the MOODLE_25_STABLE branch of each plugin in GitHub at some point early in December 2013.

Caveats

  • The above dates for our future releases are not definite and are subject to change!
  • You can extrapolate from this to Moodle 2.6 (June 2014 live date), Moodle 2.7 (December 2014), etc - although we might change our release pattern by then, who knows.
  • As soon as we release a new stable version, we stop supporting the previous stable version. For example, once we have a MOODLE_24_STABLE branch, there will be no more changes to our MOODLE_23_STABLE branch. If anybody wants to support these old branches, we suggest forking our GitHub repositories.

Occasionally asked questions

  • I want to use your plugin in a new Moodle version before these dates, does it work? No idea! Please try it. (Sometimes plugins intended for a particular Moodle version will actually work unmodified in the next version. Unfortunately there are lots of cases where this does not hold true.) If it doesn't work, you can feel free to file a GitHub issue, but we probably won't fix it sooner than those dates.
  • It doesn't work but I still want to use it before these dates, what can I do? Please fix it yourself and send us the patch. smile
  • I have a patch that fixes your plugin for the next Moodle version, do you want it? Yes we do! And thank you for fixing it. But because of how our release process works, the 'master' branch of our plugins has to refer to the release currently in development at the OU. For example, if you submit a patch now for Moodle 2.5, we won't be able to put it on the master version of the plugin until about the start of September 2013 (about a month earlier than the dates I gave above). Of course, if you've put your patch on a GitHub issue, then other users can still benefit from the patch sooner by applying it manually themselves.

 

Permalink Add your comment
Share post

OU plugins updated

Visible to anyone in the world

Boring post but for info, the OU plugins on GitHub (ForumNG, OU wiki, OU blog, subpage, etc.) have been updated today:

  • The MOODLE_23_STABLE branch is now based on our latest live code from our March release last week, which includes patches/bugfixes.
  • The master branch includes our latest fixes from our 2.4 development release.

So anyone waiting for a 2.4 version, you might like to try it again. We think most of them basically work now but there may still be some problems (especially subpage might only work under certain conditions). If something's still broken in 2.4, feel free to file issues in the relevant GitHub project.

Permalink Add your comment
Share post

OU Moodle plugins and 2.4

Visible to anyone in the world

Oh dear, it's my first blog post of the year. smile

There is probably something more interesting I could talk about, but we're getting a bunch of questions about 2.4 versions of OU plugins lately.

To recap the OU schedule:

  • We typically go live with a Moodle release about six months after it comes out, unless something goes wrong.
  • Development for our release takes a three-month period that ends about a month before our release.

In this particular case for 2.4:

  • We should have stable versions of our plugins for 2.4 (MOODLE_24_STABLE) in June.
  • We should have development versions (on the master branch) that work in 2.4 at some point between February (yes, I know it's February now) and the end of April.

Of course, some of our plugins from previous releases may continue to work unchanged - it depends whether any of the APIs in Moodle core were modified in a way that breaks our plugins in that release.

Where our plugins don't work in 2.4, feel free to file an issue on our GitHub site, but be aware that we won't fix it until our development period starts. If it's a simple problem and you want to fix it for us by submitting a patch, please do - we'll try to apply these patches early in the development period, i.e. more like February than April. (We can't apply them before the development period starts though, sorry.)

Permalink Add your comment
Share post

Moodle filesystem size and repositories

Visible to anyone in the world

Since I haven't written a blog post for a while, thought I'd do one about the small project I just finished. This is something we're not particularly intending to release publicly because it's probably not useful to anybody else, but it might be slightly interesting to other large-scale Moodle users. Or not. Let's see.

Basically we had two problems with our Moodle system that this is trying to solve:

  • People host large files (e.g. 1GB) on the system, but Moodle isn't optimised for serving large files; this isn't causing a problem now but we're concerned it might do in future.
  • Our Moodle filesystem is quite large and rapidly increasing in size. Certain maintenance operations, like when we take a copy of the entire system onto our acceptance test servers each time we test a release, are getting slower.

The OU happens to have an EMC Atmos file store, a commercial product which is obviously designed specifically to store files, so the suggestion was that we should move large files out of Moodle and into Atmos. That way we don't have to include them in the copy for acct (we'll give our acct system read-only access to the filestore so they will 'still be there') and serving the files can also be passed off to Atmos.

Conveniently, there is a new feature in Moodle 2.3 which allows files to be stored in the Moodle filesystem by 'reference' - instead of storing the actual file, it stores a reference to an external repository system.

Using this feature, I built a repository that, during cron, finds all files larger than 10MB. It checks (based on contenthash) to see if they are already in the Atmos system and, if not, transfers them to Atmos, deletes them from Moodle and replaces with the reference. Then, when users click to download a file, it redirects to an Atmos shared link (complete with security key to prove they can access it; we don't lose security by this process, it only happens after the Moodle security checks).

This wasn't very difficult; one of the 'gotchas' is that you have to make sure you set your reference's content to nothing (empty string); if you let the reference keep its original contenthash then Moodle won't delete the file.

A few things in Moodle meant the implementation is not quite as nice as it should be. First there's no way to have a repository that only lets you create one instance, or only lets you create it at system level. I hacked it so that in the creation process it throws an error if you try to create two. Second I couldn't find a way to stop this repository showing up on the file picker for admin users. For everyone else, it doesn't show because I made a capability and didn't give it to anyone.

Overall, though, neither of these are serious problems and the process appears to work well. Although I should admit - I finished the code but it hasn't been through testing yet. smile

Just by the way - this is only part of our 'filesystem too large' solution. I think this will take out about a third of the filesystem size; another third will come by removing course backup files. Those can be huge and they appear to persist forever; we'll probably do a local plugin that deletes them after a month or something. (We don't use those files for backup purposes.) So anyone concerned about filesystem size should also check how many backup files there are in course/user areas - for many of our courses, one backup can be well over a gigabyte, so these add up quickly.

Permalink 2 comments (latest comment by Sam Marshall, Tuesday, 8 Jan 2013, 16:31)
Share post

News block

Visible to anyone in the world
Edited by Sam Marshall, Thursday, 12 Jul 2012, 20:28

We've just released a new plugin for Moodle 2.2+. (Yes, '+': purely by coincidence, this one already works on Moodle 2.3!)

It's a block that displays news messages, and we've called it the news block. (Not a very original name. Sorry. We did consider 'the boggleogglesquoop block' but it didn't fit in the dropdown.)

This is a replacement for the standard Moodle 'news forum + latest news block' combination.

I made a short (5m) screencast demonstrating how to use the block (running on standard Moodle 2.3). Assuming I haven't run out of bandwidth, you might as well watch the screencast; there aren't any jokes but at that length you can probably survive it anyway.

Features include various options for how messages display (e.g. hiding the author name, if the author isn't important), setting up messages in advance to appear on a certain date, multiple news blocks on the same course if you want to separate different types of news, grouping support so that some messages only appear to a subset of students, and integrated RSS input and output. Combined, you can use the RSS features to post shared news (for example, have a science faculty news block on one course, then include those messages automatically in news blocks on each of a dozen science courses).

The block was originally designed by me; we paid for development by Catalyst. After that initial development we've done minor changes in-house (bug-fixing etc.), and Anthony Forth, another OU lead developer, added grouping support.

To get the block for your own system, download it via the GitHub site.

Disclaimer: the block probably hasn't been much tested on systems that are different to the OU's. I recommend that you try out any new plugins on a test system before using them in production. Unless you really like downtime. smile

Permalink 4 comments (latest comment by Sam Marshall, Monday, 6 Aug 2012, 10:33)
Share post

New blog post

Visible to anyone in the world

Just an update for any other sites using the Open University's Moodle 2 modules that I am responsible for, such as ForumNG, OU blog, OU wiki, and subpage.

Currently these modules are available and maintained for Moodle 2.2, with stable versions updated monthly (and occasionally at other times) when there are changes. In order to get the updates, you have to grab them from our GitHub site as I do not have time to do the manual update process for the Moodle plugin site.

Moodle 2.3 is about to be released but the OU will not be using it immediately. There are  changes in Moodle 2.3 which break some of our modules, especially ForumNG. If you rely on our modules and don't want to do without them, it would be advisable not to upgrade to 2.3 until we do.

I don't promise dates and these might change but I would expect that we will have 2.3-compatible but untested versions of these modules in our code repositories at some point during September. It looks likely that our first 2.3 stable branch will be in early December.

Before then, if people wish to submit patches (using our GitHub site) to fix problems so that the modules do actually work in 2.3 earlier, you could do and if it seems safe I will include it. But please test the patch fully against 2.2 before submitting. smile

Permalink 9 comments (latest comment by Sam Marshall, Tuesday, 5 Mar 2013, 10:50)
Share post

New blog post

Visible to anyone in the world
Edited by Sam Marshall, Friday, 2 Mar 2012, 15:03

This is a brief blog post to let ForumNG users know that I've now, finally, set up a stable branch.

MOODLE_21_STABLE branch at GitHub

This branch will shortly (from Tuesday) match the version on our live servers and should continue to do so until we stop using Moodle 2.1 (which is currently scheduled for April 6th). After that it becomes unsupported and we'll hopefully have a MOODLE_22_STABLE branch.

You can still use the 'master' version of the code, if you want our current untested development version. As of just now, there is a change to the master version which fixes a critical Moodle 2.2 bug (email sending was broken), but unfortunately also means that the master version won't work on Moodle 2.1 any more.

So to summarise:

  • We now provide a MOODLE_21_STABLE branch which is the same as our live servers so should be stable and reliable and will receive critical security fixes. It does not work on MySQL.
  • As of today, the master branch requires Moodle 2.2 or above. It does work on MySQL - thanks to Brian King for providing the answer to the problem from my last post. smile

If you really really want a version that works on both 2.1 and MySQL, I guess you can grab the relevant commit from GitHub (i.e. the one just before the commit where I say it requires 2.2).

Permalink Add your comment
Share post

New blog post

Visible to anyone in the world
Edited by Sam Marshall, Tuesday, 21 Feb 2012, 14:37

Any MySQL + Moodle experts out there?

Somebody's reported a problem with ForumNG on Moodle 2.x, and I believe it's likely a MySQL problem. There is a really complicated query with several subqueries and the database error is something about f.id not existing (which it blatantly does).

CONTRIB-3480

If anybody fancies installing ForumNG on their Moodle 2.x/MySQL test system, please let me know in the above tracker issue (a) if it works/fails for you (confirm version numbers please?), or (b) if you can tell me what's wrong with MySQL and/or my query, and what I could change to make it work.

I do not currently have a MySQL test install so I would prefer 'I have tried this and it fixes it' type answers rather than 'You could try this, it might fix it...' smile

I did already look at the MySQL documentation because I know there is a hideous limitation in a slightly similar situation (if you have subqueries in a DELETE, they can't refer to the table you're deleting from? Something like that), but I didn't find anything that obviously seemed to apply to this.

(I've disabled comments on this blog post, please add suggestions directly to the tracker issue.)

Permalink
Share post

New blog post

Visible to anyone in the world

By special request, here's a super-boring post on the ForumNG 'features' plugin infrastructure! (Not a developer? Look away now.)

It's very simple: you stick standard Moodle subplugins in the 'feature' folder. Each one must have a class which extends either mod_forumng_discussion_feature or mod_forumng_discussion_list_feature (which themselves extend mod_forumng_feature).

The difference is that 'discussion features' will be placed at the bottom of the discussion page, and 'discussion list features' just below the list of discussions on the main forum view.php. Discussion features are passed a $discussion parameter for a bunch of things, and discussion list features are passed a $forum parameter.

The classes for this basically have three methods that probably need implementing: get_order (returns an integer indicating the order relative to other features), should_display (default returns true if you have moderator permission) and display, which returns some HTML.

This doesn't really add up to much; it just means you can put a button (or pair of buttons, or dropdown, or whatever) in the area below the list of discussions or at bottom of a discussion.

Then you can put the code for handling actions in PHP scripts inside your feature.

That's about it except there are some helper functions/classes to make certain things easier (making the button form; plus for example if you want to use the 'post selector' feature which integrates with a lot of built-in JavaScript as well as non-JS alternative code, so that you can let users do some action [export or forward by email] on whatever posts they want).

There's a catch here, which is - well, let's take the 'delete discussion' feature as an example. So there's a 'delete' discussion feature and it provides the button you use to delete (or undelete) a discussion, and the script that runs when you click on it with the 'are you sure' prompt, and it checks permissions and stuff - but it doesn't really do the actual work for deleting a discussion because that clearly ought to be, and is, part of the forum back-end in the mod_forumng_discussion class. (Note: If you want to look at the code for a forum feature, the delete one might be a pretty good place to start.)

That's also true for a lot of the built-in features: the 'feature' architecture actually only includes the UI, and the real back-end work is done inside the core class, because it sort of has to be that way.

But it does still allow some degree of separation.

One of the reasons I wanted it is that ForumNG at the OU includes this totally useless 'show who read this discussion' function. Supposedly this shows you who has read a discussion and when, but in fact it's impossible to tell. So the list, though sort of correct, is basically a lie. I had to do it because they made me. smile But I didn't want to include that in the public version; no reason to spread our bad practice around the world. The 'features' system meant I could just exclude that one folder from the distribution without changing anything else.

Note: An obvious extension would be to add mod_forumng_post_feature class but we did not find a need for this yet - possibly because the 'post selector' approach (where you click a button at discussion level, but then can select one or more posts with checkboxes) is more flexible.

 

Permalink Add your comment
Share post

New blog post

Visible to anyone in the world
Edited by Sam Marshall, Tuesday, 7 Feb 2012, 18:32

Important note: This is just my personal opinion as a developer. Nothing in here relates in any way to the Open University's official position on any of these matters!

Internet Explorer 6. Three words that strike fear into the spine of any web developer.

As part of our move to the new Moodle 2-based system for some of our module websites, we agreed - well in advance - that we would no longer support IE 6. Google dropped support in 2010, Microsoft run a website encouraging people to stop using it... it's game over.

Why did we drop it? Well, anyone reading this who isn't a web developer might not be aware, but IE6 isn't just equivalent to any other old browser version you might encounter, like say Firefox 3.6 or something. IE6 behaves radically worse than every other browser in terms mainly of its CSS (layout) and JavaScript (interactive) support. If you want things to work in IE6, you have to do a significant amount of extra work in testing and coding workarounds to all the problems. We would rather spend our development time improving the system for 98.5% of users, not fixing problems for 1.5%.

So, we don't support IE6. That means not only do we not test with it, but if anyone reports problems we don't fix the problems, just tell them to use a different browser. And because we developed a new theme without supporting IE6 from the very start (unlikely our previous system, where we fully supported IE6 at the time our theme was originally implemented), it was likely there would in fact be problems.

Turns out that in fact, the main portion of the page doesn't display at all in IE6. It's blank. Oh well, whatever. We don't support it, right? Not a problem.

Until last week when everything hit the fan. Specifically, it turns out that certain government institutions are currently still using IE6, and preventing their staff from installing other browsers. And those staff often don't have the opportunity to simply study at home instead.

Which institutions?

The ones with guns.

Ooops.

I could wear a bullet-proof jacket, but that wouldn't solve the real problem. Which is, we don't want to support IE6, because it costs a lot of developer time. We can't very well sort of half support it; if we support it then that means we have to test everything in IE6 because it's not like any other browser (we don't test everything in Opera either, but that usually just works) - most things probably won't work and will need workarounds. We'd lose all the benefit of ditching support.

But then I had an idea. I suggested it to my bosses and they were okay with it, so we went ahead.

IE6 is a disability. These students are being forced to use an ancient browser; it's kind of like they're being forced to walk down the street blindfolded.

We support blind users. Why not support IE6 the same way?

There's no problem with IE6 as a basic HTML browser. If your website has no style and no interactive features it will work fine. Coincidentally, this is roughly the same way that blind users (through screen readers) experience the internet. So we've already made our sites work if you don't 'see' the styles or the interactive features.

So if you're using IE6 and you look at our new module web sites system (not this one), you'll now see a totally plain screen - we've stripped out all the styles and interactive scripts. It's just like the experience you get with a screen reader - complete with skip links. Not exactly pretty (and we still don't officially support IE6), but you can access all, or nearly all, of the content.

Image Hosted by UploadHouse.com

Because I did this at a theme level, it should apply in general. We don't need to test in IE6 - if we build a new feature, as long as it works with JavaScript turned off and no styles (which it ought to, for accessibility) then it should be fine in IE6 too.

That's all. I thought this might be interesting for other people struggling with IE6 issues. There was an embarrassing/amusing screw-up when we deployed this change, but this post is already long enough so I think I'll leave it out. smile

PS To avoid confusion, I should make clear: there are many OU websites which still fully support IE6. This post is only about the new system for module websites, which doesn't.

Permalink 5 comments (latest comment by Sam Marshall, Thursday, 9 Feb 2012, 10:10)
Share post

New blog post

Visible to anyone in the world
Edited by Sam Marshall, Tuesday, 7 Feb 2012, 17:50

Update on my previous post: three out of the four issues (all except media embedding MDL-29624) are now resolved and included in the relevant upcoming Moodle versions. Yay! Thanks to all Moodle HQ developers and other people who assisted with getting these approved and included.

This is a mini-post. I'm going to do a real blog post next.

Permalink Add your comment
Share post

New blog post

Visible to anyone in the world
Edited by Sam Marshall, Friday, 13 Jan 2012, 13:47

For my first blog post of the year, I thought I'd just give a brief rundown on the current issues (features and fixes) that I've coded and are waiting in the review process for core Moodle. I'm afraid this isn't a barrel of laughs, but at least it's informational.

Admittedly this is partly because I want to encourage HQ developers to review things, but also partly because I thought other people from the Moodle community might be interested in some of these changes! Feel free to vote on issues if you like them. smile

All these issues are ones seen as vitally important by our staff for various reasons.

Features (for 2.3):

  • MDL-29624 Media embedding should be consistent and customisable
    At the moment, there are three totally separate mechanisms for embedding audio/video/Flash: the media filter, the File module, and the 'preview' window used when uploading video files. The latter two cannot be customised by any plugin.
    This change standardises it so that all three places use the same system, and also makes it so that the embedding can be customised by writing code inside a custom theme. For example, if you want to change the Flash player used to embed video, or to support different formats, this would become possible without having to change core Moodle.
  • MDL-31121 File resource: option to display size, type
    A couple of tickboxes on the File resource form; if you turn them on, then on the course page where it shows the link to the file, it also shows e.g. '24.1MB PDF document' in small text.

Bugfixes (for 2.1+):

  • MDL-31122 Navigation block: hide weeks that just contain Label
    The navigation block already does not include course weeks that have no activities, because that would be stupid (you can't navigate to them). But, currently it does contain weeks that have a Label and nothing else. This change fixes that.
  • MDL-31015 File/URL resource: 'Open' option doesn't work reliably
    If you choose 'Open' display type on a File or URL resource (meaning, open the file or go to the link immediately), this only works when the link is clicked from the main page. If you click it from navigation, ctrl-click to open in new tab, or open in any other way, you get an intermediate page with the link on, which you then have to click again to get the file/URL.
    Users really hate this because it's like 'I clicked a link to go to the URL - now it's telling me to click a link to go to the URL! that's what I just did!'
    This change simplifies the code (removing a chunk of JavaScript) and makes it so the 'Open' display type always works, wherever you come from.

Exciting, no? Okay, no. Still, now you know.

Finally, just as a reminder - if anyone uses any of the OU's custom modules for Moodle 2, please do make sure to keep up with the relevant GitHub repositories in case there are bugfixes that are important to you. (If you do upgrade as a result, don't forget to check the new version on a test server first.) And incidentally, developers working for the company NetSpot recently submitted a few bugfixes to those modules too, so thanks for that. You can see their contributions in the git history.

 

Permalink 2 comments (latest comment by Sam Marshall, Monday, 30 Jan 2012, 10:11)
Share post

New blog post

Visible to anyone in the world

This is a special super bonus blog post. Not only are there two blog posts in one day, but this second one has two screencasts in it!

In today's Moodle developer meeting, I suggested (rather rudely, sorry) that the OU's subpage module was a better way to handle the 'everything in Moodle course has to go on the same single page which is stupid' problem than implementing some kind of 'show things on separate pages' feature in every single course format.

Subpage - quick demonstration of subpage using the standard theme so it looks pretty much like standard Moodle (about 5 minutes)

Subpage in real life - how we use subpage at the OU (about 3 minutes) - you get to see our pretty theme in this one

Here are some reasons I think subpage is a good approach:

  • Not dependent on course formats, or inconsistent depending on which format you're using.
  • Obvious; easily understandable by students and staff. (Except the name! The name sucks, but I couldn't think of a better one.)
  • You can keep using convenient views that take advantage of the course format to see the entire structure of the course at once (only it won't be ridiculously long now), or to show N weeks around current, or whatever it is.
  • Generally more flexible - for instance, want to nest pages within pages? With subpage, you can. (It's probably a bad idea, though!)

There are some problems with the way subpage is implemented; basically, it's a leetle bit of a hack. In order to make it un-hacky, some of the following things would need to happen to core:

  • Add all the 'come back to this URL afterward' features that we included in a short patch.  Without that, in core Moodle every time you do anything in a subpage you end up back at the course page. (For instance, add a forum? Okay, great, but after it's added you're back on the course page.) It isn't really practical to use without this patch even though it does basically work if you struggle through.
  • Implement something (ownercmid?) in the sections table to indicate that they're owned by a module and therefore take them out of the ordinary numbering. Apply some changes to backup and restore and navigation and formats to make this slicker. (It works OK without, we're using it, but we do have a few special things in our course format to handle it as well...)
    Note that this change would let authors create other modules that can include sections/Moodle activities, which could actually be pretty cool.

Subpage code repo (...may or may not work at the moment, I hope it does though).

Permalink 8 comments (latest comment by Sam Marshall, Tuesday, 16 Apr 2013, 17:22)
Share post

New blog post

Visible to anyone in the world

It's slightly out of date browser stats time!

Here's our browser stats for the end of last month from our Moodle 1.9-based system (...this one!) which is responsible for the vast majority of our current course ('module') websites.

Oct 2011 browser stats (Google Docs)

This includes all off-campus access (so basically, all students and tutors, but not internal staff such as academics and me) during a week at the end of last month.

The counting method is based on page views not on individuals or IP addresses, so if you use a mobile phone to access the system occasionally, but you mainly use IE9, then (when combined with everyone else) these proportions will be replicated in the stats. And if you use the system a lot (maybe you're a very active student studying several courses) you will be counted more times than somebody who only checks their course homepage once a month.

If you saw the last lot, you might be interested as to why IE7 has dropped precipitously (from about 13% to about 6%). That's because there was an error in the stats program before and it was incorrectly counting IE8 and 9 users, who had selected compatibility view, as IE7 users. (Our site forces these browsers to behave as their real version, but the user agent still has the wrong one in. However there is a way to tell.) I recalculated the old numbers too and the trend graph shows the kind of gradual decline you'd expect.

Here's hoping most of the people who are genuinely still using IE7 get new computers for Christmas. smile

Permalink Add your comment
Share post

New blog post

Visible to anyone in the world

(Of interest to Moodle 2 users at other sites, developers etc.)

I've just submitted another minor administration feature for Moodle 2.2. It makes consistent across most parts of the system which user fields are displayed in lists, and lets you choose from a selection of fields; the default is to display email (same as current behaviour in most places, just a bit more consistent) but you can change it to show, e.g., idnumber and department if you like. Also adds security control for the feature.

MDL-26647 - has more complicated info and screenshots

As I've only just submitted it, I don't know for sure yet whether the feature will be accepted by HQ (possibly with revisions, of course). I really hope it is. We need it at the OU because administration is extremely difficult when you are trying to e.g. add somebody to a course, and you can't tell which of the ten Sam Marshalls in our database you should actually add!

It's actually top of the priority list I got from the person in charge of managing our transition from VLE1 (based on Moodle 1.9) to VLE2 (based on Moodle 2.1+), in the section labeled 'Hyper important'. (The other two sections are 'Super important' and 'Very important'. I found this amusing.)

I was also pleased and slightly surprised to see that there are 19 votes for the feature, only 17 of which are from me under various aliases*. That puts this in the top 100 most requested Moodle features. Hopefully that increases my chances. smile

* That was a lie! Honest... ;)

Permalink Add your comment
Share post

New blog post

Visible to anyone in the world

I haven't done a public blog post for a while, so here's one! It is mainly of interest to anyone using our add-on modules (ForumNG, OU blog, OU wiki); apologies for general boringness.

The OU is now live with our VLE2, based on Moodle 2.1.x, on a couple of courses. So it's a slowish launch but trust me, there have still been plenty of panics along the way. smile

On an entirely unrelated matter, if you are using the beta versions of OU modules for Moodle 2, especially ForumNG, it would be a good idea to upgrade to the latest version from our github site - after testing it on your system first of course. Just saying.

Seriously - I do have a system in place to transfer single commits from our system into our GitHub public repositories now, so you can actually see what hideous bugs got fixed. The commits do reference numbers in our own bug tracking system, which is internal so you can't see it, but you can read the summary and see the code changes. This might be enough to achieve most of the comedy value, or alternatively to make an informed decision about including the same code patch in your own servers.

It would be a good idea to get a GitHub account if you don't already have one and 'watch' the repository if you use one of our plugins (or use the Atom feeds or whatever else they provide) so that you can see each change as it comes in, just in case it might be important to you

At present, the version on our repositories is what we're still calling a beta - it is typically ahead of the live server, but as we are currently in a rather wild-west deployment phase, not by much! There will be a point where we switch to basing the repositories on our stable branches so that only critical bugfixes appear there. I'll no doubt blog about that at the time. smile Until then you should be extra careful about testing before you upgrade.

 

 

Permalink Add your comment
Share post

ForumNG, OU blog and OU wiki update

Visible to anyone in the world
Edited by Sam Marshall, Friday, 9 Sep 2011, 12:48

Today I've done the first proper release of three Open University contributed modules for Moodle 2.2. You can get all these from our public GitHub repository site:

https://github.com/moodleou

Click into the appropriate repository:

  • moodle-mod_forumng (ForumNG)
  • moodle-mod_ouwiki (OU wiki)
  • moodle-mod_oublog (OU blog)
  • moodle-local_ousearch (OU search back-end, supports all three)

Within each repository you will find a readme (scroll down to see it) with information about the plugin. To install a module, use the 'Download' button; unpack the zip file; rename the folder inside the zipfile (e.g. rename 'moodle-mod_oublog' folder to 'oublog') and put it in the right place in your Moodle install.

I have also gone through the somewhat tedious process of adding these to the new Moodle plugins repository as well, so you can probably also get them from the official site instead (this may save you the rename step mentioned). At time of writing, they haven't been approved there yet, but I'm sure that will happen soon.

Important cautions:

  • These are now considered beta versions (as opposed to the alpha versions I put out before). They have had some testing, but still not (quite) live use with students.
  • Nobody's lately checked them on a standard Moodle install or when using MySQL as a 'database'. (OU moodle is now actually pretty much a standard Moodle 2.1.x, just with a huge stack of plugins added, so hopefully it should be fine, but.)
  • We can't offer individual support. Please don't contact me if you have problems with them - try to get peer support from suitable moodle.org forums (it's possible I might reply there, if I have time). However, if you find a bug, please do feel free to report it in the appropriate area of the Moodle tracker.

I hope we will release release-quality versions at some point after our next mid-October release so that from that point, these publicly released versions can be the same as our live ones.

Permalink 9 comments (latest comment by Sam Marshall, Tuesday, 31 Jan 2012, 14:58)
Share post

Moodle 2.2 - Show descriptions

Visible to anyone in the world

I made a screencast about a new feature I wrote that will be part of Moodle 2.2.

Moodle already has the ability to write descriptions for a resource (link, file, ...) or activity (forum, ...), but these display on a separate screen. Using this new feature, you can choose to display descriptions directly under the link on the course main page.

View the screencast on screencast.com.

Thanks to Moodle HQ - Eloy (code review), Martin (approval, suggestions), Helen (suggestions, wording), Sam Hemelryk (testing). Also to Tim for helping with communication while I was away. And a few other people were involved as well - see the issue MDL-27001 for full details.

A bit of background about this feature - we particularly need it at the OU because in our 1.9 system we had the 'resourcepage' module which let people create pages with lists of files and links for students - and those files and links could have descriptions. Moving to 2.x, we replaced resourcepage with a new 'subpage' module - this lets you do a similar thing, but the key is that it now uses completely standard Moodle resources and activities, just like the course main page. So we had a problem when converting a course from one to the other: the descriptions disappeared. Ooops. Now it should be okay. smile

You could achieve something similar already by adding labels in all over the place, but the spacing is wrong (the system doesn't know your label is supposed to 'go with' the thing above it, compared to other labels that don't, so the space between is too much) and it's a pain if you ever have to move items (as you now have to move both the link and the label below it).

So, I think the feature should be useful for lots of people - not just us.

And by the way, using this feature doesn't reduce performance when viewing the course page, because when you turn on this option the descriptions are stored in the course 'modinfo' cache.

Permalink 3 comments (latest comment by Hartmut Scherer, Tuesday, 13 Sep 2011, 09:08)
Share post

This blog might contain posts that are only visible to logged-in users, or where only logged-in users can comment. If you have an account on the system, please log in for full access.

Total visits to this blog: 244481