I'm writing a blog post about the cron system in Moodle 4.2 because I wanted to understand all the settings!
Cron overview
Normally, you set the cron script to be launched once per minute. These runs can happen in parallel (one keeps running while the next one starts), which is the only way that multiple tasks can run at once. So if you have just started the server, then only one task can run, but a minute later there will be another cron launch and now two tasks can run, and so on.
Processing within a single cron runner
1 If we are not already at the concurrency limit for scheduled tasks (task_scheduled_concurrency_limit - default 3):
a Get the next scheduled task and run it
b Repeat from (a) until there are no tasks due or we hit the scheduled task runner lifetime (task_scheduled_max_runtime - default 30 min)
2 If we are not already at the concurrency limit for adhoc tasks (task_adhoc_concurrency_limit - default 3):
a Get the next adhoc task and run it
b Repeat from (a) until there are no tasks due or we hit the adhoc task runner lifetime (task_adhoc_mask_runtime - default 30 min)
3 If we haven't already reached the keep alive limit (cron_keepalive - default 3 minutes), wait a second and repeat from (1).
Implications of default settings
No tasks
If there are no tasks to run, on default settings there will usually (after server startup) be 3 instances of the cron runner at a time, because of the 3 minute keep alive. (The one that started 3 minutes ago has just ended; the one that started 2 minutes ago and the one that started one minute ago are still running; a new one has just atarted.)
Each second, all 3 tasks will acquire the scheduled lock, check for scheduled tasks, release the lock, acquire and adhoc lock, check for adhoc tasks, release the lock.
Infinite tasks
If there sre infinite tasks of both types, on default settings there will usually be 6 instances of the cron runner (=total of concurrency limits). At any given time, three of the instances will be dedicated to scheduled tasks (because of the task_scheduled_maxruntime) while three will be dedicated to adhoc tasks (because of hitting the scheduled concurrency limit, then running until the task_adhoc_maxruntime).
Server startup with infinite tasks
On server startup, at default settings it will be 3 minutes before any adhoc tasks are processed; the first three runners will be dedicated to scheduled tasks, then the next three will be dedicated to adhoc tasks.
Impact of changing settings
task_scheduled_concurrency_limit, task_adhoc_concurrency_limit
These are the biggies. Be careful with these settings.
Low: If you turn down these settings, the cron server will require less server memory at busiest times, and there will be less database/storage impact rom task processing. This could be important because some tasks (GDPR, backup) can occupy quite a lot of both resources.
If you set it low you need to be careful you have also applied other limits to individual tasks (by configuring scheduled tasks to run only when needed, and by seting the limits so that adhoc tasks don't run too many at once, e.g. $CFG->task_concurrency_limit_default = 1 or something more specific). Otherwise it's very easy for a task to eat all the concurrency meaning no other tasks of that type can run. For example, if a GDPR data export task takes 12 hours to run, and you have concurrency limit of 2, then 2 of those would eat it.
High: You need to make sure the cron server has enough memory to cope, and the database can cope with the processing without reducing performance for student requests..
Overall: Depends on your cron server performance, especially memory, and also how heavyweight the tasks you normally run are.
task_scheduled_max_runtime, task_adhoc_max_runtime:
Low: If you set these settings lower it will stop a single task runner from being dedicated to a single type of task (scheduled for 30 mins if there is a slot available, then adhoc for 30 mins) in cases where there are lots of tasks to run. Instead the task runner will alternate between each type of task more quickly.
This will mean it keeps acquiring and releasing locks, so there might be a slight performance cost.
I'm not clear there is any particular benefit in doing this.
If you set these settings very low then it might be harder to reach the concurrency settings. For instance, if you set these to 1 minute each and cron_keepalive at default 3 minutes, and there are no long-running tasks, then the task runner will always exit after 4 minutes (1+1, 1+1), so there cannot be more than 4 tasks running in parallel. (In practice if you need many parallel tasks then you probably do have long-running tasks so this might not be a problem.)
High: If you set the settings very high then it will keep a single runner operating for a long time (up to the sum of both settings). This is probably a bad idea because you might as well exit the PHP process periodically (it could clear up any memory that got allocated, any static variables that got set, generally anything that might get encrusted). There's another one along in a minute!
You don't need to set this high if a single task takes a long time, the task runner won't exit mid-task anyway.
Overall: The default is probably OK but a lower value like 10 minutes (= max runner lifetime approx. 20 minutes assuming there are no long-running tasks) might make more sense.
cron_keepalive:
Low: If you reduce this setting it will mean fewer task runners waiting when the system is idle. For example, if you set this to 60 seconds, there will usually be just one cron runner (if there are no tasks to run). This is both a sliight performance advantage (less locking etc) and also a disadvantage in terms of how quickly the system can scale up to run multiple tasks when busy, bearing in mind it can only add one parallel task per minute.
For example, if the system is idle with one task runner, and then at least three scheduled tasks totalling at least three minutes of work become due at once, no ad-hoc tasks will run for the next three minutes as the existing idle runner and the next two task runners will be occupied with scheduled tasks.
If you set to less than a minute, then even if the system is idle, users will have to wait for the next minute for an adhoc task to run, rather than it happening immediately
High: If you increase the setting then more task runners will wait idle. I don't think there is likely to be very much of a case for increasing it beyond the default though..
Overall: You probably want it set to at least 60 seconds. If there are no big concerns over delay in parallel tasks starting, then 60 seconds is probably sufficient, otherwise maybe it needs to be higher.
It's a long time since I wrote a blog post but I thought I should make an effort to talk about the global search enhancements that I worked on for the Moodle 3.4 release.
At
the Open University, which has a large Moodle installation with many
custom plugins, we are now beginning to use the Moodle global search
facility (replacing internal alternatives). This is happening very
gradually and we intend for students not to notice! (If you're an OU
student or staff member, the first place you'll see it is on subject
websites, probably fairly early in 2018. But it will look almost the
same as our current search.)
This
is good news for other Moodle users because the current global search
doesn't seem to have been all that widely used at scale yet so people might be a bit hesitant about adopting it. We've found and fixed (or helped fix)
several problems with it - in terms of how easy it is to manage, and how
well it works. These bugfixes are included in Moodle 3.4, and although
I'm not suggesting it's perfect now, if you are another institution
thinking about using global search, Moodle 3.4 might be a good time.
It's a pretty neat system!
Here are some specifics about the core changes I implemented which are relevant to institutions implementing Moodle global search.
As usual, thanks go to the many HQ developers, and others, who have assisted significantly with these changes! There are also other search fixes in 3.4 (for example Matt Porritt at Catalyst has improved indexing to include more files that weren't previously indexed), which I'm not listing.
Any content in the HTML block is now searchable (provided the block is added to a course page). Also, developers can add support to search their own custom blocks - useful if there are cases where lots of content is provided in a custom block.
When you backup and restore a course, any content in the restored course probably has modified dates in the past. As a result, it was not previously indexed, because the search system only looked for content newer than its index. So if you needed to back up and restore courses (for example when creating a new copy of the course for each cohort) then the new ones wouldn't return any search results! We've added a rather complicated new mechanism so that everything in a restored site is indexed regardless of date. (This also means developers now have a way to re-index a specific context if necessary.)
Unless you are a system administrator, the search system used to search only courses you are enrolled in. Now it also works on courses you can access for other reasons: because it has been configured to be open to everyone without login, or because your role has been overridden to have the moodle/course:view capability. This was really important for our systems as we have many courses that are open to all students.
This is an option, which you need to turn on using the the Manage global search screen, because in some unusual cases it might make search results worse for your users.
There are a few situations where content in Moodle might get a time in the future This content was previously indexed, but that would prevent any other content added between now and that future time from being indexed. We've changed it so that content dated in the future is not indexed until that date arrives.
The Solr connection previously ignored proxy settings; if your servers are behind a firewall and need to use a proxy to access the Internet, and if the Solr server is hosted on somebody else's computer, outside your network, then it didn't work. This now works.
However, you should be aware that during indexing, there will be a very large number of requests to the Solr server - it might still be better for your infrastructure if you can configure it to allow those requests through the firewall directly, and add your search server to the proxy bypass option.
This project isn't over. We have plans to add more enhancements, some of which will make the interface better for students. With thanks to everyone who helped so far, I hope we can get some more changes in for Moodle 3.5. :)
We are now developing against Moodle 2.9, which means the new 'AMD' method for including JavaScript in Moodle plugins is available to us. (I mentioned this in my last blog post.)
The good news is that it's a lot simpler than previous methods for including JavaScript. The bad news is that it's still a change that developers need to understand, and the Moodle documentation is a bit hard to follow.
So I gave a short introductory presentation for OU developers about how to use this new method. And In case anybody else would be interested, I've attached my slides here as a PDF.
A few details mention OU things but most of it would be applicable to all Moodle developers. (To be clear, this is an introduction and doesn't get into advanced details like performance or whatever.)
This post is strictly for Moodle developers. If you're not a developer you can stop reading now as this is entirely technical - sorry about that. :)
In Moodle 2.9 the preferred way to write JavaScript is using jQuery and a module framework called AMD. This is documented in the Javascript Modules page of the Moodle developer documentation.
So what if you're writing a new plugin now, which currently has to run in 2.8, but you'd like to update it for 2.9 later with minimal work?
I did a hack to make this work (sort of). In case anyone else is interested, here's how...
1. Wrote my JavaScript using the 'new' approach, so that it begins:
define(['jquery'], function($) {
(For the purpose of this development, I can't currently specify any dependencies other than 'jquery' here; it's hard-coded.)
2. Added a simple wrapper that simulates the AMD module framework without implementing any of it. I put this in a file called define.js. Because this is all done within my plugin, I named the global variables according to my plugin's name. Here it is:
window.format_oustudyplan_modules = {};
window.format_oustudyplan_pending = [];
window.define = function(ignored, fn) {
window.format_oustudyplan_pending.push(fn($));
};
window.format_oustudyplan_add_pending = function(module) {
var next = window.format_oustudyplan_pending.shift();
window.format_oustudyplan_modules[module] = next;
};
window.require = function(modules, fn) {
var params = [];
for (var i = 0; i < modules.length; i++) {
params.push(window.format_oustudyplan_modules[modules[i]]);
}
fn.call(this, params);
};
3. Wrote a PHP function to 'load a module':
public static function load($module) {
global $PAGE;
if (!self::$donefirst) {
$PAGE->requires->jquery();
$PAGE->requires->js(new \moodle_url('/course/format/oustudyplan/js/define.js'));
self::$donefirst = true;
}
$PAGE->requires->js_init_code('format_oustudyplan_add_pending("' . $module . '");');
$PAGE->requires->js(new \moodle_url('/course/format/oustudyplan/js/' . $module . '.js'));
}
4. Wrote another one to 'call a module function':
public static function call($module, $fn, array $params = array()) {
global $PAGE;
$jsparams = '';
foreach ($params as $param) {
if ($jsparams !== '') {
$jsparams .= ',';
}
if (is_int($param)) {
$jsparams .= $param;
} else if (is_string($params)) {
$jsparams .= '"' . addslashes_js($param) . '"';
} else {
throw new \coding_exception('Unexpected JS param');
}
}
$PAGE->requires->js_init_code(
'window.format_oustudyplan_modules.' . $module .'.' . $fn . '(' . $jsparams . ')');
}
5 To include my JavaScript, I do this (the above two functions were in a 'js' class):
js::load('sections');
js::call('sections', 'init');
These PHP functions are not the same syntax as used in the 'real' function which is a single function js_call_amd, but are similar. The intention of this approach is that I should be able to move to 2.9 syntax without having to change my JavaScript code. What I will have to do:
Hope this might be useful to somebody else :)
Moodle uses the Behat testing framework, which does automated functional testing (that's the type of automatic testing where it clicks on things in a browser window, in exactly the same way as if a user was doing it).
As a practical tool, this works great - yes, it's hard to install, only really works properly using Firefox at the moment, and is very slow, but these things are probably unavoidable.
I have two issues with it, so I'm going to write a rant. :)
Behat tests are written as a sequence of sentences like:
And I click on the 'span.someclass' 'css_element'
Each sentence is easy to write but there's lots of room for error. In this case, what if there are two elements on the page with that class? What if I got the CSS class wrong? What if I'm not actually on the right page that has that element yet, and I need to click another link first?
The answer to those 'what ifs' is that you have to figure out why it didn't work, change it, then run the whole test again. That's a familiar cycle for software developers, but in this case it's unfortunate because bearing in mind that Behat tests always begin from the home page of an empty Moodle site, there are usually a good few steps before you even get to the main part of your test (log in, go to the course page, create an activity, etc). As a result you probably have to wait at least thirty seconds before you can see if your change works - and if the problem is near the end of a complex test, it could be several minutes.
For anyone reading this who isn't a software developer: thirty seconds is a Long Time. You know when you just missed the bus and have to wait 25 minutes in the rain for the next one? That's about how long thirty seconds is to a software developer.
When writing a Behat test, what you're doing is taking an interactive activity (clicking on things in a web browser) and turning it into a script. That's fine, but it would be much more efficient if it were possible to actually do this interactively.
In other words, I could type my example sentence above into a Behat console; if it didn't work, rather than 'test failed, now you have to run it again from the start, loser' I would simply get an immediate 'nope' and be able to try a different variant of the command. And, with the web browser (that Behat uses for testing) conveniently open already at the right place, I could even use the web developer tools to check on the current page structure to see why it wasn't working.
Once you've got a test working, the current process is fine, but for actually developing it, an interactive console would be a huge advance. There's an issue about this here and a demo where somebody has made it here (link in demo doesn't work so I don't know how sketchy that is) but it doesn't seem to be a solved problem.
There are theories about how you do this type of testing. It's strictly from the user's point of view. You don't write a test unless you can define what the benefit of that feature is for the user. Each scenario contains a single independent test. Yadda, yadda, yadda.
All of this is just so much meaningless junk and distracts from the really important thing which, in practice, Behat testing actually does. What it actually does is, in a repeatable and fully automated manner, carry out a sequence of steps and check that they have the expected result. So it's a shame it gets hidden behind Fully Agile Runtime Testing methodology, or whatever it's called, with everyone having to make up stupid prefixes at the top of the file like:
In order to take over the world
As a 37-year old vegetarian who hates dogs
I need to make sure the damn availability conditions page in Moodle actually works, okay?
That example could be included in a real test script in Moodle - I might have some effort getting it past code review, but other than that, it would work exactly as well as the real text in that test, which is to say, it would have no practical effect at all and be of no use even in documenting the test.
This is a waste of time. We already have a user interface feature and at this point, nobody should care why or who it's for. We want to know if it works or not. Maybe it needs testing from the perspective of different roles, maybe it doesn't; that's a detail that can be handled in the test. Somebody should have already decided it's useful (or if not it can be deleted) - that's a completely independent question from testing it. Do you need to pretend that your testing system is a part of your 'agile' requirements list? Nope. Completely independent things, much better kept separate.
You know what would be better? A documentation block at the top of a test that describes what is covered by the test. (Unless that's already obvious from the name.) Sort of like you have in other programming languages.
Just to hammer it home, imagine doing this in another programming language (and yes Behat scripts are a programming language), like PHP. Here's an example:
/**
* In order to have users collaborate with each other
* As a teacher
* I need to let Moodle APIs know the current version of the forum module.
*/
(That is, in 'agile' theory, the comment that should go at the top of mod/forum/version.php.)
The same type of issue applies to each sentence line, which starts with a word that's ignored, something like:
Given I follow "C1"
When I edit the section "1"
Then "Restrict access" "fieldset" should not exist
That first word (Given/When/Then) doesn't do anything, it's just part of the 'how you are supposed to express a test for no good reason because it fits somebody's theory' junk. And to make matters worse, Moodle have an actual coding standard requirement that you can't use it sensibly. Your scenario has to have exactly one 'Given', 'When', and 'Then' (you can have as many 'And' lines between them as you like). Back in reality we obviously want to check multiple things in a single test scenario because otherwise it would take a million years to do a Behat run instead of the mere 48,000 it takes at present, so this means we start every line with 'And'.
You know what would be better than these meaningless words? Bullet points.
- I follow "C1"
- I edit the section "1"
- "Restrict access" "fieldset" should not exist
Not sure that's allowed in Behat syntax, but it would be a nice improvement. Another nice improvement would be, you know, leaving out entirely the word that doesn't do anything.
-
Right, rant over. :) Behat does work pretty well, but the first issue makes it rather more painful to create tests than it should be, and the second just makes it annoying (at least if you're me).
Oh dear, it's been a very long time since my last blog post!
This is about a new enhancement I've been working on, hopefully for Moodle 2.7, to the conditional activity availability / 'restrict access' system. Basically it's a new API and quite a large change. The key improvements to Moodle out of the box are:
I made a screencast to demonstrate the user interface which I have attached below. For best results, make the video full-screen. Also, for some reason I don't understand, it won't start playing until it downloads the whole thing - sorry about that. And finally... drat! It won't play on this page unless you have an OU student/staff login. If you don't, then please right click or otherwise download the link in order to save it to your computer / device and watch there. Sorry about that. (I should report that to someone...)
Screencast (20MB .mp4, 8 minutes)
This isn't quite finished yet but it's very very close (a few days) and I wanted to get it out there. :)
It isn't shown in the screencast because I wanted to concentrate on the UI but another key advantage is that you can add plugins if you want to make availability conditional on something else.
Finally just to make clear, this does replicate absolutely all the functionality of the existing interface - you might have noticed there's no button for user profile conditions, but that's only because I didn't finish the user interface for it yet. It will be there. :)
This blog post is about the work that the Open University has funded (and I've coded) to improve backup and restore in Moodle 2.6.
I was going to talk about this in the Moodle developer meeting, but decided to skip my item because it isn't that critical for developers to know about, I had problems connecting to the technology, and the meeting would have run late. So I'm writing a blog post instead!
There were a number of problems that meant backup and restore was not reliable for large courses. These included:
To solve these problems I made a number of changes, mostly listed under MDL-38189.
My basic approach was to start with the last problem - using the excellent 'generator' mechanism already in Moodle, I built a simple admin tool to create large courses for testing. The tool had options for different sizes ranging from XS to XXL (this last size I never tried; it may possibly be stupidly big!) with L designed to be similar to the largest courses on our system, and XL designed to be larger than those largest courses on our system.
After that I simply tried to backup, and then restore, courses at each size. The XS and S sizes worked immediately, but the M size failed. Once I got the M course to work I tried with the L course and finally the XL course.
The changes I made that correspond to the above problems are:
What's next? Well, I'm not planning any more large changes. But first, it's still possible there might be regressions caused by some of this code which hopefully will be spotted before 2.6 release. And second, I expect there will be some areas of backup and restore which weren't exercised by my large test courses, and which can still potentially time out - so we might need to add progress reporting calls to more areas of the code that I didn't yet spot. The general principle is that if some operation might take longer than, say, a minute, it's probably necessary to find a way of calling into the progress API periodically during that operation.
Hopefully this is more than anyone wanted to know about the backup/restore changes. :) For most people I expect all they'll notice is that they'll get a progress bar when they do backup or restore.
Finally, thanks to all the developers who helped with peer reviews and integration reviews for the many changes related to this! And also to those who worked on (or are still working on) their own separate changes to improve it.
People frequently ask the question 'when will [an OU-developed Moodle plugin] be available for [a new or upcoming Moodle version]?' So I'm writing a blog post to cover it. Not very amusing I'm afraid...
In general, OU releases are about six months behind the community Moodle version (possibly seven months now the community schedule has been shifted). This is necessary to ensure reliability but also to give us time to update our plugins, while maintaining new feature development.
If you want to use a released, tested version of our plugins, you'll need to follow our release schedule. Or if you want to live life on the edge, you should probably find that the untested 'master' branch of our plugins will have been updated to work on the next Moodle version by about two months before our live release, i.e. about four months behind moodle.org.
This was probably hard to understand, so let me do headings and bullet points.
Boring post but for info, the OU plugins on GitHub (ForumNG, OU wiki, OU blog, subpage, etc.) have been updated today:
So anyone waiting for a 2.4 version, you might like to try it again. We think most of them basically work now but there may still be some problems (especially subpage might only work under certain conditions). If something's still broken in 2.4, feel free to file issues in the relevant GitHub project.
Oh dear, it's my first blog post of the year. :)
There is probably something more interesting I could talk about, but we're getting a bunch of questions about 2.4 versions of OU plugins lately.
To recap the OU schedule:
In this particular case for 2.4:
Of course, some of our plugins from previous releases may continue to work unchanged - it depends whether any of the APIs in Moodle core were modified in a way that breaks our plugins in that release.
Where our plugins don't work in 2.4, feel free to file an issue on our GitHub site, but be aware that we won't fix it until our development period starts. If it's a simple problem and you want to fix it for us by submitting a patch, please do - we'll try to apply these patches early in the development period, i.e. more like February than April. (We can't apply them before the development period starts though, sorry.)
Since I haven't written a blog post for a while, thought I'd do one about the small project I just finished. This is something we're not particularly intending to release publicly because it's probably not useful to anybody else, but it might be slightly interesting to other large-scale Moodle users. Or not. Let's see.
Basically we had two problems with our Moodle system that this is trying to solve:
The OU happens to have an EMC Atmos file store, a commercial product which is obviously designed specifically to store files, so the suggestion was that we should move large files out of Moodle and into Atmos. That way we don't have to include them in the copy for acct (we'll give our acct system read-only access to the filestore so they will 'still be there') and serving the files can also be passed off to Atmos.
Conveniently, there is a new feature in Moodle 2.3 which allows files to be stored in the Moodle filesystem by 'reference' - instead of storing the actual file, it stores a reference to an external repository system.
Using this feature, I built a repository that, during cron, finds all files larger than 10MB. It checks (based on contenthash) to see if they are already in the Atmos system and, if not, transfers them to Atmos, deletes them from Moodle and replaces with the reference. Then, when users click to download a file, it redirects to an Atmos shared link (complete with security key to prove they can access it; we don't lose security by this process, it only happens after the Moodle security checks).
This wasn't very difficult; one of the 'gotchas' is that you have to make sure you set your reference's content to nothing (empty string); if you let the reference keep its original contenthash then Moodle won't delete the file.
A few things in Moodle meant the implementation is not quite as nice as it should be. First there's no way to have a repository that only lets you create one instance, or only lets you create it at system level. I hacked it so that in the creation process it throws an error if you try to create two. Second I couldn't find a way to stop this repository showing up on the file picker for admin users. For everyone else, it doesn't show because I made a capability and didn't give it to anyone.
Overall, though, neither of these are serious problems and the process appears to work well. Although I should admit - I finished the code but it hasn't been through testing yet. :)
Just by the way - this is only part of our 'filesystem too large' solution. I think this will take out about a third of the filesystem size; another third will come by removing course backup files. Those can be huge and they appear to persist forever; we'll probably do a local plugin that deletes them after a month or something. (We don't use those files for backup purposes.) So anyone concerned about filesystem size should also check how many backup files there are in course/user areas - for many of our courses, one backup can be well over a gigabyte, so these add up quickly.
We've just released a new plugin for Moodle 2.2+. (Yes, '+': purely by coincidence, this one already works on Moodle 2.3!)
It's a block that displays news messages, and we've called it the news block. (Not a very original name. Sorry. We did consider 'the boggleogglesquoop block' but it didn't fit in the dropdown.)
This is a replacement for the standard Moodle 'news forum + latest news block' combination.
I made a short (5m) screencast demonstrating how to use the block (running on standard Moodle 2.3). Assuming I haven't run out of bandwidth, you might as well watch the screencast; there aren't any jokes but at that length you can probably survive it anyway.
Features include various options for how messages display (e.g. hiding the author name, if the author isn't important), setting up messages in advance to appear on a certain date, multiple news blocks on the same course if you want to separate different types of news, grouping support so that some messages only appear to a subset of students, and integrated RSS input and output. Combined, you can use the RSS features to post shared news (for example, have a science faculty news block on one course, then include those messages automatically in news blocks on each of a dozen science courses).
The block was originally designed by me; we paid for development by Catalyst. After that initial development we've done minor changes in-house (bug-fixing etc.), and Anthony Forth, another OU lead developer, added grouping support.
To get the block for your own system, download it via the GitHub site.
Disclaimer: the block probably hasn't been much tested on systems that are different to the OU's. I recommend that you try out any new plugins on a test system before using them in production. Unless you really like downtime. :)
Just an update for any other sites using the Open University's Moodle 2 modules that I am responsible for, such as ForumNG, OU blog, OU wiki, and subpage.
Currently these modules are available and maintained for Moodle 2.2, with stable versions updated monthly (and occasionally at other times) when there are changes. In order to get the updates, you have to grab them from our GitHub site as I do not have time to do the manual update process for the Moodle plugin site.
Moodle 2.3 is about to be released but the OU will not be using it immediately. There are changes in Moodle 2.3 which break some of our modules, especially ForumNG. If you rely on our modules and don't want to do without them, it would be advisable not to upgrade to 2.3 until we do.
I don't promise dates and these might change but I would expect that we will have 2.3-compatible but untested versions of these modules in our code repositories at some point during September. It looks likely that our first 2.3 stable branch will be in early December.
Before then, if people wish to submit patches (using our GitHub site) to fix problems so that the modules do actually work in 2.3 earlier, you could do and if it seems safe I will include it. But please test the patch fully against 2.2 before submitting. :)
This is a brief blog post to let ForumNG users know that I've now, finally, set up a stable branch.
MOODLE_21_STABLE branch at GitHub
This branch will shortly (from Tuesday) match the version on our live servers and should continue to do so until we stop using Moodle 2.1 (which is currently scheduled for April 6th). After that it becomes unsupported and we'll hopefully have a MOODLE_22_STABLE branch.
You can still use the 'master' version of the code, if you want our current untested development version. As of just now, there is a change to the master version which fixes a critical Moodle 2.2 bug (email sending was broken), but unfortunately also means that the master version won't work on Moodle 2.1 any more.
So to summarise:
If you really really want a version that works on both 2.1 and MySQL, I guess you can grab the relevant commit from GitHub (i.e. the one just before the commit where I say it requires 2.2).
Any MySQL + Moodle experts out there?
Somebody's reported a problem with ForumNG on Moodle 2.x, and I believe it's likely a MySQL problem. There is a really complicated query with several subqueries and the database error is something about f.id not existing (which it blatantly does).
If anybody fancies installing ForumNG on their Moodle 2.x/MySQL test system, please let me know in the above tracker issue (a) if it works/fails for you (confirm version numbers please?), or (b) if you can tell me what's wrong with MySQL and/or my query, and what I could change to make it work.
I do not currently have a MySQL test install so I would prefer 'I have tried this and it fixes it' type answers rather than 'You could try this, it might fix it...' :)
I did already look at the MySQL documentation because I know there is a hideous limitation in a slightly similar situation (if you have subqueries in a DELETE, they can't refer to the table you're deleting from? Something like that), but I didn't find anything that obviously seemed to apply to this.
(I've disabled comments on this blog post, please add suggestions directly to the tracker issue.)
By special request, here's a super-boring post on the ForumNG 'features' plugin infrastructure! (Not a developer? Look away now.)
It's very simple: you stick standard Moodle subplugins in the 'feature' folder. Each one must have a class which extends either mod_forumng_discussion_feature or mod_forumng_discussion_list_feature (which themselves extend mod_forumng_feature).
The difference is that 'discussion features' will be placed at the bottom of the discussion page, and 'discussion list features' just below the list of discussions on the main forum view.php. Discussion features are passed a $discussion parameter for a bunch of things, and discussion list features are passed a $forum parameter.
The classes for this basically have three methods that probably need implementing: get_order (returns an integer indicating the order relative to other features), should_display (default returns true if you have moderator permission) and display, which returns some HTML.
This doesn't really add up to much; it just means you can put a button (or pair of buttons, or dropdown, or whatever) in the area below the list of discussions or at bottom of a discussion.
Then you can put the code for handling actions in PHP scripts inside your feature.
That's about it except there are some helper functions/classes to make certain things easier (making the button form; plus for example if you want to use the 'post selector' feature which integrates with a lot of built-in JavaScript as well as non-JS alternative code, so that you can let users do some action [export or forward by email] on whatever posts they want).
There's a catch here, which is - well, let's take the 'delete discussion' feature as an example. So there's a 'delete' discussion feature and it provides the button you use to delete (or undelete) a discussion, and the script that runs when you click on it with the 'are you sure' prompt, and it checks permissions and stuff - but it doesn't really do the actual work for deleting a discussion because that clearly ought to be, and is, part of the forum back-end in the mod_forumng_discussion class. (Note: If you want to look at the code for a forum feature, the delete one might be a pretty good place to start.)
That's also true for a lot of the built-in features: the 'feature' architecture actually only includes the UI, and the real back-end work is done inside the core class, because it sort of has to be that way.
But it does still allow some degree of separation.
One of the reasons I wanted it is that ForumNG at the OU includes this totally useless 'show who read this discussion' function. Supposedly this shows you who has read a discussion and when, but in fact it's impossible to tell. So the list, though sort of correct, is basically a lie. I had to do it because they made me. :) But I didn't want to include that in the public version; no reason to spread our bad practice around the world. The 'features' system meant I could just exclude that one folder from the distribution without changing anything else.
Note: An obvious extension would be to add mod_forumng_post_feature class but we did not find a need for this yet - possibly because the 'post selector' approach (where you click a button at discussion level, but then can select one or more posts with checkboxes) is more flexible.
Important note: This is just my personal opinion as a developer. Nothing in here relates in any way to the Open University's official position on any of these matters!
Internet Explorer 6. Three words that strike fear into the spine of any web developer.
As part of our move to the new Moodle 2-based system for some of our module websites, we agreed - well in advance - that we would no longer support IE 6. Google dropped support in 2010, Microsoft run a website encouraging people to stop using it... it's game over.
Why did we drop it? Well, anyone reading this who isn't a web developer might not be aware, but IE6 isn't just equivalent to any other old browser version you might encounter, like say Firefox 3.6 or something. IE6 behaves radically worse than every other browser in terms mainly of its CSS (layout) and JavaScript (interactive) support. If you want things to work in IE6, you have to do a significant amount of extra work in testing and coding workarounds to all the problems. We would rather spend our development time improving the system for 98.5% of users, not fixing problems for 1.5%.
So, we don't support IE6. That means not only do we not test with it, but if anyone reports problems we don't fix the problems, just tell them to use a different browser. And because we developed a new theme without supporting IE6 from the very start (unlikely our previous system, where we fully supported IE6 at the time our theme was originally implemented), it was likely there would in fact be problems.
Turns out that in fact, the main portion of the page doesn't display at all in IE6. It's blank. Oh well, whatever. We don't support it, right? Not a problem.
Until last week when everything hit the fan. Specifically, it turns out that certain government institutions are currently still using IE6, and preventing their staff from installing other browsers. And those staff often don't have the opportunity to simply study at home instead.
Which institutions?
The ones with guns.
Ooops.
I could wear a bullet-proof jacket, but that wouldn't solve the real problem. Which is, we don't want to support IE6, because it costs a lot of developer time. We can't very well sort of half support it; if we support it then that means we have to test everything in IE6 because it's not like any other browser (we don't test everything in Opera either, but that usually just works) - most things probably won't work and will need workarounds. We'd lose all the benefit of ditching support.
But then I had an idea. I suggested it to my bosses and they were okay with it, so we went ahead.
IE6 is a disability. These students are being forced to use an ancient browser; it's kind of like they're being forced to walk down the street blindfolded.
We support blind users. Why not support IE6 the same way?
There's no problem with IE6 as a basic HTML browser. If your website has no style and no interactive features it will work fine. Coincidentally, this is roughly the same way that blind users (through screen readers) experience the internet. So we've already made our sites work if you don't 'see' the styles or the interactive features.
So if you're using IE6 and you look at our new module web sites system (not this one), you'll now see a totally plain screen - we've stripped out all the styles and interactive scripts. It's just like the experience you get with a screen reader - complete with skip links. Not exactly pretty (and we still don't officially support IE6), but you can access all, or nearly all, of the content.
Because I did this at a theme level, it should apply in general. We don't need to test in IE6 - if we build a new feature, as long as it works with JavaScript turned off and no styles (which it ought to, for accessibility) then it should be fine in IE6 too.
That's all. I thought this might be interesting for other people struggling with IE6 issues. There was an embarrassing/amusing screw-up when we deployed this change, but this post is already long enough so I think I'll leave it out. :)
PS To avoid confusion, I should make clear: there are many OU websites which still fully support IE6. This post is only about the new system for module websites, which doesn't.
Update on my previous post: three out of the four issues (all except media embedding MDL-29624) are now resolved and included in the relevant upcoming Moodle versions. Yay! Thanks to all Moodle HQ developers and other people who assisted with getting these approved and included.
This is a mini-post. I'm going to do a real blog post next.
For my first blog post of the year, I thought I'd just give a brief rundown on the current issues (features and fixes) that I've coded and are waiting in the review process for core Moodle. I'm afraid this isn't a barrel of laughs, but at least it's informational.
Admittedly this is partly because I want to encourage HQ developers to review things, but also partly because I thought other people from the Moodle community might be interested in some of these changes! Feel free to vote on issues if you like them. :)
All these issues are ones seen as vitally important by our staff for various reasons.
Features (for 2.3):
Bugfixes (for 2.1+):
Exciting, no? Okay, no. Still, now you know.
Finally, just as a reminder - if anyone uses any of the OU's custom modules for Moodle 2, please do make sure to keep up with the relevant GitHub repositories in case there are bugfixes that are important to you. (If you do upgrade as a result, don't forget to check the new version on a test server first.) And incidentally, developers working for the company NetSpot recently submitted a few bugfixes to those modules too, so thanks for that. You can see their contributions in the git history.
This is a special super bonus blog post. Not only are there two blog posts in one day, but this second one has two screencasts in it!
In today's Moodle developer meeting, I suggested (rather rudely, sorry) that the OU's subpage module was a better way to handle the 'everything in Moodle course has to go on the same single page which is stupid' problem than implementing some kind of 'show things on separate pages' feature in every single course format.
Subpage - quick demonstration of subpage using the standard theme so it looks pretty much like standard Moodle (about 5 minutes)
Subpage in real life - how we use subpage at the OU (about 3 minutes) - you get to see our pretty theme in this one
Here are some reasons I think subpage is a good approach:
There are some problems with the way subpage is implemented; basically, it's a leetle bit of a hack. In order to make it un-hacky, some of the following things would need to happen to core:
Subpage code repo (...may or may not work at the moment, I hope it does though).