OU blog

Personal Blogs

Design Museum

Learning how to assess - not an AL

Visible to anyone in the world
Edited by Jonathan Vernon, Wednesday, 17 Dec 2014, 19:35
From E-Learning V

 

I'm doing my learning with The OU, but on another platform. I'm emerging from an extraordinary eight weeks on the 'Start Writing Fiction' MOOC on FutureLearn. This draws upon the 'Start Writing Fiction' OpenLearn content and on the various BA and MA Creative Writing courses from The OU.

What's worth reflecting upon as a person who has taken this journey is the nature of the learning process in a highly charged, collaborative and 'massive' environment, the community self-help spirit that is engendered, partly by its being free and open, but also because scale means that that percentage of people who are contributory and giving is large enough to make an impact. And finally, as a learning experience - have I learnt something? Does it 'change behaviours' ? which is the ultimate test of learning - you come out changed.

The course is about creative writing, with the emphasis in a few hours a week on one thing only: characterisation. 

I've taken the view that even if I finished the course a week ago, that it is still term time so I have a duty to stick around. As well as carefully going back through all seven weeks so far I have now done 16 reviews. The reward is always two way.

In this instance I see in every writer ways I'd like to get it right, as well as ways that I too need to stop getting it wrong. An hour spent on a review is typical, though not always possible if either the piece is brilliant and a greater joy each time I read it, or because the piece struggles to reveal that the author has taken part or taken in much at all from the last seven weeks. The reviewer cannot relive the course pointing these things out that have been missed. Perhaps this is the difference between me and a professional tutor or lecture marking an assignment and given constructive feedback. 

Even amongst thousands you get to recognise a hundred or so people who are active in the discussion groups, desperate to put right what they are getting wrong and willing to try anything - I'm in that category and can truthfully say that my writing has been changed for the better, forever. I hope with these reviews that I am able to put back in a fraction of what I have been able to take out. From the learning perspective my behaviour has been changed. I know that writing is 20% of the task, so now I bash on with it as fast as I can ... as the real job, the 80% is the edit. Maybe with experience the balance will shift a bit. More 30% to 70% as I get things right first time. 

There's a discussion about how reviews are shared out. The system has to be automated. I believe the participation at the start of the MOOC was 20,000 and has gone up to 23,000 even 25,000. I know that stats so an exponential decline (is that the right term) sees 50% never even start and another 50% drop out after weeks one or two. Under 10% complete, possibly under 6%.

Reviews of work cannot and are not carried out by ALs. The cost would be astronomic and it would take years. Instead we rely on peer review. Over the eight weeks, beautifully choreographed (learning design) there was been a review of a 250 word piece, then a 500 word piece ... and now the equivalent of our EMA and a 1000 word piece. Without exception people are finally understanding that 1000 words means exactly that. People had the ignorance, arrogance or temerity to post 2,500 word pieces in the firs assignment. Some ignore the course, but wanted people to review their brilliance sad This is what occurs on an open platform. 

Regarding these pieces occasional requests have been made to have all of these on public view so that we could pick and choose the pieces to review. It would be quite wrong though to reveal what can be a sensitive and personal exchange between author and reviewer in a very vulnerable moment. It would, as I've seen in open 'classes' turn into a bit of a bun fight where, in the worst instances, like in the playground, you get people applauding one author and ridiculing another ... or simply join in on the back of what others have said. i.e. the learning experience is thwarted, even abused. It matters that the reviewer knows that their own words matter, without being influenced by what others have written. That said, some reviewers take a cavalier approach saying they don't like a thing, and then saying no more. If that is a student's only review received you can well understand their frustration. Even with the numbers involved somehow these pieces need to be returned and churned through the system, ideally until three to six reviews are received each. More work needs to be done to help students do reviews too so that they feel confident about doing so.

What we all benefit from this process is both learning to review, and learning to receive feedback.

The recommendation I make to everyone is to keep reviewing until you become good at it. If and when you can master reviewing, then you will be in a far better position to fairly review and edit your own work - a lesson that has finally sunk in. Writing is easy, the fun part. Jazz writing I call it. Top of the head stuff. The editing is the pain that crafts a piece so that others can enjoy it too. This pain is reduced the better you get at it.

What's revealing here is, as I've seen in the reviews, is that there are pieces that suggest that the author hasn't learnt anything at all from the course. I cannot make that assumption, so I review on the basis that it is genuine. From a formal assessment point of view, as I've learnt as a student with The OU for four years, is that a tutor looks for repeated indications that the student is using what they should have learnt from the course - if that is not present then alarm bells should ring. How can I give them points?

The problem of course in a MOOC (Massive Open Online Course) is that those who don't do the this task properly are especially taking away from genuine participants who want several considered reviews in return. Only if we get perhaps three or more reviews each will the 1 in 3 'non-review' be cancelled out.

Of 12 pieces I have come to expect 2 excellent and 2 ... how can I put it politely other than to say 'dire'. I'm not to know why this is, I can only judge what I have to review and have in mind the brief and the content of the course over the last seven weeks. Do some people paste in something they wrote months ago that has no bearing at all on the course? It seems that way. Might someone post in a piece that has been published, that is on brief? That is possible too. Of the remaining eight pieces these tend to be where errors and corrections based on the lessons of the course are most easy to make. It takes time. Time and focus. I admire those tutors, here and elsewhere who so clearly have gone to such lengths. Decades after the event I see lengthy comments on pieces for A levels from teachers who were clearly putting in a huge amount of work ... with no word limit on essays too.

In contrast, though not from The OU, I have had reviews of work that were laughable - one may have muddled me up with another student, while the other might have been written in the pub over a pint. One I made a polite complaint, gained 10 points and a distinction. The other I am about to challenge as this 'pub' idea might be close to the truth. And they are paid to assess a piece. In this instance the criticism over my missing a key point is unfounded as I make the required point a) in the introduction b) in the conclusion and c) developed the idea in the main body of the essay. Their comment, 'looks rushed' - which to my sensibilities is an indication of exactly how the tutor behaved - they are the one who were in a rush. 

Students need to be put on a confidence building exercise as they start university so that they feel, as fee payers, able to 'complain' without being stonewalled. This is another theme, but fee-paying students should and will change the attitude of institutions to their fee paying clients, rather than students on a grant-based 'freebie'.

Permalink Add your comment
Share post
Design Museum

Learning how to learn online with FutureLearn and The OU

Visible to anyone in the world
Edited by Jonathan Vernon, Friday, 12 Dec 2014, 07:18
From E-Learning V

Fig.1 My progress on The OU MOOC on FutureLearn 'Start Writing Fiction' (c) FutureLearn 2014

More than any module or exercise I have done over my four years with The OU, it is a MOOC in FutureLearn that is giving me the most thorough experience of where the future or learning lies. I'm in week seven of eight weeks of 'Start Writing Fiction' from The OU, on the FutureLearn platform. Just in these few weeks I've seen the site change to solve problems or to enhance the experience. Subtle lifts and adjustments that make a positive out of constant adjustment. Those tabs along the top: activity, replies where under a tab. I think 'to do' is new while 'progress' was elsewhere. This is a responsive platform that listens to its students.

In the final week we submit our third piece of work.

As assessments go these are far less nerve racking than a TMA. The first piece was 300, the second 500 and the last will be 1000. These are assessed by fellow students. In my case I had one, then two reviews. Most people seem to get at least two sometimes three. The system is designed, I'm sure, to try and ensure that everyone's work is reviewed at least once. Tens of thousands, certainly thousands of people are on the course.

We're here to the 19th of December or so ... if you follow the tracks as laid.  

I hazard a guess that between 20-100 have posted there final piece already. Some, I know, got to the end of the entire course a few weeks ago; I looked ahead to see out of curiosity. There have always been 20 who post comments one, two even three weeks ahead. If 20 are posting I hazard a guess knowing my stats on these things that another couple of hundred could be clicking through the pages to read and observe. They may, like me, be coming back later. They may only be following the course, but not participating. Often, it is like standing on a stage looking into the gloom of the auditorium. Someone probably out there. One or two let you know. The rest don't.

I hope those that race ahead come back ...

I find that if I get ahead then I slow down and retrace my steps. To learn in this connected and collaborative way you are far better off in the pack ... it is not a race to get to the end first. In fact, those who do this have already lost. They've missed the point. I'd suggest to people that if they have the time to do the week over. That's been my approach anyway - the beauty of these things is everyone can come and go as they please, at a pace that suits them. Skip a bit. Go back. Follow it week by week, day by day ... or not. Whatever works works?

There's another very good reason to stay with the 'pack' or to come back and do a week over - the platform depends not on tutors and moderators commenting and assessing work, but us students doing a kind of amateur, though smart, peer review. This is what make a MOOC particularly vibrant, memorable and effective. Not listening to an educator telling us what's what, but the contributors sharing, figuring it out, answering each other's problems in multiple ways. We all learn in different ways and at a pace that shifts too. I find that often a point I don't get first time round, on the second, or third, or even the fourth visit to an activity someone, somewhere puts it in a way that suddenly brings complete clarity - their way of seeing a thing, or expressing it, makes more sense than the writes of the course could manage. Because they can only write one version, not the 'tartan' that comes from an intelligent, threaded online conversation.

Permalink Add your comment
Share post
Design Museum

H809 Paper 1 Scrutiny of a research paper

Visible to anyone in the world
Edited by Jonathan Vernon, Sunday, 10 Feb 2013, 20:01

I should dig out the research but hasn't it now been shown that as long as students get to choose between classroom / lecture based, blended or online learning they are all equally happy with the outcome? This might suggest that institutions need to offer this mix ... but it shows how much better it is to start on a positive rather than feeling you have to have your module deliver in a set way whether you like it, or get along with this approach or not.

The other piece of reading I need to reference concerns a study, I think from the 1960s, and mentioned in The Gutenberg Galaxy by Marshall McLuhan that I am reading, that there are significant differences in how people interpret a piece of text, that we bring significant baggage to it, drawing conclusions and seeing patterns, making links and connections that are very much are own. This is particularly the case with 'open texts' that invite thought and require us to construct our own meaning compared to 'closed texts' which aim to present a thesis as an absolute. Is this why so many research papers are dry? Why they leave little impact? It suggests that a paper should be written up in two forms - for peer review and scientific scrutiny on the one hand, and to invite comment, feedback and contributions on the other.

A reason to blog? Your paper is published, then your write it up in a blog in a more accessible and 'open' manner?

Permalink
Share post

This blog might contain posts that are only visible to logged-in users, or where only logged-in users can comment. If you have an account on the system, please log in for full access.

Total visits to this blog: 13019838