Personal Blogs

William. D (2017) Embedded Formative Assessment: (Strategies for Classroom Assessment That Drives Student Engagement and Learning) (The New Art and Science of Teaching) : Second Edition. Bloomington, USA. Solution Tree Press. 

Dylan Wiliam is to formative assessment what Chris Witty is to Covid

I’m getting used to Dylan Wiliam - I think of him as the David Walliams of Education, the Will-I-Am of Formative Assessment.

I’ve been pointed at his webinars, sat through a keynote or two, read the book and the read pamphlet ‘Inside the Black Box: raising standards from classroom assessment’ which he produced with Paul Black in 1998 and established his reputation. 

In my first pass of note taking I had over 60 quotes and ideas here, not all from Dylan William. With some editing I’ve got this down to 19 ‘Top Tips’ which are all set out ready to quote. 

I’ll go back to the book if needs be for clarification. In this case it is an eBook which makes it very easy as the digital version will correlate to actual print page numbers.

This is how I learn. 

It’s a slow and repetitive process: skim read a book to get the lay of the land, read it with an open notebook, then transfer the notes to a document like this. Then edit the notes and where necessary go back to the original book. Most important of all - give it a go. Over the last five weeks I’ve had a chance to apply some of the thinking to both online remote teaching and in the classroom face-to-face.  

When a book strikes me as really important I am likely to have it in print and eBook - they read differently and you take different things from each. 

These notes are for me to browse, refer back to and use when I next have a formal rationale or observation to write, as well as for every day reflection on the learning and e-learning experience I am currently going through. 

Myth Busting

Wiliam puts the evidence of learning above myth and has a number of bugbears. 

Good teaching is difficult

It is relatively easy to think up cool stuff for students to do in classrooms, but the problem with such an activity-based approach is that too often it is not clear what the students are going to learn. (Wiliam 2017 p.94) 

Wiliam disregards learning styles, ‘which have no discernible impact on student achievement at all’. (Wiliam 2017 p.11 also Adey, Fairbrother, William, Johnson, & Jones, (1999). Rather - ‘as long as teachers vary their teaching style, then it is likely that students will get some experience of being pushed beyond it’. (Wiliam 2017 p.48) 

Nor is Wiliam a  fan of ‘performance of a learning task’ as a predictor of long-term retention of learning (Wiliam 2017 p.47 from Bjork, R.A. (1994) Any mention of ‘neuroscience’ as the panacea annoys him. (Wiliam 2017 p.50) 

There is no shortcut

Wiliam’s firm belief is that formative assessment improves performance. (Wiliam 2017 p.11) His view - ‘the use of assessment for summative purposes - grading, sorting, and ranking students - gets in the way’ of learning. (Wiliam 2017 p.56) Education is overly prescriptive with rubrics. (Wiliam p.93 in Alfied Kohn (2006)

No one can do the learning for the student who does not engage.

Our classrooms seem to be based on the principle that if teachers try really hard, they can do the learning for the learners. (Wiliam 2017 p.225) 

According to Wiliam there is little evidence that the following ‘tricks’ have any impact of student achievement:

• Summarisation

• Highlighting

• Keyword mnemonic

• Image use for text learning

• Rereading (Wiliam 2017 p.225) 

Wiliam sets out five key stages of formative assessment : 

1. Clarifying, sharing and understanding learning intentions and success criteria.

2. Eliciting evidence of learning

3. Providing feedback that moves learning forward

4. Activating learners as instructional resources for one another

5. Activating learners as owner of their own learning

The Assessment Reform Group puts it this way. Assessment to improve learning requires five elements to be in place (cited Broadfoot et al., 1999) (Wiliam 2017 p.61)

1. Providing effective feedback to students

2. Actively involving students in their own learning

3. Adjusting teaching to take into account the assessment results

4. Recognising the profound influence assessment has on student’s motivation and self-esteem, both of which are crucial influences on learning.

5. Needing students to be able to assess themselves and understand how to improve.

Challenge them!

I like Wiliam’s thinking that ‘good instruction creates desirable difficulties’ - that we learn and retain knowledge through struggling with it. (Wiliam, 2017 p.11) I know from personal experience that the greater the struggle I am willing to endure, the deeper and the longer lasting my learning. It is when I can face the struggle … that I struggle.  Good instruction creates what he describes as ‘desirable difficulties’ (Wiliam, 2017 p.47) quoting Bjork (1994, p.193).

Pedagogy Over Curriculum

‘A bad curriculum well taught is usually a better experience for students than a good curriculum badly taught’. (Wiliam, 2017 p.11) I rather think we can apply this to many situations, for example, that pedagogy comes first - not EdTech. 

Formative Assessment above all else 

The Wiliam mantra is that, ‘attention to minute-by-minute and day-to-day formative assessment is likely to have the biggest impact on student outcomes’. (Wiliam 2017 p.42) He defines formative assessment as ‘the process used by teachers and students to recognise and respond to student learning in order to enhance that learning, during the learning’. (Wiliam 2017 p.59). Formative assessment, according to Bloom (1969) is a kind of evaluation - ‘a brief test used by teachers and students as aids in the learning process. (Wiliam, 2017 p.53 in Bloom (1969) It is ‘the process used by teachers and students to recognise and respond to student learning in order to enhance that learning, during the learning. (Cowie & Bell, 1999 p.32) It is ‘assessment carried out during the instructional process for the purpose of improving teaching or learning’. (Wiliam 2017 p.59 in Shepard et al., 2005 p.275) 

I like this : ‘Frequent, interactive assessment of students’ progress and understanding to identify learning needs and adjust appropriately.’ (Wiliam 2017 p.59 from Looney 2005)

And if you need more ways to think of it, try this: ‘assessment, for learning tells ‘us’ ‘what progress each student is making toward meeting each standard while the learning is happening - when there’s still time to be helpful (Wiliam 2017 p.62 from Looney 2005 (pp1-2) Stiggins (2005)

All kinds of formative assessment are not equal

(Wiliam 2017 p.62) 

‘The evidence is clear that the shorter the assessment - interpretation - action cycle becomes the greater the impact on student achievement’. (Wiliam, 2016). He continues, ‘short-cycle formative assessment has to be the priority for schools and teachers, because the impact on students is greater. (Wiliam p.75) 

‘ … regular use of minute-by-minute and day-to-day classroom formative assessment can substantially improve student achievement’. (Wiliam 2017 p.81)

Design backwards from the learning outcome

(Wiliam 2017 p.81 from Grant Wiggins and Jay McTighe (2000)

Three issues in the development of learning intentions and success criteria that may be useful to think about. (Wiliam p.95)

1. Task-specific versus generic scoring rubrics.

2. Product-focused versus process-focused criteria

3. Official versus student-friendly language

NOTE :> By being specific about what we want, we focus the students learning too much. ‘He who never made a mistake, never made a discover’ (Wiliam 2017 p.95 quoting Samuel Smiles (1862 p.275)

There are a number of techniques which appealed when I read about them and I have used them. These are techniques to help students understand and achieve learning intentions. (Wiliam 2017 p.104 from Clarke (2001)

This is simple, these phrases work 

WALT ‘We are learning to … ‘

WILF ‘What I’m looking for … ‘

TIB ‘This is because … ‘ 

Student Engagement Techniques

The teacher asks a question, selects a student to answer the question and then responds to the student’s answer, which is generally some kind of evaluation of what the student said. (Wiliam 2017 p.124) Teacher-Led classroom discussion.

Try some of these:

• Wait time

• Evaluative listening

• Interpretive listening

• Questions shells

• Hot-seat questioning

• All student response systems

• ABCD Cards

• Mini Whiteboards

• Exit passes

• Discussion vs. diagnostic questions

• Alternative questions

Pose a question, pick a student at random

Pose-pause-pounce-bounce

• Pose

• 5 seconds - pause

• Pounce-random choice

• Bounce-what do you think (Wiliam 2017 p.129) 

App or Lollipop sticks (Wiliam 2017 p.129) 

These techniques are used to ensure that all students realise that all are expected to tke part and that it is ok to make mistakes. If someone does not provide a response then come back to them and tell them  “OK, I’ll come back to you”.  (Wiliam 2017 p.132) 

Then seek Response 2 and 3 and then return to the person who gave no response and ask them which reply they liked the best and ask them why.

‘Engagement and responsiveness - are at the heart of effective formative assessment’. (Wiliam 2017 p.142) 

Managing Challenging Behaviour

To change the behaviour criticise the behaviour not the student. (Wiliam 2017 p.171) 

It is quality rather than the quantity of praise that is most important - teacher praise is far more effective if it is infrequent, credible, contingent, specific, and genuine. (Brophy, 1981 in Wiliam 2017 p.171)

The use of feedback improves performance when it is focussed on what needs to be done to improve, and particularly when it gives specific details about how to improve. (Wiliam 2017 p.180)

Motivation

“It's up to me, and I can do something about it”.  (Wiliam 2017 p.183)

When students have to struggled in the learning task, the quality of their performance on this task reduces, but the amount of learning that takes placed increases (Wiliam 2017 p.190)

Feedback functions formatively only if the learner uses the information feedback to him or her to improve performance. If educators intend the information fed back to the learner to be helpful but the learner cannot use it to improve his or her performance, it is not formative.

Motivation is not a cause but a consequence of achievement (In Wiliam 20176 p.234) from Garon-Carrier et al., 2016)

Like sports coaching, teaching takes time to master

It takes years for even the most capable of coach to break down a long learning journey from where the student is right now - to where he or she needs to be. (Wiliam 2017 p.193)

Feedback should cause thinking

Feedback for Future Action. (Wiliam 2017 p.194) 

To be effective, feedback needs to direct attention to what's next, rather than focusing on how well or poorly the student did on the work, and this rarely happens in the typical classroom.

The response from the student to feedback should be ‘cognitive rather than emotional’ (Wiliam 2017 p.205) In other words, feedback should cause thinking by creating desirable difficulties.

Peer tutoring can be more effective than one-on-one tutorial instruction from a teacher. This is because of the ‘change in power relationships’. (Wiliam 2017 p.209)

And regarding students online not using their webcams he believed you can see how a student is really taking it by seeing their faces (Wiliam 2017 p.209)

Student Reporter

Put students in a group towards the end of the class so that they can discuss then report back on what has been taught.

Two Techniques that work 

(Wiliam 2017 p230)

1. Practice Testing 

2. Distributed Practice

These received high ratings because they were effective with learners of different ages and abilities and were shown to boost students’ performance across many kinds of tasks, and there was plenty of evidence that they worked in educational contexts.

If students complete a practice test and get immediate feedback on their answers, students will get the benefit of the hypercorrection effect for those questions where they were correct. (Wiliam 2017 p.231)

Setting Goals

Students are more motivated to reach goals that are specific, are within reach, and offer some degree of challenge. (Wiliam 2017 p.236 in Bandura, 1986)

When the goals seem out of reach students may give up on increasing competence and instead avoid harm, by focusing on lower-level goals they know they can reach or avoiding failing altogether by disengaging from the task. (Wiliam 2017 p.236)

TIPS

1. Think ‘how am I going to teach this and what are the pupils going to learn?’ (Wiliam 2017 p.79) 

2. Having an in-depth understanding of the curriculum may be of more benefit to student progress than advanced study of a subject on the part of the teacher.

3. Students don't learn what we teach. (Wiliam 2017 p.77) 

4. Teaching the goal. Driving as teaching. (Wiliam 2017 p.78)

5. ‘When the pressure is on, most of us behave as if lecturing works but deep down inside we know it’s ineffective’. (Wiliam 2017 p.80)

6. The teacher’s job is not to transmit knowledge, nor to facilitate learning. It is to engineer effective learning environments for students. (Wiliam 2017 p.80)

7. Learners of all ages need to understand what it is that they need to learn and be able to monitor their progress toward their goal. (Wiliam 2017 p.95) 

8. Don’t simply plan the instructional activity, but also plan how you are going to find out where the students are in their learning. You need to be clear about what we want students to learn (Wiliam 2017 p.115)

9. Ask questions either to cause thinkin and to provide information for the teacher about what to do next. (Wiliam 2017 p.126)

10. Beware - those avoiding engagement are forgoing the opportunities to increase their ability.

11. The most important factor influencing learning is what the learner already knows - the teachers job is to ascertain this and to teach accordingly. (Wiliam 2017 p.166) 

12. Much of the feedback that students get has little or no effect on their learning, and some kinds of feedback are actually counterproductive. (Wiliam 2017 p.167) 

13. Do not mix grades and comments, just stick to comments. (Wiliam 2017 p.167) 

14. Oral feedback is best. (Wiliam 2017 p.174)

15. Too often feedback is counterproductive. (Wiliam 2017 p.178)

16. Don’t provide students with feedback unless you allow time, in class, to work on using the feedback to improve their work. (Wiliam 2017 p.195) 

17. Feedback should be more work for them than you! (Wiliam 2017 p.195) 

18. A simple approach to feedback. Pick out two things to praise, then express what they need to do a constructive wish. He calls the technique ‘two stars and a wish’. (Wiliam 2017 p.214)

19. Teachers have a crucial role to play in designing the situations in which learning takes place, but only learners create learning. Wiliam 2017 p.246)

REFERENCES

Adey, Fairbrother, William, Johnson, & Jones, (1999) p.36 A review of learning styles and learning strategies. London. King’s College London Centre for Advanced Thinking. 

Ausubel (1968) Educational psychology. A Cognitive view. New York. Holt, Rinehart & Winston

Bjork, R.A. (1994) Memory and metamemory consideration in the training of human beings. In J.Metcalfe & A.P. Shimamura (eds) Metacognition : Knowing about knowing (pp.188-205) Cambridge, MA MIT Press. 

Bloom, B.S. (1969) Some theoretical issues relating to educational evaluation, In H.G. Richey & R.W. Tyler (Eds). Educational evaluation : New roles, new means, part 2 (Vol.68, pp. 26-50_ Chicago : University of Chicago Press.

Brophy, H (1981) Teacher praise: A functional analysis. Review of Educational Research, 51 (i), 5-32

Clarke, S (2001) Unlocking Formative Assessment. London, Hodder & Stoughton

Cowie, B & Bell, B (1999) A Model for formative assessment in science education. Assessment in Education : Principles, Policy and Practice. 6(1), 101-116

Looney, J (ed) (2005) Formative Assessment: improving learning in secondary classroom. Paris. Organisation for Economic Co-Operation and Development.

Shepard et al., 2005 p.275 in L.Darling-Hammon & J.Bransford (Eds) preparing teaching for a changing world : what teachers should learn and be able to do (pp.215-326)

Soderstrom, N.C., & Bjork, R.A. (2015) Learning versus performance : An integrative review. Perspectives on Psychological Science 10 (2), 176-199. 

Stiggins R.J. (2005) Assessment for learning defined. 

Wiggins, G., & McTighe, J. (2000) Understanding by design. New York. Prentice Hall.

William, D. (2016) Leadership for teacher learning . Creating a culture where all teachers improv so that all students succeed. West Palm Beach , FL, Learning Sciences International.

William, D (2017) Embedded Formative Assessment: (Strategies for Classroom Assessment That Drives Student Engagement and Learning) (The New Art and Science of Teaching) : Second Edition 

This MOOC on Coursera from Melbourne University looks interesting.

From E-Learning V

I'm doing my learning with The OU, but on another platform. I'm emerging from an extraordinary eight weeks on the 'Start Writing Fiction' MOOC on FutureLearn. This draws upon the 'Start Writing Fiction' OpenLearn content and on the various BA and MA Creative Writing courses from The OU.

What's worth reflecting upon as a person who has taken this journey is the nature of the learning process in a highly charged, collaborative and 'massive' environment, the community self-help spirit that is engendered, partly by its being free and open, but also because scale means that that percentage of people who are contributory and giving is large enough to make an impact. And finally, as a learning experience - have I learnt something? Does it 'change behaviours' ? which is the ultimate test of learning - you come out changed.

The course is about creative writing, with the emphasis in a few hours a week on one thing only: characterisation. 

I've taken the view that even if I finished the course a week ago, that it is still term time so I have a duty to stick around. As well as carefully going back through all seven weeks so far I have now done 16 reviews. The reward is always two way.

In this instance I see in every writer ways I'd like to get it right, as well as ways that I too need to stop getting it wrong. An hour spent on a review is typical, though not always possible if either the piece is brilliant and a greater joy each time I read it, or because the piece struggles to reveal that the author has taken part or taken in much at all from the last seven weeks. The reviewer cannot relive the course pointing these things out that have been missed. Perhaps this is the difference between me and a professional tutor or lecture marking an assignment and given constructive feedback. 

Even amongst thousands you get to recognise a hundred or so people who are active in the discussion groups, desperate to put right what they are getting wrong and willing to try anything - I'm in that category and can truthfully say that my writing has been changed for the better, forever. I hope with these reviews that I am able to put back in a fraction of what I have been able to take out. From the learning perspective my behaviour has been changed. I know that writing is 20% of the task, so now I bash on with it as fast as I can ... as the real job, the 80% is the edit. Maybe with experience the balance will shift a bit. More 30% to 70% as I get things right first time. 

There's a discussion about how reviews are shared out. The system has to be automated. I believe the participation at the start of the MOOC was 20,000 and has gone up to 23,000 even 25,000. I know that stats so an exponential decline (is that the right term) sees 50% never even start and another 50% drop out after weeks one or two. Under 10% complete, possibly under 6%.

Reviews of work cannot and are not carried out by ALs. The cost would be astronomic and it would take years. Instead we rely on peer review. Over the eight weeks, beautifully choreographed (learning design) there was been a review of a 250 word piece, then a 500 word piece ... and now the equivalent of our EMA and a 1000 word piece. Without exception people are finally understanding that 1000 words means exactly that. People had the ignorance, arrogance or temerity to post 2,500 word pieces in the firs assignment. Some ignore the course, but wanted people to review their brilliance This is what occurs on an open platform. 

Regarding these pieces occasional requests have been made to have all of these on public view so that we could pick and choose the pieces to review. It would be quite wrong though to reveal what can be a sensitive and personal exchange between author and reviewer in a very vulnerable moment. It would, as I've seen in open 'classes' turn into a bit of a bun fight where, in the worst instances, like in the playground, you get people applauding one author and ridiculing another ... or simply join in on the back of what others have said. i.e. the learning experience is thwarted, even abused. It matters that the reviewer knows that their own words matter, without being influenced by what others have written. That said, some reviewers take a cavalier approach saying they don't like a thing, and then saying no more. If that is a student's only review received you can well understand their frustration. Even with the numbers involved somehow these pieces need to be returned and churned through the system, ideally until three to six reviews are received each. More work needs to be done to help students do reviews too so that they feel confident about doing so.

What we all benefit from this process is both learning to review, and learning to receive feedback.

The recommendation I make to everyone is to keep reviewing until you become good at it. If and when you can master reviewing, then you will be in a far better position to fairly review and edit your own work - a lesson that has finally sunk in. Writing is easy, the fun part. Jazz writing I call it. Top of the head stuff. The editing is the pain that crafts a piece so that others can enjoy it too. This pain is reduced the better you get at it.

What's revealing here is, as I've seen in the reviews, is that there are pieces that suggest that the author hasn't learnt anything at all from the course. I cannot make that assumption, so I review on the basis that it is genuine. From a formal assessment point of view, as I've learnt as a student with The OU for four years, is that a tutor looks for repeated indications that the student is using what they should have learnt from the course - if that is not present then alarm bells should ring. How can I give them points?

The problem of course in a MOOC (Massive Open Online Course) is that those who don't do the this task properly are especially taking away from genuine participants who want several considered reviews in return. Only if we get perhaps three or more reviews each will the 1 in 3 'non-review' be cancelled out.

Of 12 pieces I have come to expect 2 excellent and 2 ... how can I put it politely other than to say 'dire'. I'm not to know why this is, I can only judge what I have to review and have in mind the brief and the content of the course over the last seven weeks. Do some people paste in something they wrote months ago that has no bearing at all on the course? It seems that way. Might someone post in a piece that has been published, that is on brief? That is possible too. Of the remaining eight pieces these tend to be where errors and corrections based on the lessons of the course are most easy to make. It takes time. Time and focus. I admire those tutors, here and elsewhere who so clearly have gone to such lengths. Decades after the event I see lengthy comments on pieces for A levels from teachers who were clearly putting in a huge amount of work ... with no word limit on essays too.

In contrast, though not from The OU, I have had reviews of work that were laughable - one may have muddled me up with another student, while the other might have been written in the pub over a pint. One I made a polite complaint, gained 10 points and a distinction. The other I am about to challenge as this 'pub' idea might be close to the truth. And they are paid to assess a piece. In this instance the criticism over my missing a key point is unfounded as I make the required point a) in the introduction b) in the conclusion and c) developed the idea in the main body of the essay. Their comment, 'looks rushed' - which to my sensibilities is an indication of exactly how the tutor behaved - they are the one who were in a rush. 

Students need to be put on a confidence building exercise as they start university so that they feel, as fee payers, able to 'complain' without being stonewalled. This is another theme, but fee-paying students should and will change the attitude of institutions to their fee paying clients, rather than students on a grant-based 'freebie'.

From E-Learning V

Fig.1 My progress on The OU MOOC on FutureLearn 'Start Writing Fiction' (c) FutureLearn 2014

More than any module or exercise I have done over my four years with The OU, it is a MOOC in FutureLearn that is giving me the most thorough experience of where the future or learning lies. I'm in week seven of eight weeks of 'Start Writing Fiction' from The OU, on the FutureLearn platform. Just in these few weeks I've seen the site change to solve problems or to enhance the experience. Subtle lifts and adjustments that make a positive out of constant adjustment. Those tabs along the top: activity, replies where under a tab. I think 'to do' is new while 'progress' was elsewhere. This is a responsive platform that listens to its students.

In the final week we submit our third piece of work.

As assessments go these are far less nerve racking than a TMA. The first piece was 300, the second 500 and the last will be 1000. These are assessed by fellow students. In my case I had one, then two reviews. Most people seem to get at least two sometimes three. The system is designed, I'm sure, to try and ensure that everyone's work is reviewed at least once. Tens of thousands, certainly thousands of people are on the course.

We're here to the 19th of December or so ... if you follow the tracks as laid.  

I hazard a guess that between 20-100 have posted there final piece already. Some, I know, got to the end of the entire course a few weeks ago; I looked ahead to see out of curiosity. There have always been 20 who post comments one, two even three weeks ahead. If 20 are posting I hazard a guess knowing my stats on these things that another couple of hundred could be clicking through the pages to read and observe. They may, like me, be coming back later. They may only be following the course, but not participating. Often, it is like standing on a stage looking into the gloom of the auditorium. Someone probably out there. One or two let you know. The rest don't.

I hope those that race ahead come back ...

I find that if I get ahead then I slow down and retrace my steps. To learn in this connected and collaborative way you are far better off in the pack ... it is not a race to get to the end first. In fact, those who do this have already lost. They've missed the point. I'd suggest to people that if they have the time to do the week over. That's been my approach anyway - the beauty of these things is everyone can come and go as they please, at a pace that suits them. Skip a bit. Go back. Follow it week by week, day by day ... or not. Whatever works works?

There's another very good reason to stay with the 'pack' or to come back and do a week over - the platform depends not on tutors and moderators commenting and assessing work, but us students doing a kind of amateur, though smart, peer review. This is what make a MOOC particularly vibrant, memorable and effective. Not listening to an educator telling us what's what, but the contributors sharing, figuring it out, answering each other's problems in multiple ways. We all learn in different ways and at a pace that shifts too. I find that often a point I don't get first time round, on the second, or third, or even the fourth visit to an activity someone, somewhere puts it in a way that suddenly brings complete clarity - their way of seeing a thing, or expressing it, makes more sense than the writes of the course could manage. Because they can only write one version, not the 'tartan' that comes from an intelligent, threaded online conversation.

From River Ouse Low Tide

My tried and tested methodology, beyond the doomed 'winging it' is 'concrete aggregate'. Other weeks or months I accumulate a lot of stuff, much of it in a blog like this; not quite a relational database but the 'stuff' is here, tagged and of reasonable relevance. In a now defunct OU ePortfolio called 'MyStuff' or 'MyOU' - I forget, you could then shuffle and rank your gobbets of nonsense and so, discounting the volume of stuff, potentially, have a treatment that could then be turned into an essay.

Such stuff, if it contains, 10,000 words, often with chunks of verbatim passages, can be a hell of a task to hack into shape. You build in bold forms out of concrete and can only get it to look like a garden, or park sculpture, with a pneumatic drill and chisel. Sometimes it works. You get there. It is dry and workable. You'll more than pass. It depends on the subject, the module and the specific expectations of the assignment. Where you need to tick many boxes this approach may work well.

Clay is the better way forward in most situations. Here you build up your arguments in logical steps then refine them at the end. This, particularly in the social sciences, is where the tutor wants to see how you argue you case, drawing together arguments and facts, mostly those you've been exposed to in the module, though allowing for some reading beyond the module. You have to express your opinion, rather than listing the views of others. Get it right and this is the only way to reach the upper grades? Get it wrong, which is the risk, and you may end up with a hollow or limp structure with grades to match. 

I cracked some ten days ago; unwell, not so unwell I couldn't flop about the house, bored and managing to read, but my brain too lacklustre to take notes or think. So I tried this.

The pilot episode was magic - like the second act of Macbeth. It all happens and you're left eager for more. 36 episodes later and it goes through a slow patch, but now I'm on the home run and can see the premise of it all. This happened to coincide with a sessions I attended on writing fiction in which I learned that 'character is plot' - in this case character is also premise and story arc. So, not on a creative writing course but there's a load to learn about writing fiction here.

$3m an episode!? Shot on 35mm film. $200m for all six series?!! 

It shows. The production values are those of a movie.

The Open University has equivalent production values; you get into bed with the BBC on 'The Frozen Planet' or some such and spin off Open Learn tasters and various modules. But as a learning platform does video teach? Rather does it inspire and motivate?

Drama is about how you are made to feel, not how you come to think?

Advertising persuades, it has to. Why are 'educational videos' more like TV commercials then?

Donkeys years ago we made 'advertorials' - extended, persuasive, corporate, sponsored learning content put out on video, then distributed by satellite and eventually put on internal networks. I have to wonder if a leaflet would not have worked better and achieved as much for less, but as it was pointed out, no one read the leaflets, at least a training manager knew his workforce had watched 'the video' because they had the attendance sheets.

Were they given a test afterwards though?

That's the way to turn it into learning. 'After this video, or episode you will be given a test on .... '

Looking back over four years can be revealing. In 2010 I was struggling with my third Tutor Marked Assignment (TMA03)  in the Master of Arts Open and Distance Education (MAODE). I queried the assignment process. In particular I felt as if it was akin to making a tapestry. Three years on I feel my last couple of TMAs and EMAs weren't simply making a tapestry - but doing so wearing roller-skates on a ship in a heavy sea.

Nothing would stitch together. I had far too much wool. 

The criteria, meant to be helpful, detailing paragraph, by paragraph if not sentence by sentence what the examiner is looking for ties my head in knots. 

I contrast this, favourably, with the MA I am doing with the University of Birmingham. Clunky but effective. I have a reading list. We attend lectures and take part in seminars and I write a 4000 word essay drawn from a list of 12 to 16 titles.

This allows me to be fluid, rather than the ground beneath my feet.

Throughout the MAODE I think the only module the regularly had this 'essay writing' approach was H809: Research practices in educational technology. 

I can be accused of over thinking and over preparing a TMA or EMA - yet this, too often, is how the things have been designed. Less would be more. Simpler would not be easier, writing is hard enough without having to second guess what a third party will be thinking as they read while running down a check list to give you a tick and therefore a mark and ultimately a grade.

Reflecting on four years I can see marks in TMAs, and EMAs especially, improve. I think TMAs in 60s, and 70s and the odd EMA in the 40s, then 50s give rise to TMAs of 80s, even in the 90s, though my best EMA was a 76. Of course, in their wisdom, my student grades for each module simply reads 'PASS'. I feel this rather diminishes the effort and evidence. There is certainly a different between a candidate scoring in the 40s and 50s between one scoring in the 60s and 70s and 80s. 

I met an MBA student who had achieved a distinction in every module. I was in awe. Not your usual OU student (are any from the Business School). She had a first in her first degree from Oxford: Classics. Some people have a mind for these things. Perhaps it is my head that sloshes around like the proverbial storm, rather than the system I have been part of?

I just watched Daphne Koller's TED lecture on the necessity and value of students marking their own work. (for the fifth time!)

Whilst there will always be one or two who cheat or those who are plagiarists, the results from 'Big Data' on open learning courses indicate that it can be a highly effective way forward on many counts.

1) it permits grading where you have 1,000 or 10,000 students that would otherwise be very expensive, cumbersome and time consuming

2) as a student you learn from the assessment process - of your work and that of others

3) student assessment of other's work is close to that of tutors though it tends to be a little more harsh

4) student assessment of their own work is even closer to the grade their tutor would have given with exceptions at opposite ends of the scale - poor students give themselves too high a grade and top students mark themselves down.

Conclusions

a) it works

b) it's necessary if learning reach is to be vastly extended

c) isn't human nature a wonderful thing?! It makes me smile. There's an expression, is it Cockney? Where one person says to another 'what are you like?'

Fascinating.

'What are we like?' indeed!

Sitting an exam for the first time in 28 years got me thinking how so much is assessed by assignments and 'doing'.

Just clinging to a pen for more than 5 minutes is a novelty to me.

Surely the technology we now have is capable of 'getting into my head' to show that I do or do not know my stuff. But here's the difference, have I been taught to pass an exam which could only prepare me to become an academic, or have I been applied to apply what I have learned which is very different.

I know what it takes but don't feel that the course has prepared me for a written exam.

We need to have been writing an essay a week, not one every six weeks.

Then, armed with your best essays you reduce these down to key themes.

From these themes and a list of must have authors I dream up a few mnemonics and mind maps that I can 'dump' onto a blank sheet in the first 5 to 10 minutes so I have these as an aide memoire.

Recalling distant early summers revising for exams I am doing all of this in the sun, either on the South Downs or today on the shingle beach at Seaford.

There are three course books and three blocks.

I am ploughing through these at the rate of 3 hours per day reducing each to key thoughts and must have ideas. I'll then test myself repeatedly until I can get a good range of ideas and evidence onto a sheet. Whilst I don't like exams I can there is no alternative, that assessments alone sent in electronically are too prone to patchy work even plagiarism, that being galanised to get the right stuff into your head means something, all the better if it is applied.

Between formal and informal learning design styles

Highly prescriptive vs call up the information you need as you go along. So defined instructions vs figuring it out for themselves. Perhaps using tools to guide and inform. We were given the simplest of tasks, teaching students how to cross a busy road in safety.

I made a point about any group requiring leadership or a champion.

I made a point on Randy Pausch whose 3D lecture series included mixing up student groups who then had to vote on each other's levels of collaboration in the group.

There was a discussion on informal peer assessment that I didn't entirely follow, certainly my notes are somewhat cryptic. Hopefully the session was recorded so I can listen back.

It is the unexpected insights in a synchronous session that prove valuable, especially the asides in the break-out room.

Assemble the books and papers you plan to refer to ahead of writing.

This is a new one to me. I prefer to write what I want and deal with the referencing after rather than fitting the assignment to the books and papers.

Perhaps a combination of the two is required.

The learning plan you produce may not be followed closely given the myriad of ways people respond. Many are drawn in by the assessment but not all.

Can such divergent styles be accommodated?

As the Elluminate discussion progressed, four students, one tutor moderator, I did a doodle.

Having shared the idea I then corrected it.

From Wenger (1998:233) 

'There is an inherent uncertainty between design and its realization in practice, since practice is not the result of design but rather a response to it'.

Phenomonology explains why people may still be adrift of the desired response.

The notes reads 'design as well as we can ... 'the students share the outcome. We set the learning, that is then displaced to or set in the context of each learner. We might have a learning objective, but students can and diverge from this'. (A good thing if you want diversity and originality)

As a learning designer you have to anticipate a variety of behaviours and plan for not too many being wildly divergent. This can be achieved by understanding the students.

My concern, as I'm guilty of it as a blogger, is that it changes how you 'talk.'

My blogging voice is light and journalistic - it is difficult to escape this in an assignment, however many references I put on or re-writes I do.

A different mindset is required, literally wearing a different hat and taking a different approach from the start.

As a professional writer I ought to be able to write for different audiences. I'm not sure I can, indeed my voice has always bee the 'spoken word' for TV and Video with scripts design to visualise rather than say anything at all.

I find and consider Twitter to be an invaluable exercise in being succinct; stylistically this has to be a good thing.

However, as I found myself doing recently, something I wrote, after an edit, looked and read like x16 140 character Tweats strung together.

Surely engagement of any kind, a conversation over coffee, over lunch, Tweating or blogs, helps internalise and sought out issues and confusion in the student's mind. It is an activity even if it is being measured?

I wonder if a 'viva voce' in a video conference (Skype, Elluminate) wouldn't demonstrate the value of social networks in education, that it would be apparent that those who are talking about their topic in cyberspace are more likely to have formed some points of view of their own.

(49002)