OU blog

Personal Blogs

Design Museum

Embedded Formative Assessment by Dylan Wiliam

Visible to anyone in the world


Pghe2QUydgC8RiqGNR_40Nuvy1cCflch60xtgto46zvdi6KG-bqJKalQTelVd_9MbAgDQUVzW-TseQHVOxV3OqpMezznrISOkgqoBatyTSepKqWFd9ojF49SZyvgJ3qlGjYQ7U6q

William. D (2017) Embedded Formative Assessment: (Strategies for Classroom Assessment That Drives Student Engagement and Learning) (The New Art and Science of Teaching) : Second Edition. Bloomington, USA. Solution Tree Press. 

Dylan Wiliam is to formative assessment what Chris Witty is to Covid

I’m getting used to Dylan Wiliam - I think of him as the David Walliams of Education, the Will-I-Am of Formative Assessment.

I’ve been pointed at his webinars, sat through a keynote or two, read the book and the read pamphlet ‘Inside the Black Box: raising standards from classroom assessment’ which he produced with Paul Black in 1998 and established his reputation. 

In my first pass of note taking I had over 60 quotes and ideas here, not all from Dylan William. With some editing I’ve got this down to 19 ‘Top Tips’ which are all set out ready to quote. 

I’ll go back to the book if needs be for clarification. In this case it is an eBook which makes it very easy as the digital version will correlate to actual print page numbers.

This is how I learn. 

It’s a slow and repetitive process: skim read a book to get the lay of the land, read it with an open notebook, then transfer the notes to a document like this. Then edit the notes and where necessary go back to the original book. Most important of all - give it a go. Over the last five weeks I’ve had a chance to apply some of the thinking to both online remote teaching and in the classroom face-to-face.  

When a book strikes me as really important I am likely to have it in print and eBook - they read differently and you take different things from each. 

These notes are for me to browse, refer back to and use when I next have a formal rationale or observation to write, as well as for every day reflection on the learning and e-learning experience I am currently going through. 

Myth Busting

Wiliam puts the evidence of learning above myth and has a number of bugbears. 

Good teaching is difficult

It is relatively easy to think up cool stuff for students to do in classrooms, but the problem with such an activity-based approach is that too often it is not clear what the students are going to learn. (Wiliam 2017 p.94) 

Wiliam disregards learning styles, ‘which have no discernible impact on student achievement at all’. (Wiliam 2017 p.11 also Adey, Fairbrother, William, Johnson, & Jones, (1999). Rather - ‘as long as teachers vary their teaching style, then it is likely that students will get some experience of being pushed beyond it’. (Wiliam 2017 p.48) 

Nor is Wiliam a  fan of ‘performance of a learning task’ as a predictor of long-term retention of learning (Wiliam 2017 p.47 from Bjork, R.A. (1994) Any mention of ‘neuroscience’ as the panacea annoys him. (Wiliam 2017 p.50) 

There is no shortcut

Wiliam’s firm belief is that formative assessment improves performance. (Wiliam 2017 p.11) His view - ‘the use of assessment for summative purposes - grading, sorting, and ranking students - gets in the way’ of learning. (Wiliam 2017 p.56) Education is overly prescriptive with rubrics. (Wiliam p.93 in Alfied Kohn (2006)

No one can do the learning for the student who does not engage.

Our classrooms seem to be based on the principle that if teachers try really hard, they can do the learning for the learners. (Wiliam 2017 p.225) 

According to Wiliam there is little evidence that the following ‘tricks’ have any impact of student achievement:

  • Summarisation

  • Highlighting

  • Keyword mnemonic

  • Image use for text learning

  • Rereading (Wiliam 2017 p.225) 

Wiliam sets out five key stages of formative assessment : 

  1. Clarifying, sharing and understanding learning intentions and success criteria.

  2. Eliciting evidence of learning

  3. Providing feedback that moves learning forward

  4. Activating learners as instructional resources for one another

  5. Activating learners as owner of their own learning

The Assessment Reform Group puts it this way. Assessment to improve learning requires five elements to be in place (cited Broadfoot et al., 1999) (Wiliam 2017 p.61)

  1. Providing effective feedback to students

  2. Actively involving students in their own learning

  3. Adjusting teaching to take into account the assessment results

  4. Recognising the profound influence assessment has on student’s motivation and self-esteem, both of which are crucial influences on learning.

  5. Needing students to be able to assess themselves and understand how to improve.

Challenge them!

I like Wiliam’s thinking that ‘good instruction creates desirable difficulties’ - that we learn and retain knowledge through struggling with it. (Wiliam, 2017 p.11) I know from personal experience that the greater the struggle I am willing to endure, the deeper and the longer lasting my learning. It is when I can face the struggle … that I struggle.  Good instruction creates what he describes as ‘desirable difficulties’ (Wiliam, 2017 p.47) quoting Bjork (1994, p.193).

Pedagogy Over Curriculum

‘A bad curriculum well taught is usually a better experience for students than a good curriculum badly taught’. (Wiliam, 2017 p.11) I rather think we can apply this to many situations, for example, that pedagogy comes first - not EdTech. 

Formative Assessment above all else 

The Wiliam mantra is that, ‘attention to minute-by-minute and day-to-day formative assessment is likely to have the biggest impact on student outcomes’. (Wiliam 2017 p.42) He defines formative assessment as ‘the process used by teachers and students to recognise and respond to student learning in order to enhance that learning, during the learning’. (Wiliam 2017 p.59). Formative assessment, according to Bloom (1969) is a kind of evaluation - ‘a brief test used by teachers and students as aids in the learning process. (Wiliam, 2017 p.53 in Bloom (1969) It is ‘the process used by teachers and students to recognise and respond to student learning in order to enhance that learning, during the learning. (Cowie & Bell, 1999 p.32) It is ‘assessment carried out during the instructional process for the purpose of improving teaching or learning’. (Wiliam 2017 p.59 in Shepard et al., 2005 p.275) 

I like this : ‘Frequent, interactive assessment of students’ progress and understanding to identify learning needs and adjust appropriately.’ (Wiliam 2017 p.59 from Looney 2005)

And if you need more ways to think of it, try this: ‘assessment, for learning tells ‘us’ ‘what progress each student is making toward meeting each standard while the learning is happening - when there’s still time to be helpful (Wiliam 2017 p.62 from Looney 2005 (pp1-2) Stiggins (2005)

All kinds of formative assessment are not equal

(Wiliam 2017 p.62) 

‘The evidence is clear that the shorter the assessment - interpretation - action cycle becomes the greater the impact on student achievement’. (Wiliam, 2016). He continues, ‘short-cycle formative assessment has to be the priority for schools and teachers, because the impact on students is greater. (Wiliam p.75) 

‘ … regular use of minute-by-minute and day-to-day classroom formative assessment can substantially improve student achievement’. (Wiliam 2017 p.81)

Design backwards from the learning outcome

(Wiliam 2017 p.81 from Grant Wiggins and Jay McTighe (2000)

Three issues in the development of learning intentions and success criteria that may be useful to think about. (Wiliam p.95)

  1. Task-specific versus generic scoring rubrics.

  2. Product-focused versus process-focused criteria

  3. Official versus student-friendly language

NOTE :> By being specific about what we want, we focus the students learning too much. ‘He who never made a mistake, never made a discover’ (Wiliam 2017 p.95 quoting Samuel Smiles (1862 p.275)

There are a number of techniques which appealed when I read about them and I have used them. These are techniques to help students understand and achieve learning intentions. (Wiliam 2017 p.104 from Clarke (2001)

This is simple, these phrases work 

WALT ‘We are learning to … ‘

WILF ‘What I’m looking for … ‘

TIB ‘This is because … ‘ 

Student Engagement Techniques

The teacher asks a question, selects a student to answer the question and then responds to the student’s answer, which is generally some kind of evaluation of what the student said. (Wiliam 2017 p.124) Teacher-Led classroom discussion.

Try some of these:

  • Wait time

  • Evaluative listening

  • Interpretive listening

  • Questions shells

  • Hot-seat questioning

  • All student response systems

  • ABCD Cards

  • Mini Whiteboards

  • Exit passes

  • Discussion vs. diagnostic questions

  • Alternative questions

Pose a question, pick a student at random

Pose-pause-pounce-bounce

  • Pose

  • 5 seconds - pause

  • Pounce-random choice

  • Bounce-what do you think (Wiliam 2017 p.129) 

App or Lollipop sticks (Wiliam 2017 p.129) 

These techniques are used to ensure that all students realise that all are expected to tke part and that it is ok to make mistakes. If someone does not provide a response then come back to them and tell them  “OK, I’ll come back to you”.  (Wiliam 2017 p.132) 

Then seek Response 2 and 3 and then return to the person who gave no response and ask them which reply they liked the best and ask them why.

‘Engagement and responsiveness - are at the heart of effective formative assessment’. (Wiliam 2017 p.142) 

Managing Challenging Behaviour

To change the behaviour criticise the behaviour not the student. (Wiliam 2017 p.171) 

It is quality rather than the quantity of praise that is most important - teacher praise is far more effective if it is infrequent, credible, contingent, specific, and genuine. (Brophy, 1981 in Wiliam 2017 p.171)

The use of feedback improves performance when it is focussed on what needs to be done to improve, and particularly when it gives specific details about how to improve. (Wiliam 2017 p.180)

Motivation

“It's up to me, and I can do something about it”.  (Wiliam 2017 p.183)

When students have to struggled in the learning task, the quality of their performance on this task reduces, but the amount of learning that takes placed increases (Wiliam 2017 p.190)

Feedback functions formatively only if the learner uses the information feedback to him or her to improve performance. If educators intend the information fed back to the learner to be helpful but the learner cannot use it to improve his or her performance, it is not formative.

Motivation is not a cause but a consequence of achievement (In Wiliam 20176 p.234) from Garon-Carrier et al., 2016)

Like sports coaching, teaching takes time to master

It takes years for even the most capable of coach to break down a long learning journey from where the student is right now - to where he or she needs to be. (Wiliam 2017 p.193)

Feedback should cause thinking

Feedback for Future Action. (Wiliam 2017 p.194) 

To be effective, feedback needs to direct attention to what's next, rather than focusing on how well or poorly the student did on the work, and this rarely happens in the typical classroom.

The response from the student to feedback should be ‘cognitive rather than emotional’ (Wiliam 2017 p.205) In other words, feedback should cause thinking by creating desirable difficulties.

Peer tutoring can be more effective than one-on-one tutorial instruction from a teacher. This is because of the ‘change in power relationships’. (Wiliam 2017 p.209)

And regarding students online not using their webcams he believed you can see how a student is really taking it by seeing their faces (Wiliam 2017 p.209)

Student Reporter

Put students in a group towards the end of the class so that they can discuss then report back on what has been taught.

Two Techniques that work 

(Wiliam 2017 p230)

  1. Practice Testing 

  2. Distributed Practice

These received high ratings because they were effective with learners of different ages and abilities and were shown to boost students’ performance across many kinds of tasks, and there was plenty of evidence that they worked in educational contexts.

If students complete a practice test and get immediate feedback on their answers, students will get the benefit of the hypercorrection effect for those questions where they were correct. (Wiliam 2017 p.231)

Setting Goals

Students are more motivated to reach goals that are specific, are within reach, and offer some degree of challenge. (Wiliam 2017 p.236 in Bandura, 1986)

When the goals seem out of reach students may give up on increasing competence and instead avoid harm, by focusing on lower-level goals they know they can reach or avoiding failing altogether by disengaging from the task. (Wiliam 2017 p.236)

TIPS

  1. Think ‘how am I going to teach this and what are the pupils going to learn?’ (Wiliam 2017 p.79) 

  1. Having an in-depth understanding of the curriculum may be of more benefit to student progress than advanced study of a subject on the part of the teacher.

  1. Students don't learn what we teach. (Wiliam 2017 p.77) 

  1. Teaching the goal. Driving as teaching. (Wiliam 2017 p.78)

  1. ‘When the pressure is on, most of us behave as if lecturing works but deep down inside we know it’s ineffective’. (Wiliam 2017 p.80)

  1. The teacher’s job is not to transmit knowledge, nor to facilitate learning. It is to engineer effective learning environments for students. (Wiliam 2017 p.80)

  1. Learners of all ages need to understand what it is that they need to learn and be able to monitor their progress toward their goal. (Wiliam 2017 p.95) 

  1. Don’t simply plan the instructional activity, but also plan how you are going to find out where the students are in their learning. You need to be clear about what we want students to learn (Wiliam 2017 p.115)

  1. Ask questions either to cause thinkin and to provide information for the teacher about what to do next. (Wiliam 2017 p.126)

  1. Beware - those avoiding engagement are forgoing the opportunities to increase their ability.

  1. The most important factor influencing learning is what the learner already knows - the teachers job is to ascertain this and to teach accordingly. (Wiliam 2017 p.166) 

  1. Much of the feedback that students get has little or no effect on their learning, and some kinds of feedback are actually counterproductive. (Wiliam 2017 p.167) 

  1. Do not mix grades and comments, just stick to comments. (Wiliam 2017 p.167) 

  1. Oral feedback is best. (Wiliam 2017 p.174)

  1. Too often feedback is counterproductive. (Wiliam 2017 p.178)

  1. Don’t provide students with feedback unless you allow time, in class, to work on using the feedback to improve their work. (Wiliam 2017 p.195) 

  1. Feedback should be more work for them than you! (Wiliam 2017 p.195) 

  1. A simple approach to feedback. Pick out two things to praise, then express what they need to do a constructive wish. He calls the technique ‘two stars and a wish’. (Wiliam 2017 p.214)

  1. Teachers have a crucial role to play in designing the situations in which learning takes place, but only learners create learning. Wiliam 2017 p.246)

REFERENCES

Adey, Fairbrother, William, Johnson, & Jones, (1999) p.36 A review of learning styles and learning strategies. London. King’s College London Centre for Advanced Thinking. 

Ausubel (1968) Educational psychology. A Cognitive view. New York. Holt, Rinehart & Winston

Bjork, R.A. (1994) Memory and metamemory consideration in the training of human beings. In J.Metcalfe & A.P. Shimamura (eds) Metacognition : Knowing about knowing (pp.188-205) Cambridge, MA MIT Press. 

Bloom, B.S. (1969) Some theoretical issues relating to educational evaluation, In H.G. Richey & R.W. Tyler (Eds). Educational evaluation : New roles, new means, part 2 (Vol.68, pp. 26-50_ Chicago : University of Chicago Press.

Brophy, H (1981) Teacher praise: A functional analysis. Review of Educational Research, 51 (i), 5-32

Clarke, S (2001) Unlocking Formative Assessment. London, Hodder & Stoughton

Cowie, B & Bell, B (1999) A Model for formative assessment in science education. Assessment in Education : Principles, Policy and Practice. 6(1), 101-116

Looney, J (ed) (2005) Formative Assessment: improving learning in secondary classroom. Paris. Organisation for Economic Co-Operation and Development.

Shepard et al., 2005 p.275 in L.Darling-Hammon & J.Bransford (Eds) preparing teaching for a changing world : what teachers should learn and be able to do (pp.215-326)

Soderstrom, N.C., & Bjork, R.A. (2015) Learning versus performance : An integrative review. Perspectives on Psychological Science 10 (2), 176-199. 

Stiggins R.J. (2005) Assessment for learning defined. 

Wiggins, G., & McTighe, J. (2000) Understanding by design. New York. Prentice Hall.

William, D. (2016) Leadership for teacher learning . Creating a culture where all teachers improv so that all students succeed. West Palm Beach , FL, Learning Sciences International.

William, D (2017) Embedded Formative Assessment: (Strategies for Classroom Assessment That Drives Student Engagement and Learning) (The New Art and Science of Teaching) : Second Edition 

Permalink Add your comment
Share post
Design Museum

What I don't know about teaching

Visible to anyone in the world
Edited by Jonathan Vernon, Monday, 14 Dec 2020, 06:53

Book Cover for kate Clanchy's 'Some Kids I Taught and What They Taught Me'.

A reivew of sorts written in several parts over 12 hours between 8:00am and 8:00pm this wet, dull, claustrophic, semi-lockdown Sunday on the Western edges of Lewes ... 

I'm 60 pages in and when I open the book I still get a waft of fresh paper. I need a book like this as a reality check and reminder of who matters in all of this education malarkey - the student ! 

I'll grow to feel in control, learn quickly ways to support their learning.

I like this:

'It's a bodily experience, like learning to be a beekeeper, or an acrobat: a series of stinging humiliations and painful accidents and occasional sublime flights which leave you either crippled or change'. p.1

How can teaching online via webcam be a bodily experience? An 'out body' experience perhaps? Not helped where none of the students are present - all have their webcams turned off. 

Kate Clanchy supports my idea regarding motivation, she makes this optimistic remark about her students (she teaches in school) 'all children will behave perfectly ... if they want to know something very much, about sex or anything else, and an adult sincerely sets out to tell them.' p.15

The issue of course if neither the students not the teacher want to impart knowledge: take 'Black Lives Matter'. Not my view, but I have had others ask why it is being taught to a group of all white students - or harder, how to teach it to a class where one student is black. Should it be awkward? Would it be like teaching menstruation to a class of predominantly boys with one girl present? What if the class was entirely black but for one white kid? I think the teachers missed the point, the institution failed in selling the purpose of teaching the class and in fact should it not be the case that only those who know what they are talking about get up to teach?

A further insight into the students comes where the author sums up her experience of working (her choice) with students who had been isolated in the 'inclusion' portacabin on the other side of the playing field. These children had misbehaved so badly or so often that they were separated from the rest of the school. Kate Clanchy is a saint; she has a lot to teach us. She tells a story of how she encouraged them to write her notes and put them anonymously in a box. Those that were scrumpled up and dropped in the bin proved more revealing as it told stories of physical and mental abuse and neglect. We know that how children are raised has a profound impact on their behaviour and response to the wider world. 

'No one is bad, though many are sad, and a few are mad' she writes on page 56.

I'm reading a lot at the moment. 

In between reviewing 1 hour 42 minutes of Dylan Wiliam on 'Formative Assessment' and the need for actions in schools to be based on evidence and checking through Dave White on 'Visitors and Residents' in the digital world, I riggle my way through the rest of Kate Clanchy. 

There are no surprises that she uncovers systemic racism in poetry competitions she enters her pupils for, no surprises at how awkwardly church schools fit into secular, or rather multiracial Britain, nor how middle-class parents tend to point their kids towards middle-class and aspiration schools leaving the general population lacking in a proper understanding of the communities around them.

Very Quiet Foreign Girls is worth Googling for their poems. Like 'Dead Poets Society' this is a group of underprivileged girls, rather than privileged boys, who met to read and compose poetry. The multiracial and international mix of students is extraordianry: Khurds, Iranians, Somalians, Poles and Hungarians, Moroccan, Afghan, Indian and Pakistani with a suitable mix fo relgions across branches of Islam, as well as Hindu, Cathollic and no religion at all.

Deprivation can be a shocker: the way the children live, their poverty, how treated at home, the uniform a release from having to find anything special or different to wear, shoes in all weathers a pair of flip flops, travel to London from Essex, let alone 'abroad' a signal of something 'beyond' and out of their reach to the point of feeling like impostors to be with anyone so privileged. 

'Poverty is stronger than plumbing' Kate Clanchy writes (p.160), 'stronger than medicine, stronger than art'. 

The first taste I got of it was on benefits in London in April 1985; not an expected path for an Oxford graduate who'd been spoilt for choices at the end of the Milk Round the Year Before. Then doing odd jobs, in a flat in Willesden and joining the Tricycle Youth Theatre and being around as many black faces as white. 

I'd not lived; I'd not travelled in my own country. I've been rubbing off the public school ever since and have taken a long time to reclaim Oxford rather than simply stating that I went to 'college'. Elitism comes in many forms and I am guilty of belonging to a few of them in the past (not always by choice).

Education is national, it is the community, it ought to be a melting pot, it ought to be a leveller. It should not be the fragmented, privileged, excluding, isolating experience that it is in Britain where too many children's experience is amongst 'their own kind' geographically, and by race, religion, class and wealth. 

Kate Clanchy has been what has made this a weekend, not a workday; my working week extended with the Dylan Wiliam to digest (it will take three passes through my gut like a cow chewing the cud). What surprises me is when out of the blue she drops her pen and smacks into Black and Wiliam and the entire idea of Formative Assessment (WALT) with the enthusiasm of a vengeant pugilist. I like her for it. My first notes on Wiliam are to question the keynote I have just sat through as a self-serving literature review which makes a lot of poor research conducted in the States simply so that he can destroy it. More of Wiliam elsewhere - I applaud 'evidence based' responses to any problem (though not, Kate Clanchy would say) at the expense of creativity and poetry in particular. is Wiliam and his brand of formative assessment most suited to math and engineering rather than the arts? Would fine art, pottery or make up benefit from or be destroyed by formative assessment. At what point does formative assessment in the training of a competitive sailor have to give way to intuition, for the musician to play the piece their way? Is formative assessment the scales and these alone will make for a dull clone?

We get into the apostrophe in English as a defining standard for how well it is taught, or not, and taken up by Grammar Schools but not the Comprehensive - or not. It is a detail too far for me. I take her point that the simplest advice to those in doubt over the use of the apostrophe is to never use it.

From p.207 to p.217 the PostIts cover most of each page. Her attack on formative assessment is heartfelt. I need, in the parlance, to ‘unpack it’. Kate Clanchy ‘began teaching thirty years ago’ (1990s). Since when there has been ‘the inexorable rise of the thing called ‘formative assessment’, and its lumpen classroom equivalent, the WALT’. '(p.207) 

WALT stands for ‘We Are Learning To …’

They should head a trainee teacher’s lesson plan and guide any observation.

The theory goes, Kate Clanchy explains, is that they ‘interact seamlessly with the curriculum and let everyone know where they are at’. 

‘They break up the lesson into simple learning objectives that the children themselves understand’. (p.207) 

‘This is formative assessment because it forms and changes he student as well as marking them’. (p.208).

[I can only think in terms of old school essay writing for homework. Formative assessment at its most successful, for feedback and differentiation - surely? And then the five hundred year old Oxbridge tutorial were students armed with essays debating one with their tutor and mentally marking their own effort as excellent, average, mediocre or non-existent, while forming a view of their own months, even years before any summative assessment into a formal written exam]. 

‘Formative assessment does not allow for ineffable processes,’ she writes (p.210) as she expands on a case study of a student who grew into himself and developed self awareness and confidence as a result of his creative writing, something she is sure would have been stymied by WALT and overly prescriptive formative assessment. 

She has a dig at something called the ‘Black Box’ which is an idea that Black and Wiliam also developed around WALT and formative assessment; I am currently ignorant of it. She argues that often there is less need for this kind of formative assessment and a greater need for summative assessment in the form of a concluding ‘well done’. (p.210).

Then she bemoans how an English test has been reduced to using a Wikipedia entry on Titanic to compose a nonfiction essay. A task that she considers thin and limited because it starts from not much and has little opportunity to flower the way the simple experience of listening to a poem and then writing one of your own can have. (p.211).

‘In my dress,’ she writes on p.212, we never need to write another WALT. In my dreams my colleagues are trusted to choose great, rich texts to teach, and we all trust the texts to teach the children. We assess both creative and critical responses to them as their final exams’. 

Some decades later Kate Clanchy spotted this her most promising student from those days of poetry and creative english. She got up the courage to contact him by LinkedIn.

‘When we read books with you,’ he wrote, ‘the world opened up. Your lessons were I learnt who I was,’ he continued, ‘became conscious of myself, grew up. That time was important to me, a free space’. (p.217)

She goes on to provide illustrative anecdotes for dyslexia and ASD I like that as much is devoted to what those with such issues can do, rather than what they can not do. Then we cover body image, the Audrey Hepburn like nymph who gains weight so quickly and others too, another raped and a third caught up a child marriage or engagement in Pakistan.






Permalink Add your comment
Share post

This blog might contain posts that are only visible to logged-in users, or where only logged-in users can comment. If you have an account on the system, please log in for full access.

Total visits to this blog: 6524548