OU blog

Personal Blogs

Design Museum

Q is for QStream

Visible to anyone in the world
Edited by Jonathan Vernon, Tuesday, 3 Jun 2014, 12:15


Format for a randomized control trial used by 'Spaced-Ed' (now QStream)

Ok, this is an odd one, but for me, over the last four years, finding out about, then reading the papers on this learning platform has shaped a good deal of my thinking and motivation. Developed by a Harvard Medic and masters postgraduate Dr Price Kerfoot, 'Spaced-Ed' as it was first called tackles the problem of forgetting; it is, to put it simply, an electronic set of flash cards. Say this to junior doctors with a hundred such cards to learn over a few months in order to pass a compulsory written exam though. Content is vital of course, but then the platform simply feeds you the 'cards' by email and link to a webpage as frequently or as infrequently as you wish - six questions in batched of three twice a week worked for me. You get all the questions back at least three times even if you get them right, while you keep getting the questions for those you get wrong UNTIL you have got it right three times. It works. Randomised controlled trials with hundreds, even thousands, show that it is an effective way to put knowledge into heads. That's just the start though. Education changes behaviour, yes, it makes better doctors, it improves decisions making, it gets them through exams.

Quality Assurance of course ought to be my 'Q' but this is like enthusing about cars Top Gear style and then saying you must remember to check the oil, breaks and tyres and have an annual MOT. I have worked professionally in QA too - it matters to check everything in the finest detail if learning is to work and clients are to be pleased.

QR Codes is simple a form of link to webpages that can be used for learning. There are other quick links to the web, including 'near field codes' and image recognition. This is like replacing the traditional car key with an electronic fob; it speeds things up. There are imaginative ways to use QR codes. During H818 I developed the idea of putting them on Commemoration Poppies to link directly to people remembered from the First World War.

Permalink Add your comment
Share post
Design Museum

What e-learning looks like

Visible to anyone in the world
Edited by Jonathan Vernon, Tuesday, 5 Feb 2013, 12:01


Fig. 1.

Spaced education Randomized controlled trial. Kerfoot et al 2008.


Since 2006 Harvard Medical School has used a straightforward spaced education tool to deliver sets of questions to medical students. These multiple choice questions are both the resource and the test, they teach as you go through the 'pack'. In the case of urology there are 100 questions. A randomised controlled test shows how those who use the system do so much better than those who do not. It helps to lodge information in the long term memory, by feeding questions back to participatns, even once they have got them right, the natural tendency to forget is thwarted.


Kerfoot, B, P (2008) Interactive Spaced-Education to Teach the Physical Examination: A Randomized Controlled Trial. Armstrong,E.G., O’Sullivan,P.M., JOURNAL OF GENERAL INTERNAL MEDICINE Volume 23, Number 7 Harvard Medical International.

Kerfoot, B, P., & Brotschi, E (2009a), ‘Online spaced education to teach urology to medical students: a multi-institutional randomized trial’,American Journal Of Surgery, 197, 1, pp. 89-95

Matzie, K, Kerfoot, B, P., Hafler, J, & Breen, E (2009b), ‘Spaced education improves the feedback that surgical residents give to medical students: a randomized trial’, American Journal Of Surgery, 197, 2, pp. 252-257

Kerfoot, B, P., Armstrong, E, & O'Sullivan, P n.d., (2009c) 'Interactive spaced-education to teach the physical examination: A randomized controlled trial', Journal Of General Internal Medicine, 23, 7, pp. 973-978, Science Citation Index, EBSCOhost, viewed 5 February 2013.

Shaw, T, Long, A, Chopra, S, & Kerfoot, B,P. (2011), ‘Impact on clinical behavior of face-to-face continuing medical education blended with online spaced education: A randomized controlled trial’, Journal Of Continuing Education In The Health Professions, 31, 2, pp. 103-108



Share post
Design Museum

We forget, it's only natural - what can we do about it?

Visible to anyone in the world
Edited by Jonathan Vernon, Friday, 21 Dec 2012, 08:40


Fig.1. The Forgetting Curve. Ebbinghaus (1885)

'The psychological conclusion demands a distribution of repetitions such that some of them should be produced at a later time, separated from the first repetition by a pause'. (Vygotsky, 1926)

More recently, in the last ten years in fact, Dr B Price Kerfoot of Harvard Medical School (2006) created a platform called SpacedEd (now Qstream) that uses multichoice questions, typically and most successfully with first year medical students, where sets of questions are randomised then sent out as text or email to tackle, I suppose, what Ebbinghaus (1885) identified with his 'Forgetting Curve'. An evidence based paper on the effectiveness of 'spaced learning' showed how there was better retention three months, six months and a year down the line.


Ebbinghaus, H (1885) Memory: A contribution to experimental phsychology.

Kerfoot, B, P (2006) SPACED EDUCATION. Interactive Spaced-Education to Teach the Physical Examination: A randomized Controlled Trial.

Vygotsky, L (1926) Educational Psychology


Formative Tests Aid Retention


Permalink Add your comment
Share post
Design Museum


Visible to anyone in the world
Edited by Jonathan Vernon, Sunday, 4 May 2014, 11:43

SpacedEd is a platform designed to allow learners and teachers to harness the educational benefits of spaced education.

Spaced education is a novel method of online education developed and rigorously investigated by Dr. B. Price Kerfoot (Associate Professor, Harvard Medical School).

It is based upon two core psychology research findings:the spacing effect and the testing effect.

In more than 10 randomized trials completed to date, spaced education has been found to:

  1. Improve knowledge acquisition
  2. Increase long-term knowledge retention (out to 2 years)
  3. Change behavior
  4. Boost learners' abilities to accurately self-assess their knowledge.
  5. In addition, spaced education is extremely well-accepted by learners.
  6. The spacing effect refers to the psychology research finding that information which is presented and repeated over spaced intervals is learned and retained more effectively, in comparison to traditional bolus ('binge-and-purge') methods of education.
  7. The testing effect refers to the research finding that the long-term retention of information is significantly improved by testing learners on this information.
  8. Testing is not merely a means to measure a learner's level of knowledge, but rather causes knowledge to be stored more effectively in long-term memory.
  9. The spaced education methodology is content-neutral and thus can be utilized to learn most anything.
  10. Potential applications range from teaching chemistry concepts to high school students to reinforcing Arabic language skills among health workers in the Middle East.
  11. It can also be used to reinforce educational material which was initially presented in the classroom.
  12. The full multi-media capabilities of the Internet can be harnessed to create a rich and effective learning experience.
Share post
Design Museum

Learn - repeat - learn

Visible to anyone in the world
Edited by Jonathan Vernon, Monday, 15 Jul 2013, 17:13

'The psychological conclusion demands a distribution of repetitions such that some of them should be produced at a later time, separated from the first repetition by a pause'. (Vygotsky, 1926:Location 2686)

So wrote the educational psychologist Lev Vygostsky over 80 years ago. Putting this into practice using email (and now Smart Phone apps and eventually Facebook), the team at Spaced-Ed have created a learning system that works.


Interactive Spaced-Education to Teach the Physical Examination: A randomized Controlled Trial


Several studies have documented that physical examination knowledge and skills are limited among medical trainees.


The objective of the study is to investigate the efficacy and acceptability of a novel online educational methodology termed ‘interactive spaced education’ (ISE) as a method to teach the physical examination.


Randomized control trial.


170 second year medical students.


  • Spaced-education items (questions and explanations)
  • Validated by two experts
  • Piloted and 36 items selected for inclusion
  • 6 spaced-education e-mails each week for a 6 week cycle.
  • Students submitted answers to the questions online and received immediate feedback
  • An online end-of program survey was administered.

RESULTS: Successful 85% participants recommended the ISE programme for students the following year.

CONCLUSIONS: ISE can generate significant improvements in knowledge of the physical examination and is very well-accepted by students.

While many studies have documented the dearth of physical examination knowledge and skills among trainees, ISE has the potential to remediate these deficiencies across the spectrum of medical education (p977)

Why necessary?

Students do the training, but may still have poor recall a year later. Spacing works.

The spacing effect is the psychological finding that educational encounters that are spaced and repeated over time (spaced distribution) result in more efficient learning and improved learning retention, compared to massed distribution of the educational encounters (bolus education). (P973)

As Vygotsky expressed it 80 years previously:

'It should also be emphasized that every person has his own customary rate of response, and that any change in this rate, either speeding it up or slowing it down, weakens the force of recall'. (Vygotsky, 1926:Location 2686)

And so here students can tailor the timing of questions they are fed - spacing them out or bunching them as they see fit and circumstances change. Go try it, there are courses you can do on topics that a far less demanding that second year physical examinations. Try world history in maps, for example, or getting the most out of your iPhone.

A distinct neurophysiologic basis for the spacing effect has been identified

‘Spaced education’ refers to online educational programs that are structured to take advantage of the pedagogical benefits of the ‘spacing effect’.

Interactive spaced-education (ISE) combines the pedagogical merits of both the ‘spacing effect’ and the ‘testing effect. (974)

Each spaced-education item consists of an evaluative component (a multiple choice question based on a clinical scenario) and an educational component (the answer and explanation)

Psychometric analysis of the questions was performed using the Integrity test analysis software (http://integrity.castlerockresearch.)

Based on item difficulty, point-biserial correlation, and Kuder–Richardson 20 score, 36 of the questions were selected for inclusion in the ISE program.

  • Students receive spaced-education e-mails at designated time intervals which contain a clinical scenario and question (evaluative component).
  • Upon clicking on a hyperlink in this e-mail, a web-page opens which displays pertinent images and allows the student to submit an answer to the question.
  • Upon downloading this answer to a central server, students are then immediately presented with a webpage displaying the correct answer to the question and an explanation of the curricular learning point (the educational component).
  • By having the provider submit a response before receiving the correct answer and an explanation, this process requires greater interactivity, which educational theory argues may improve learning outcomes.
  • The submitted answers of students were recorded using the MyCourses™ web-based education platform

The survey was constructed and administered online using the SurveyMonkey web-based platform.

Future developments and assimilation of spaced-education alongside traditional medial school teaching methods

For example, as ISE utilizes traditional web-pages for the submission of answers and for the presentation of learning points, it should be possible to use all of the functionalities of web-pages within the ISE program to meet the training needs of care providers. For example, physician trainees learning how to auscultate the heart can be presented with ISE items which contain an audio recording of an unknown heart sound, and then, trainees can be asked to identify the murmur.


Micro-learning is favoured over more substantial time being given to this. I can imagine many applications.

This finding is in stark contrast to the strong resistance we encountered when conducting a recent trial of web-based teaching modules among 693 medical residents and students. In this trial focusing on systems based practice competency education, trainees were expected to spend 20 minutes per week over 9 weeks completing web-based teaching modules (interactive web-pages and online narrated slide presentations). (p977)

This high acceptability also likely reflects the ease of use of the spaced-education delivery system, the immediate relevance of the content, and the importance that students attribute to learning the physical examination. (p977)

Some items to cover if you are thinking of being a professional and thorough as the team at Harvard and Spaced-Ed:

  • · Conflict of Interest:
  • · Funding/Support:
  • · Financial Disclosures:
  • · Author Contributions:
  • · Conception and design:
  • · Acquisition of data:
  • · Analysis and interpretation of data:
  • · Drafting of the manuscript:
  • · Critical revision of the manuscript for important intellectual content:
  • · Statistical analysis:
  • · Obtaining funding:
  • · Ethical Approval to Perform the Study:
  • · Corresponding Author:

And a finaly word from Lev Vygotsky.

'Rhythm plays a decisive role in the learning process, unifying some of the material, conferring on it a sequential symmetry, and, finally, organizing the various elements into a unified whole'. (Vygotsky, 1926)


Kerfoot, B, P (2006) SPACED EDUCATION. Interactive Spaced-Education to Teach the Physical Examination: A randomized Controlled Trial.

Vygotsky, L (1926) Educational Psychology





Permalink Add your comment
Share post
Design Museum

Paste as ...

Visible to anyone in the world
Edited by Jonathan Vernon, Saturday, 3 Nov 2012, 18:03

Someone points out how to cut and paste text across formats with ease and you find it is used as often as the Caps or Enter key.

I take it for granted that I copy, 'paste as unformatted' text all the time.

This allows you to pick text from any source and immediately rid yourself of the HTML coding and formatting; this puts you in control. It is quicker too.

Personally, I find manipulating text in various ways like making up a recipe with ingredients that I want in a way that I understand.

Dull, but Arial font, 12 point, is how EVERYTHING ends up. If think my brain is ready for information expressed in this way and fast-tracks the synapses.

P.S. With a computer in each room of the house, usually on, I find I can tap away, and tap into stuff on a whim.

Currently picking my way through the development history of Spaced-Ed as an interactive spaced education platform for first year medical students at Harvard. I can't fault it. Indeed, I would propose what I am doing as a two week module in the yet to be developed module 'The E-learning Entrepreneur.'

Commercialisation demands professionalism and accountability. Generating income is better that stats, not only are people using something you've invented, but they are willing to pay for it. The cash allows you to develop it further, and in due course fight off or buy up the competition.

Why in the UK are we determined to do it all for free? Is this an excuse?

Permalink Add your comment
Share post

This blog might contain posts that are only visible to logged-in users, or where only logged-in users can comment. If you have an account on the system, please log in for full access.

Total visits to this blog: 5190562