OU blog

Personal Blogs

Design Museum

Take Your Teaching Online Week 8 : The Power of Analytics

Visible to anyone in the world
Edited by Jonathan Vernon, Sunday, 6 Dec 2020, 11:29

Professor Bart Rienties of the Institute of Educational Technology at The Open University

January 2018 > https://youtu.be/GIWrygqmOIs 

Creative Commons Attribution licence (reuse allowed)

45 minutes? This is how long 'Take Your Teaching Online' gives to undertake the activity. It is far, far too little time to do justice to it. 

That's 35 minutes viewing to begin with add taking notes and 10 minutes to construct a reply. I gave it two hours.

This is worth this amount of effort and more; I will be going back to it. 

There are two reasons why is essential viewing for anyone venturing into teachign online:

1) This is an excellent lesson in how to deliver a lecture

  • The pace, variety, personal story telling, top and tailing (literally) with dogs and then heavy duty data expressed in tables and charts. All the while having the audience to feed in with a poll. 

2) The conclusions that Prof. Bart Rienties draws are profound 

  • Just in the period since the OU changed its fee structure (much more expensive) what students look for has shifted increasingly towards the relevance of the materials and qualifications to their job

The data blows away past perceptions and methods while reinforcing what had been an indicator of excellence vs failure all along. 

  • Student satisfaction surveys bear no relation at all to peformance. 
  • Giving students nothing to do can result in the most activity - it becomes at chance for those 60% of more who are a little behind, or a long way behind, to catch up. 
  • Those who are always leaving it to the last minute and think that they can catch up in a last minute splurge of activity are likely to be those who just pass or fail. 

The best approach all along, and an indicator of excellence, is to get ahead of the curve. 

Not least it gives you breathing space to go back to something when you've had time to think about it, or to hunt down and check through alternative insights. As well as engaging on the subject if you wish outside the class. 

Wzq9IBOoFAgYpmx2qi_LtaqcCLLE1_oe1kOZ0EZYAYjIZICk61M_G3PKYUKNdd4JL4fVLqosVnUHsLGB7yzaQn0TJ0UJox4v4j1FRcW8IROGscZIk9p5xSblUVRFAEtOi5iDYIZwYXpQTcBzHtH7920bOAlcmZ_FdKkeTmUVPA1_8UqVaT8nTbf1mzeS4aNvaZav4Q0F-XR1N6PxPE5qqjD4ugFoKnxMaJW3bzBKZZ2ypO4afSIYvF6nNbD5uHfk_L_s78MLYCzLLUfw

I ask myself, 'these are adults, they are academics, they are Profs and PHDs, and yet some use their phones to give quiz answers answer before they've been given the question. What does this say about us humans? Is it the gambler's gambit?' 

Prof. Bart Rienties was using PollEV.com 

cqAj8HYN0TMQIh5eHb-Mt42gVMyGAN2ZmypoRDwpblPqPbriN_BONTDka7HqfSYqrhibZoriZyvH-57u66JP08P5nr8BTqOrnwZzxK-npEidM24ep0vpTwmV3NkMgU0rqXrxgtHf

Bart gets our immediate attention with a puppy. We are introduced to Tabatha from Canine Partners > https://caninepartners.org.uk/ 

By way of engaging metaphors data is first provided on three assistant dogs 

and then from the speakers competitive cycling.

These metaphors are used to indicate different kinds of data, the kind that is useful, and the kind that is not. And the need to be measuring something in the first place 

6fWWj297c5-S7F5YXToeZg0ZEafLmJwKfu4TOJ8nYiSk686Wvoia2HWXpbbaWzThwp7enF4GyyeXFe8v0jeNxpnKHriS1NdCHfZId3XIbl770Dt1js1V3uRLdr0qdYjrKgqPXjrv

The truth will out with the data

QQ: Can we use this data to give students what they want?

Ask them at the end of every module. 

jgudFtUEjIWvLzcCnaKK6jJm1ZSqHmyc0z40SH8rzTpVHKO2Jzu5mTyzl3L-da-eo_-6J2RJf2tzwijjiOUHpSLZeWXL9HPfUFmPj93mn7QsHvdjhfi4NyKh6o4bbVP35CQ6zB3H

D_y_HpWAE5GteapfI6OYyWPu04tVi-ME9lvixhcjIQB37fLf1VW3OkwkobeuF6fy36LK5tXvPZA7KjBXxaJNGJoYQGXzo2C34U-g7pWffibFd0nWf6wa9BDcgwZfdzKUdHYY57Sz

Big data set

  • 110,000 students
  • 400 courses

QQ: What makes the course 'good' ? (As in getting results) is it: 

  • Great reachers? 

  • Links well to professional practice?

  • Links well to their career intentions?

  • Quality of the teaching materials?

  • Quality of the teaching?

This is what 40 people from the audience concluded. (40 was the limit of the licence bought from PollEV). 

8IwYypODoniZZVOSWsmeVGB9sYIfP1Po4DQKRwVzCMPz7IE-Smo0RT9nTLXl6z_tgp2o0INl5u6YfJIFx8-HLiw1g2NGKF4VrxeF83CKyH3N_24uM7i7u3U_JA7IPEtqcr98X29u

Here is the detail: 

rBqP9aIMemihD1yBuXYfHKUVc4GmnkqVwPNV8kEOEo2zUC6hA_-utFoOdsSvomZdgvvkP8nGyN6plt3pgmQeZuoyLMx1AaMr05ghG0yIbwQXeiSm_Nhyp4GRVOFDLKGTRoJVX5mq

What makes an excellent course? 

  • Really good teaching materials

  • Student approval of the assessment method. 

There has been a subtle change over the last few years

  • Perspectives changing to students expecting modules link to the qualification.

NPAr3z0biJh67-kMQLp8a1Y6NoD63seVp-Ocn9pRiseK9ursNpB6N9sAEZVjbPifZ6kU2TKR7eyeL-LQwVp1_zTZHvzd3EGrknusEKx_VgtMN7PxipAVWtFZl_xqupdtBnqBwzfH

NOTE > 

There was NO correlation between student satisfaction and student performance

Students like constructivist learning designs

This is where there is lots of stuff, we take them by the hand … providing lots of content for them to explore.

They did not like when they had to work together with a group, or talk to the teacher. 

The number predictors of passing and continuing is how teachers design the learning and how they communicate during the course.

tqIDRYWU57rwRSqPVqXgx6UXefbDq7UCYWjfQZ4gPWiQU1ITfx94yszYHxSA4K9vcJlc5bfXnjdV-1Vu6OeVzw2dKs-hUq_G1i0AYtyhMit30sc1QMLzJFwu1cp4fDupZY9XPujK

Starting to unpack the recipe what helps our students progress so using the social constructivist model.

  • Should we give students what they want? 

  • Students are different! 

  • Most will benefit from knowing what is coming up.

What are students doing on a week by week basis.

GFhNTJdf42yKHWdqjerzF_xBV_omDB5VXNnC13ExGaaiKm4pd8HO_od-kK_T8ydu_o1f2wROmih7DfGG6qD7MWCwJtLIdeT2x1gW3D6ErDpAOwS6Xkb895AagfXDk0oX99H5bw1N

Is there a link between how teachers design and how students engage?

Unpack what is really happening?

Why despite expectations of a lot of student engagement in week 20 was there not?

CZN-LCfJsQUaOvGZXB4gOWezwzGuek8joiOyQsoO3VIaintCmywSutNRCrkzAEu5CxSETp3iKb5xfOkp1M_sZg6YHCzI_NxwQGz9u-sCGaFpsmyEzFxkvjHCxGWLhxeDtJyeVdcM

Start to map out what students are doing.

G2bYpulITBc1JhJ5Ta6yfaPWYPM2hJZ5aKPNye63QphqfmBdOO2DmqY2j-5URJZYUKKkmI8ymKHVagoK4k1JrXGfkm1l1lUORbWMoKFl5TQi13r9s3l3EjiD0O7B0ov2AHYAc6QY

Week 4 time off to prepare … and becomes one of the highest peaks.

69% of what students do is determined by what teachers have designed for them to do.

9TLto5dFHOGexj2jXqfmUshdN8qc1zA4EGMlZNt5uTXD-obY-H4vSJ-8D12e2qQN5-DkBm2jk7Qagb8ED2qso_h3z3NK8ZTGnbqB8YcPwiyiDYYbCGuoqcsWtbwIunjDPGuzfCzu

Excellent students study more in advance.

CCj5mTeNCi1pCHWTU_nDPzDS72QCEeZfwgiWbdhKUU_UskHld6QbaoqXGVkuCuhvE811HcY3czo6paPILdtyMiGZZGwalRHwjss8DkxWSQBtmT2oXNf5xREdGMjeNSJgRIYcGVs5

Pass students start to get in the ‘catch up’ phase > they are going ‘off piste

B63Ee2YzWGZUn1geYxZBC8I99XdLPepYPVT79YqTE_n0Z4ckCipetCCB_U1m5U7p5iwyzmx8bJtMpmI2njHiNT1W6LAnvOI33MHSMUKdmKYI1Ys1X_KPZ5GdmSH8NJpCI9Wi5gPL

Fail group > starting to catch up, or never catch up. So how do we give them a chance? A pause to catch up. 

What are our take home messages?

Not all data we collect is meaningful.

What matters is actual behaviour. Big data without context is meaningless.

NOTE > Listening to student feedback is not linked to what they are doing or how they are performing. 

Our students are following the learning design, but many are not. Some diligently stick to the road, others take different routes. 

NOTE > We need to provide alternative effective pathways.

End with Canine partners and foster parents.

Organ donation 

Permalink Add your comment
Share post

This blog might contain posts that are only visible to logged-in users, or where only logged-in users can comment. If you have an account on the system, please log in for full access.

Total visits to this blog: 6218594