OU blog

Personal Blogs

Christopher Douce

Academic conduct review briefing

Visible to anyone in the world
Edited by Christopher Douce, Sunday, 13 Oct 2024, 12:21

On 29 February 2024 I attended an academic conduct review briefing for module teams. The aim of this blog is to share some highlights that may be useful for colleagues (and to also try to remember what was covered, since there’s always a lot going on).

During the introduction, it was highlighted that academic referrals have increased over the last 5 years. Two thirds of these typically relate to study skills and poor academic practice. In other words, this relates to what could be called inadvertent misconduct.

There are some specific challenges. Investigations take time, there is an obvious need for consistency with decision making, a need for transparency, and continually emerging threats, such as essay mills and the arrival of generative AI. To respond to this changing landscape, there is a new VLE page for staff, a new academic conduct tariff, new guidance on StudentHome, a new referral form, and new processes and study skills arrangements.

Academic integrity principles for assessment design

Assessment design is important. Good assessment design should ‘reduce the motivation and opportunities for plagarism’ and focus should be on how learning takes place, as well as what is learnt.

An important point that I noted was that module teams should look at patterns of academic conduct referrals. One way to respond to repeated issues is, of course, to consider the assessment design. To help with this, there some useful principles of assessment which I share below.

A key theme of these principles is authenticity. This is connected with need for to assessments that focus on problem solving, perhaps solving problem that might have an open ended answer.

Here are the ten principles that were shared during the session.

Principle 1: Reward academic integrity

Make sure students reflect on their skills and capabilities that demonstrate academic integrity.

Principle 2: Assess higher skills

Assess metacognitive skills where students actively create an answer, with an emphasis on construction and their own thinking. This point is, of course, linked to the principle of refletion.

Principle 3: Open-ended solutions

This principle is related to authenticity. It’s possible to see authenticity within the undergraduate computing project module.

Principle 4: Diversify assessment formats

This is linked to how students might relate assessment to their own interests and preferences. Providing a variety of different assessment formats also speaks to the importance of accessibility and diversity.

Principle 5: Authentic assessment

Making an assessment real by sharing an exciting concept can enhance motivation. Make an assessment authentic by drawing on their own experience.

Principle 6: New assignment questions

To limit the risk of academic conduct, prepare new assignment questions every year. Design the assignments in such a way that students cannot re-use a previous answer. In other words, make it difficult for students to directly benefit if the questions were to be shared.

Principle 7: Process besides product

Ask students to submit interim versions of assignments or ask them to provide a demonstration. In the computing project module, students are asked to submit a draft EMA as an assessment.

Principle 8: Making quality criteria transparent

Make rubrics transparent; share with students how assessments are made so everyone knows what the module team (and the university) is looking for. 

Principle 9: Generating and acting on feedback

Encourage students to generate their own feedback, which helps to develop critical thinking. This relates to a principle of peer assessment. This is used, to a limited extent in a software engineering module, where students comment on interim designs. 

Principle 10: Authentication methods

Are students who they say they are? There used to be a time when students had carry out written exams by attending exam centres at a specific date and time. As a part of this, students needed to show a form of ID, such as a passport or a driving licence. An alternative approach would be for a student to have a conversation with a tutor about their assignment.

TurnitIn

A recent development has been the increased availability of TurnItIn to students. Module teams and staff tutors always used to receive TurnItIn reports. These reports indicated whether a submission contained words found in other online resources. A typical TurnItIn report might highlight legitimate quotes that a student used to evidence an argument, as well as uncited text.

On many modules students can now submit an earlier version of an assignment into TurnItIn. In turn, they can review a ‘similarity report’ to gain confidence that their assignment is adhering to sound principles of good academic practice. More information about TurnItIn can be typically found within a module’s assessment handbook.

Resources

A whole host of useful resources are available to students. The blog post Study Skills Resources: what is available? offers a handy summary of some of the most useful.

Reflections

I’m not sure where these assessment principles come from. I really like them. I can bring these ten points into module team meetings. Their focus on authenticity strikes me as being really important. It is important in terms of problem solving, skill development and the application of knowledge.

Putting the assessment design principles to one side for a moment, tutors are really important. If appropriate, students can refer students for additional support sessions, and can also refer students to study advisors who work within the student support team.

Students can then be referred to additional support needs, and referral to a study advisor. In my experience tutors may sometimes get a sense when a student may have used Generative AI.

Acknowledgements

Acknowledgements are duly given to the academic conduct review team who facilitated this session. The principles shared in this blog post are their own, and I take no credit for them. I hope I’ve summarised them effectively.

Permalink Add your comment
Share post
Christopher Douce

ChatGPT school seminar

Visible to anyone in the world
Edited by Christopher Douce, Sunday, 21 May 2023, 09:49

On 19 April 2023, I arrived slightly late for an online seminar about ChatGPT and generative AI. This blog post share some of the notes that I made during the session. It might be useful to read this post in conjunction with an earlier blog that was written on the same topic that summarises a workshop organised by the OU Knowledge Media Institute (KMI). These notes are pretty rough-and-ready, since they were edited together a month after the event took place.

Seeking opinions

Mike Richards, from the School of Computing and Communications, began by summarising some research that he had carried out with a number of colleagues. Five tutors were interviewed. When it comes to reviewing and marking assignments, it was noted that tutors are sensitive to changes in formatting style, voice and vocabulary.

Tutors rely on module teams and central systems for plagiarism detection, but they can and do pick up on things themselves. ALs don’t like referring students to disciplinary processes. They are cautious; they usually have a very high level of suspicion before they contact staff tutors and invoke the academic conduct processes. In the cases where the identify issues, they take opportunities to make a teaching point to students.

Tutors wish to maintain positive relationships with students, but they are worried about the implications of raising academic conduct referrals and potential professional consequences if they raised unwarranted academic conduct concerns. Of course, there are no consequences for tutors. It is, of course, the academic conduct officers who make the decisions.

Key points

During the session, I captured the following important points. The first point was that assessment is vulnerable to ChatGPT. Specifically, highly structured essays are vulnerable, but these type of essays are used to develop student skills.

ChatGPT perform less well with anything to do with reflections about learning, since anything that is produced will not sound genuine.

There is a role for ChatGPT (or generative AI) detection software, but there are issues with detection tools, since they present a high rate of false positives. Detectors only gives you a probability that something is synthetic, but doesn’t provide evidence like TurnItIn.

Tutors are very important. They are able to spot synthetic solutions; they can identify bland, superficial, repetitive and irrelevant materials in a way that automated tools cannot. To assist with this, and to help our tutors, the university needs to provide better plagiarism training.

A recognised issue is that ChatGPT will generate superficially compelling references that are completely fake. Asking ALs to scrutinise the referencing would go some way to determine whether a chunk of text has been automatically generated. ChatGPT doesn’t currently do referencing at the moment, but there is a possibility this might change if it is connected with public databases.

The next step of this project is to write up findings and to have conversations with other faculties. There is also a university working group which aims to generate an assessment authoring guide to mitigate against generative AI. There is, of course, the need to do more studies. There might also be the need to adopt subject or discipline specific approaches. 

The closing thoughts shared during the seminar are important: we need to teach all students about the consequences of AI. Perhaps there needs to be some Open Educational Resources on the topic, perhaps something on OpenLearn that offers a sketch of what it can and cannot do. A closing point was that there are no ‘no-cost’ options. The university needs to carefully consider the role and purpose of assessments. Doing nothing is not an option.

During the discussion session, I noted down a couple of interesting questions: what question types would cause large language modules to perform sufficiently bad from caring to not caring? Also, what limits its abilities? ChatGPT writes in generalities. Its responses comes from how questions are worded. There is also the issue of concreteness. Assessment tasks are often related to specifics, in terms of activities texts, module materials, and forum posts. If generative AI cannot access the texts that students need to access and critically evaluate to develop their skills, its uses are, of course, limited.

Reflections

One of the key points that was emphasised was the importance of the tutor. They have such an important role to play in not only identifying instances of potential academic misconduct, but also in educating students about generative AI, and the risks these tools present.

It is also useful to reflect on the point that tutors can spot changes in writing style. There is the possibility that the stylistic quality of generated text is a characteristic that could be used to respond to not only ChatGPT, but also contract cheating. At the time of writing, anti-plagiarism detection tools such as TurnItIn only evaluate individual assignments. In the arms race to ensure academic integrity, the next generation of tools might analyse text across a number of submissions whilst taking into account the characteristics or structure of individual assessments.

I expect there will be a multi-faceted institutional response to generative AI. There will be education: of students, tutors, and module teams. Students will be informed about the ethical risks of using generative AI, and the practical consequences of academic misconduct. Tutors will be provided with more information about what generative AI is, and offered more development to facilitate sessions to help students. Module teams will have an increasing responsibility to develop assessment approaches that proactively mitigate against the development of generative AI. Also, technology will play a role in detecting academic misconduct, and new procedures will be developed to assist academic conduct officers.

Acknowledgements

An acknowledgement is due to Mike Richards and everyone who took part in aspects of research which is summarised here. A thank you goes to Daniel Gooch, who facilitated the event.

Permalink Add your comment
Share post
Christopher Douce

TU100 My digital life: AL development event

Visible to anyone in the world
Edited by Christopher Douce, Tuesday, 8 Oct 2013, 12:15

The second TU100 development day for associate lecturers in London and the surrounding regions was held on Saturday 7 September in the London regional centre.  The overall purpose of the day was to give associate lecturers who tutor on TU100 an opportunity to share experiences and to gather some useful feedback about the module that I could pass onto the module team.  These days are often great fun since everyone is very much up for sharing and talking (and this day was no exception).  This blog post represents a quick summary of what happened (from my own perspective, of course).

I’m writing this post for a number of reasons.  The first reason is to remember what happened on 7 September (since my memory is somewhat fallible), and the second reason is to give those tutors who couldn’t attend a bit of a feel for some of the subjects were discussed.  The third reason is to try to encourage other tutors to come along to other events that we run in the region.

There were essentially three different parts to the day.  The first part was all about teaching programming and Sense.  The second was about issues relating to student retention (where we heard about a university initiative called Project Retain), and the third was a general ‘feedback (or feedforward) to the module team’ session.

Session 1 : Teaching programming and Sense

During the first session we were put into small groups and Leslie, one of our very experienced TU100 tutors, distributed a questionnaire to inspire discussion.  These had the headings: ‘how does TU100 teach programming?’, ‘how does TU100 teach Sense?’, ‘student contact hours’ and ‘marking’.  Since I’m not a TU100 tutor I didn’t contribute too much to the group discussions, but I did make some notes of some of the themes that had emerged.

It wasn’t too long before the subject of programming cropped up.  One of the comments I’ve made is that the module doesn’t contain too much about testing.  One other thought is that early on in the module it is a good idea to emphasise the importance of Sense, particularly the Sense programming guide.  Another thing that tutors could do is to emphasise the wealth of Scratch resources that are available from MIT, and that perhaps we should more explicitly brief students that Sense is an extension of Scratch.

We soon began to talk about the on-line sessions which are presented through Blackboard Collaborate (or OU Live, as the university calls it).  One of the challenges with using the OU Live software is that it takes time to hand over screen sharing control when tutors ask students to complete certain tasks. 

An interesting point is that OU Live might not only be useful for running tutorials.  Since it contains a facility to record sessions it can also be used to record how any application is used.  Tutors (or faculty staff) could use OU Live to make ‘video’ recordings to demonstrate some programming concepts.

One of the biggest challenges that tutor’s face is the marking of assignments.  Sometimes tutors come across some puzzling situations, i.e. if students submit work where a screenshot represents a correct functioning program, but the program that is submitted isn’t actually correct.  When it comes to correspondence tuition, one of the fundamental challenges is to get into the head of the student.  This led to the question of whether we might be able to record video clips to show how students could have created correct solutions.

Plenary

After around fifteen or twenty minutes of chatting, all groups were asked to report back.  This section is a quick summary of some of the key points that some of the groups mentioned. 

TU100 doesn't contain a section that is dedicated only to programming.  Instead, programming can found in different sections throughout the module.  One point mentioned by tutors was that whilst TU100 teaches coding it doesn’t say much about how to do the 'problem solving' part of programming.  Instead, students are required to spend time discovering how to program by exploring and playing with the Sense environment.

Aware of this issue, some TU100 London tutors have started to present the fundamentals of how to break apart problems into pieces that could then be used to create code (either in the face to face sessions, or on the on-line sessions).  The precursor to TU100, M150 contained some materials to introduce students to something called structured English.  This gave way to a debate about whether some additional material might be added to TU100, but the problem is that there are already lots of materials that students and tutors need to cover. 

The point is that the foundations (in terms of learning to program) are really important, especially for students who might potentially struggle with the fundamentals of programming.  One tutor said that some students never make it to the starting line on Sense and this kind of resources could be a bridge between high level thinking and programming.  Some of the fundamentals that could be covered (by tutors) include the basic constructs of programming, which includes sequences of instructions, selection, iteration, the use of variables and debugging.

One tutor said that ‘we need to emphasise that it is important that students have a go’ (so students gain an understanding of what the building blocks of software is all about).  Also, there is need for a Sense forum, something or some area that allows sharing of materials and ideas between students and tutors. 

One piece of advice to students should be, ‘go look at what people do with Scratch’.  Another comment was, ‘add a couple of YouTube type videos about program analysis’.  The interactive nature of programming does lend itself to the use of OU Live, via application sharing, but on-line asynchronous tutorials are always going to be difficult and it takes a skilled facilitator to use more sophisticated functions such as on-line break out rooms.

Another perspective was that it might help the students if there was slightly more signposting to different resources.  (I understand that this is something that the module team have been working on for the new presentation).

Contact hours, tutorials and day schools

Different regions do different things when it comes to on-line tutorials and day schools.  When it comes to on-line time, the London region has given tutors the opportunity to schedule and run individual sessions.  The south region runs join sessions, as does the south east region.

When it comes to the face to face sessions, all the London groups come together to form a series of big day schools with the intention of creating a critical mass of both students and tutors.  In other regions tutors run sessions with pair of tutors.  The differences can be down to geography, both in terms of the location of the students and the location of the tutors.  One other thought from my side is that it is also important to emphasise to all students that they are encouraged to go to any of the tutorials that they might find in the tutorial finder (so they can discover evening as well as weekend events).

Some tutors use materials that are created by the module team, whereas others create their own materials.  One example is the London region tutors creating materials in structured English, with a view to trying to ‘plug a gap’ in the module materials (regarding how students new to programming might set about splitting a program into different components).

Another approach that some tutors adopt to use their allocated on-line time is to run on-line drop in sessions via OU live.  The idea for this is that students could just pop into an on-line room to have a chat with a tutor if they had any questions.  I personally find this a really compelling way of making use of the on-line rooms, particularly when students might be wishing to chat about programming.  The breaks with the formality of a one-to-one conversation of the technology, but also allows participants to see what is being displayed on a shared whiteboard.

Working with OU Live

The first tip (for tutors) was, ‘remember to switch on your microphone’.  Another thought was, ‘can we make headsets compulsory please?’  The reason for this is simple: when students use the microphone and headset that is built into a laptop, a whole group of participants can be easily distracted by feedback, making communications a whole lot more difficult.

In some respects, participating in an OU Live session can be quite intimidating and one observation was that there are lots of students who don’t want to speak at all.  Sometimes some students prefer to use the text chat window rather than using the microphone, which can then make if quite difficult for the tutor to keep on top of everything (which is why some regions share OU Live sessions between tutors).

One point was that it is useful to ‘do something’ every 20 or so seconds.  This might be asking students questions, requiring them to respond with yes/no answers.  Another thought is to use a series of polls to assess understanding of certain concepts.  (One thing that I have personally learnt from my experience with the South East of England training is to poll students using the, ‘happy face’ button, i.e. by asking the students, ‘is everyone happy?, can you click on your happy face?’  When you regularly ask this, it helps to keep the student’s attention).

Marking of code

This section of the plenary discussion echoed an earlier point, that when it comes to communicating what needed to be done with complicated TMA questions (which involve programming), could the module team produce a video about how things should run, or have been constructed (using Sense)?

I’ve learnt that there are two different ways to add comments into Sense code.  One way is to use something called a comment window.  Another is to add some in-line comments.  I made a note of a debate about the use of different types of comments and that in previous assignments a TMA question asked students to add comments.  The consensus was that comments help; they help students to reflect on the code that is being written and help tutors to understand what has been submitted.

Project retain

An interlude between the first and the second session was presented by Maggie King, our associate dean for teaching and learning.  One of Maggie’s responsibilities has been to be a part of a university wide project called ‘project retain’.  

Project Retain is intended to increase the university’s retention (and progression) of students across different levels of study.  The project has given the university a number of recommendations, which include: offer a guide to key learning points and module materials, schedule and communicate real-time contact sessions during the first two weeks of a module (ideally through a letter), open module materials and web sites before the module starts, and make it clear when assignments are coming up (so our students are not surprised when they have to submit their assignments).  The first year of study, it was argued, is absolutely crucial.

Session 2 : Retention

Terry, one of our experienced TU100 tutors facilitated the second main session of the day, which was also about retention (which is an issue that affects student satisfaction scores, recruitment and funding). 

Terry introduced us to HEFCE performance indicators.  These include dimensions such as the national student survey and other aspects such as the measurement of research performance.   Terry also introduced the difference between retention and progression.  Progression is all about moving from one level of study to another.  In some circumstances students can defer, allow them to take a bit of time out from study and enabling them to pick up a module again at a later date.

One of the biggest changes in the university in the forthcoming couple of years will be the introduction of something called student support teams.  Since more and more students will be registering with the intention of studying for a particular qualification, student support teams will play an important role in helping students with their choices along a student pathway – it is hoped will positively impact on student retention.

Terry covered a wealth of materials, including sharing with us points from a national audit office report, drop out rates, how retention in UK HEIs compare with the retention in other countries, and how the university compares with others in the national student survey.  During his session Terry asked us to consider the causes of student drop out during different stages of study, such as pre-entry, induction, on-programme and movement to the next level.  In the university both tutors, student advisors and module teams all have an important role to play.  The final question of the day was, ‘how can the university support you in the task of improving retention in your tutor groups?’  This was an exceptionally very good question to ask and is something that I’m keen to pick up on and delve into when I have a bit more time.

Session 3 : Open Session

I have to confess that I haven’t taken too many notes about this final session mainly because we ran out of time!  Everyone was very willing to share experiences and opinions throughout the day, which was one of its fundamental objectives.

Reflections

One tutor made the comment: ‘you can make a full time job of teaching TU100’.   TU100 is, without a doubt, a very big module: there is a lot of material and there are a lot of demands on the tutor’s time.  What struck me about this day was the willingness of tutors to do their utmost to help their students along their TU100 journey and their willingness to share experiences with each other.  The event had lots of energy and there was a lot of positive talking going on, yielding some very good ideas.  From my own perspective, I certainly hope to be running a similar event next year.  I’ve already had a couple of thoughts about what we might do.

I have learnt quite a few things from this session.  I’ve learnt about the opinions that tutors have about certain aspects of the module and I’ll be happy to forward these directly to the module team.  It is also clearly apparent that some students struggle with programming and the idea of producing some video material to help to explain certain concepts might be something could be useful.

Acknowledgements

Many thanks to all our TU100 associate lecturers who kindly gave up their valuable time to attend this event on a Saturday. If any of the tutors who have attended would like to add further comments, please don’t hesitate to comment below.

Permalink 1 comment (latest comment by Jonathan Vernon, Tuesday, 2 July 2019, 11:32)
Share post
Christopher Douce

Embedding Self and Peer Assessment and Feedback in Practice - A Principles-Based and Technology-Enabled Approach

Visible to anyone in the world
Edited by Christopher Douce, Friday, 18 May 2018, 09:06

The first HEA discipline workshop of the year, entitled Embedding Self and Peer Assessment and Feedback in Practice (HEA events page) was held at the University of Ulster on 9 March 2012.  This blog post represents my own reflections of the event.  I do hope they are useful to either the delegates who were there, or anyone else who accidentally stumbles across these notes.

Introduction

Denise McAlister, Pro-Vice Chancellor at the University of Ulster introduced the day by emphasising the importance of feedback and how it plays a central role in teaching and learning.  The student experience, it was argued, is significant affected by the provision and quality of feedback.  Denise raised a number of challenges: how pedagogic design can offer opportunities for feedback.  The opportunities have to be balanced with the potential risk of over-assessment.  She also raised concerns about group assessment, particularly when it comes to transparency.  It is therefore a necessity, of course, to provide robust processes to ensure that students are treated fairly and equally irrespective of the assessment method (or methods) that an institution may adopt.

Presentations

The first main presentation of the day was by Alan Masson, Head of Technology Facilitated Learning at the University of Ulster.  Alan's presentation was entitled 'Ulster Principles of Assessment and Feedback in learning, background and their use in promoting learning technologies'.  Alan made the point that assessment and feedback is a key part of the student experience; it is an issue that affects student retention and achievement, module and course evaluation and the wider public perceptions of an institution.  Alan mentioned something called the REAP principles (REAP website, PDF). 

Alan went onto summarise a set of seven principles that Ulster use.  These were (in essence): clarify good performance, encourage time and effort, deliver high quality feedback, provide opportunities to act on feedback, positive encouragement, develop self-assessment and reflection, and encourage interaction and dialog.  More information about these principles can be seen by visiting the University of Ulster Assessment and Feedback website.  During his presentation, Alan also directed us to something called the SCONUL Seven Pillars of Information Literacy, which was something I had not heard of before.

The second presentation of the day was by Rebecca Strachan, Northumbria University.  Rebecca's presentation was entitled 'Peer Assessment for Formative Feedback - it's Good for You'.  Rebecca introduced us to something called the Learning Pyramid, from the National Training Laboratories, which emphasises the notion that highest levels of retention (or perhaps understanding?) may arise when one student is asked to teach something to someone else.  We were also directed to the Ripples model of learning (Phil Race's website) which presents 'seven factors underpinning successful learning'.

Rebecca presented a number of challenges regarding the subject of formative peer assessment.  It is necessary to provide a motivation for the students, define the rules of engagement, and implement a system where by the tutor or instructor could offer (or provide) a moderation process.  Finally, it was considered that the process of providing feedback is as useful as the feedback itself.

Two presentations about WebPA followed.  WebPA is a web-based peer assessment tool that was developed at Loughborough University and was funded by JISC, building on earlier work.  Keith Pond presented an outline of the WebPA system through his presentation 'WebPA - Multi-disciplinary peer-mark moderation of group work'.  This was complemented by presentation about teaching practice by Neil Gordon, entitled 'Experiences and practice of using WebPA to support self and peer-assessment in teaching'. 

What I liked about Keith's presentation was the summary of the history of the project and particularly the emphasis on feedforward comments (i.e. comments to help future performance, as opposed to reflections based on existing work that has been completed).   What I liked about Neil's presentation was that he emphasised the importance of group work for software projects, speaking to an important industrial perspective that can be easily lost.    More information about WebPA can, of course, be found by visiting the WebPA project website which is hosted at Loughborough.

The final subject specific presentation of the day was by Luke Chen who compared student perceptions of self and peer assessment by using two case studies: a masters module (advanced web technologies) and a undergraduate module (advanced interactive programming).  One approach used Blackboard whilst the other used a paper based system.

Student voices

A couple of weeks ago I attended an inclusive learning conference at the Open University.  One themes that struck me from the conference was the importance of the student voice and the role that it can have in helping to inform educational practice.  It was great to see that student voices (or student views) were also brought to the fore at this workshop.

Following on from Luke's earlier presentation, Steven McComb, a postgraduate student, gave his reflections on peer assessment.  This was followed by a presentation by Tanya Fisher, a final year interactive multimedia design student, who also volunteered useful reflections.  It was great to hear such positive assessments of the assessment process.

Activities

After lunch, we had the opportunity to participate in a curriculum design activity that was based on the JISC funded Viewpoints project (University of Ulster).  We were asked to tools from the Viewpoints project to consider: standards, best practices, assessment and feedback principles, implementation ideas and the role of technologies.

The tools we were given comprised of a large laminated sheet which represented an empty module, and cards that we could stick to various parts of the module.  I interpreted the tool to enable curriculum designers to consider different alternatives or to help to facilitate communication between different designers.

Although we didn't get much time to play with the tools, the use of cards which encapsulate aspects of curriculum design as a way to facilitate design of modules is one that is compelling. 

HEA presentation

The closing presentation of the day was by Mark Ratcliffe, Discipline Lead for Computing at the Higher Education Academy.  Mark spoke briefly about the various workshops that were scheduled throughout the year and also told us about the different funding opportunities that were open to individuals as well as institutions.  Opportunities range from small grants to enable delegates to travel to workshops through to large multi-institution collaborative projects.  More information can, of course, be found by having a dig around in the HEA website, or by contacting the HEA directly.

Reflections

Feedback is a really important subject.  My own personal view is that it is something that takes a lot of skill and experience to do really well.  One thing that we can every easily overlook is the feelings that our students experience as they receive their assignments or exam results.  We need to be mindful of presenting encouragement to ensure that learners are in the right frame of mind to accept any future altering feedback that we may take time to compose and present.  Peer assessment is one way to enable learners to develop critical thinking skills whilst at the same time enabling students to engage with the concepts and theories that may be the subject of an assessment.

Pedagogy, however, remains very important.  Peer assessment, in my opinion, is likely to be an especially powerful tool if it is used in subjects where having the skills to constructively criticise is especially valued.  During the workshop, the use of group work within software teams was something that was mentioned.  Peer assessment is, of course, an invaluable tool in creative subjects, such as design.

All in all, a useful and successful workshop.  Thanks Ulster!

Before I go, I have to confess that this was my first trip to Belfast and I really enjoyed it.  One of my regrets is not having taken the opportunity to take more to explore the city and the surrounding area a bit more.  Heading home to London I made a personal 'note to self' that I am most definitely going to return and do exactly that.

Permalink Add your comment
Share post
Christopher Douce

Formative e-assessment dissemination day

Visible to anyone in the world
Edited by Christopher Douce, Monday, 19 Nov 2018, 10:40

A couple of weeks ago I was lucky enough to be able to attend a 'formative e-assessment' event that was hosted by the London Knowledge Lab.  The purpose of the event was to disseminate the results of a JISC project that had the same title.

If you're interested, the final project report, Scoping a Vision for Formative e-Assessment is available for download.  The slides for this event are also available, where you can also find Elluminate recordings of the presentations.

This blog post is a collection of randomly assorted comments and reflections based upon the presentations that were made throughout the day.  They are scattered in no particular order.  I offer them with the hope that they might be useful to someone!

Themes

The keynote presentation had the subtitle, 'case stories, design patterns and future scenarios'.  These words resonated strongly with me.  Being a software developer, the notion of a design pattern (wikipedia) is one that was immediately familiar.  When you open the Gang of Four text book (wikipedia) (the book that defines them), you are immediately introduced to the 'architectural roots' of the idea, which were clearly echoed in the first presention.

The idea of a pattern, especially within software engineering, is one that is powerful since it provides software developers with an immediate vocabulary that allows effective sharing of complex ideas using seemingly simple sounding abstractions.  Since assessment is something that can be described (in some sense) as a process, it was easy to understand the objective of the project and see how the principle of a pattern could be used to share ideas and facilitate communication about practice.

Other terms that jumped out at me were 'case stories' and 'scenarios'.  Without too much imagination it is possible to link these words to the world of human-computer interaction.  In HCI, the path through systems can be described in terms of use cases, and the settings in which products or systems could be used can be explored through the deployment of descriptive scenarios and the sketching of storyboards.

Conversational framework

A highlight of the day, for me, was a description of Laurillard's conversational framework.  I have heard about it before but have, so far, not had much of an opportunity to study it in great detail.  Attending a presentation about it and learning about how it can be applied makes a conceptual framework become alive.  If you have the time, I encourage you to view the presentation that accompanies this event.

I'm not yet familiar enough with the model to summarise it eloquently, but I should state that it allows you to consider the role of teachers and learners, the environment in which the teacher carries out the teaching, and the space where a learner can carry out their own work.  The model also takes into account of the conversations (and learning) that can occur between peers.

Representation of the confersational framework which presents space for teacher, learner and other pratice and links between teacher, student and peers, indicating the types of conversations that can facilitate learning

During the presentation, I noted (or paraphrased) the words: 'the more iterations through the conversational model you do, the higher the quality of the learning you will obtain'.  Expanding this slightly, you could perhaps restate this by saying, 'the more opportunities to acquire new ideas, reflect on actions and receive feedback, the more familiar a learner will become with the subject that is the focus of study'.

In some respects, I consider the conversational framework to be a 'meta model' in the sense that it can (from my understanding) take account of different pedagogical approaches, as well as different technologies.

Links to accessibility

Another 'take away' note that I made whilst listening to the presentation was, 'learning theories are not going to change, but how these are used (and applied) will change, particularly with regards to technology'.

It was at this point when I began to consider my own areas of research.  I immediately began to wonder, 'how might this model be used to improve, enhance or understand the provision of accessibility?'  One way to do this is to consider each of the boxes the arrows that are used to graphically describe the framework.  Many of the arrows (those that are not labelled as reflections) may correspond to communications (or conversations) with or between actors.  These could be viewed as important junctures where the accessibility of the learning tools or environments that could be applied need to be considered.

Returning to the issue of technology, peers, for instance, may share ideas by posting comments to discussion forums.  These comments could then be consumed by other learners (through peer learning) and potentially permit a reformulation or strengthening of understandings.

Whilst learning technologies can permit the creation of digital learning spaces, such as those available through the application of virtual learning environments, designers of educational technologies need to take account of the accessibility of such systems to ensure that they are usable for all learners.

One of my colleagues is one step ahead of me.  Cooper writes, on a recent blog post,  'Laurillard uses [her framework] to analyse the use of media in learning. However this can be further extended to analyse the accessibility of all the media used to support these different conversations.'  The model, in essence, can be used to understand not only whether a particular part of a course is accessible (the term 'course' is used loosely here), but also be used to highlight whether there are some aspects of a course that may need further consideration to ensure that is as fully inclusive at it could be.

Returning to the theme of 'scenario', one idea might be to use a series of case studies to further consider how the framework might be used to reason about the accessibility status of a course.

Connections

There may be quite a few more connections lurking underneath the terms that were presented to the audience.  One question that I asked to myself was, 'how do these formative assessment patterns relate to the idea of learning designs?' (a subject that is the focus of a number of projects, including Cloudworks, enhancements to the Compendium authoring tool, the LAMS learning activity management system and the IMS learning design specification).

A pattern could be considered as something that could be used within a part of a larger learning design.  Another thought is that perhaps individual learning designs could be mapped onto specific elements of the conversational model.  Talking in computing terms, it could represent a specific instantiation (or instance).  Looking at it from another perspective, there is also the possibility that pedagogical patterns (whether e-assessment or otherwise) may provide inspiration to those who are charged with either constructing new or using existing learning designs.

Summary

During the course of the day, the audience were directed, on a number of occasions to the project Wiki.  One of the outcomes of the project was a literature review, which can be viewed on-line.

I recall quite a bit of debate surrounding the differences between guidelines, rules and patterns.  I also see links to the notion of learning designs too.  My understanding is that, depending on what you are referring to and your personal perspective, it may be difficult to draw clear distinctions between each of these ideas.

Returning to the issue of the conversational model being useful to expose accessibility issues, I'm glad that others before me have seen the same potential connection and I am now wondering whether there are other researchers who may have gone even further in considering the ways that the framework might be applied.

In my eyes, the idea of e-assessment patterns and the notion of learning designs are concepts that can be used to communicate and share distilled best practice.  It will be interesting to continue to observe the debates surrounding these terms to see whether a common vocabulary of useful abstractions will eventually emerge.  If they already exist, please feel free to initiate a conversation.  I'm always happy to learn.

Acknowlegements

Thanks are extended to Diana Laurillard who gave permission to share the presentation slide featured in this post.

Permalink
Share post

This blog might contain posts that are only visible to logged-in users, or where only logged-in users can comment. If you have an account on the system, please log in for full access.

Total visits to this blog: 2360708