Personal Blogs

On 29 February 2024 I attended an academic conduct review briefing for module teams. The aim of this blog is to share some highlights that may be useful for colleagues (and to also try to remember what was covered, since there’s always a lot going on).

During the introduction, it was highlighted that academic referrals have increased over the last 5 years. Two thirds of these typically relate to study skills and poor academic practice. In other words, this relates to what could be called inadvertent misconduct.

There are some specific challenges. Investigations take time, there is an obvious need for consistency with decision making, a need for transparency, and continually emerging threats, such as essay mills and the arrival of generative AI. To respond to this changing landscape, there is a new VLE page for staff, a new academic conduct tariff, new guidance on StudentHome, a new referral form, and new processes and study skills arrangements.

Academic integrity principles for assessment design

Assessment design is important. Good assessment design should ‘reduce the motivation and opportunities for plagarism’ and focus should be on how learning takes place, as well as what is learnt.

An important point that I noted was that module teams should look at patterns of academic conduct referrals. One way to respond to repeated issues is, of course, to consider the assessment design. To help with this, there some useful principles of assessment which I share below.

A key theme of these principles is authenticity. This is connected with need for to assessments that focus on problem solving, perhaps solving problem that might have an open ended answer.

Here are the ten principles that were shared during the session.

Principle 1: Reward academic integrity

Make sure students reflect on their skills and capabilities that demonstrate academic integrity.

Principle 2: Assess higher skills

Assess metacognitive skills where students actively create an answer, with an emphasis on construction and their own thinking. This point is, of course, linked to the principle of refletion.

Principle 3: Open-ended solutions

This principle is related to authenticity. It’s possible to see authenticity within the undergraduate computing project module.

Principle 4: Diversify assessment formats

This is linked to how students might relate assessment to their own interests and preferences. Providing a variety of different assessment formats also speaks to the importance of accessibility and diversity.

Principle 5: Authentic assessment

Making an assessment real by sharing an exciting concept can enhance motivation. Make an assessment authentic by drawing on their own experience.

Principle 6: New assignment questions

To limit the risk of academic conduct, prepare new assignment questions every year. Design the assignments in such a way that students cannot re-use a previous answer. In other words, make it difficult for students to directly benefit if the questions were to be shared.

Principle 7: Process besides product

Ask students to submit interim versions of assignments or ask them to provide a demonstration. In the computing project module, students are asked to submit a draft EMA as an assessment.

Principle 8: Making quality criteria transparent

Make rubrics transparent; share with students how assessments are made so everyone knows what the module team (and the university) is looking for. 

Principle 9: Generating and acting on feedback

Encourage students to generate their own feedback, which helps to develop critical thinking. This relates to a principle of peer assessment. This is used, to a limited extent in a software engineering module, where students comment on interim designs. 

Principle 10: Authentication methods

Are students who they say they are? There used to be a time when students had carry out written exams by attending exam centres at a specific date and time. As a part of this, students needed to show a form of ID, such as a passport or a driving licence. An alternative approach would be for a student to have a conversation with a tutor about their assignment.

TurnitIn

A recent development has been the increased availability of TurnItIn to students. Module teams and staff tutors always used to receive TurnItIn reports. These reports indicated whether a submission contained words found in other online resources. A typical TurnItIn report might highlight legitimate quotes that a student used to evidence an argument, as well as uncited text.

On many modules students can now submit an earlier version of an assignment into TurnItIn. In turn, they can review a ‘similarity report’ to gain confidence that their assignment is adhering to sound principles of good academic practice. More information about TurnItIn can be typically found within a module’s assessment handbook.

Resources

A whole host of useful resources are available to students. The blog post Study Skills Resources: what is available? offers a handy summary of some of the most useful.

Reflections

I’m not sure where these assessment principles come from. I really like them. I can bring these ten points into module team meetings. Their focus on authenticity strikes me as being really important. It is important in terms of problem solving, skill development and the application of knowledge.

Putting the assessment design principles to one side for a moment, tutors are really important. If appropriate, students can refer students for additional support sessions, and can also refer students to study advisors who work within the student support team.

Students can then be referred to additional support needs, and referral to a study advisor. In my experience tutors may sometimes get a sense when a student may have used Generative AI.

Acknowledgements

Acknowledgements are duly given to the academic conduct review team who facilitated this session. The principles shared in this blog post are their own, and I take no credit for them. I hope I’ve summarised them effectively.

The second TU100 development day for associate lecturers in London and the surrounding regions was held on Saturday 7 September in the London regional centre.  The overall purpose of the day was to give associate lecturers who tutor on TU100 an opportunity to share experiences and to gather some useful feedback about the module that I could pass onto the module team.  These days are often great fun since everyone is very much up for sharing and talking (and this day was no exception).  This blog post represents a quick summary of what happened (from my own perspective, of course).

I’m writing this post for a number of reasons.  The first reason is to remember what happened on 7 September (since my memory is somewhat fallible), and the second reason is to give those tutors who couldn’t attend a bit of a feel for some of the subjects were discussed.  The third reason is to try to encourage other tutors to come along to other events that we run in the region.

There were essentially three different parts to the day.  The first part was all about teaching programming and Sense.  The second was about issues relating to student retention (where we heard about a university initiative called Project Retain), and the third was a general ‘feedback (or feedforward) to the module team’ session.

Session 1 : Teaching programming and Sense

During the first session we were put into small groups and Leslie, one of our very experienced TU100 tutors, distributed a questionnaire to inspire discussion.  These had the headings: ‘how does TU100 teach programming?’, ‘how does TU100 teach Sense?’, ‘student contact hours’ and ‘marking’.  Since I’m not a TU100 tutor I didn’t contribute too much to the group discussions, but I did make some notes of some of the themes that had emerged.

It wasn’t too long before the subject of programming cropped up.  One of the comments I’ve made is that the module doesn’t contain too much about testing.  One other thought is that early on in the module it is a good idea to emphasise the importance of Sense, particularly the Sense programming guide.  Another thing that tutors could do is to emphasise the wealth of Scratch resources that are available from MIT, and that perhaps we should more explicitly brief students that Sense is an extension of Scratch.

We soon began to talk about the on-line sessions which are presented through Blackboard Collaborate (or OU Live, as the university calls it).  One of the challenges with using the OU Live software is that it takes time to hand over screen sharing control when tutors ask students to complete certain tasks. 

An interesting point is that OU Live might not only be useful for running tutorials.  Since it contains a facility to record sessions it can also be used to record how any application is used.  Tutors (or faculty staff) could use OU Live to make ‘video’ recordings to demonstrate some programming concepts.

One of the biggest challenges that tutor’s face is the marking of assignments.  Sometimes tutors come across some puzzling situations, i.e. if students submit work where a screenshot represents a correct functioning program, but the program that is submitted isn’t actually correct.  When it comes to correspondence tuition, one of the fundamental challenges is to get into the head of the student.  This led to the question of whether we might be able to record video clips to show how students could have created correct solutions.

Plenary

After around fifteen or twenty minutes of chatting, all groups were asked to report back.  This section is a quick summary of some of the key points that some of the groups mentioned. 

TU100 doesn't contain a section that is dedicated only to programming.  Instead, programming can found in different sections throughout the module.  One point mentioned by tutors was that whilst TU100 teaches coding it doesn’t say much about how to do the 'problem solving' part of programming.  Instead, students are required to spend time discovering how to program by exploring and playing with the Sense environment.

Aware of this issue, some TU100 London tutors have started to present the fundamentals of how to break apart problems into pieces that could then be used to create code (either in the face to face sessions, or on the on-line sessions).  The precursor to TU100, M150 contained some materials to introduce students to something called structured English.  This gave way to a debate about whether some additional material might be added to TU100, but the problem is that there are already lots of materials that students and tutors need to cover. 

The point is that the foundations (in terms of learning to program) are really important, especially for students who might potentially struggle with the fundamentals of programming.  One tutor said that some students never make it to the starting line on Sense and this kind of resources could be a bridge between high level thinking and programming.  Some of the fundamentals that could be covered (by tutors) include the basic constructs of programming, which includes sequences of instructions, selection, iteration, the use of variables and debugging.

One tutor said that ‘we need to emphasise that it is important that students have a go’ (so students gain an understanding of what the building blocks of software is all about).  Also, there is need for a Sense forum, something or some area that allows sharing of materials and ideas between students and tutors. 

One piece of advice to students should be, ‘go look at what people do with Scratch’.  Another comment was, ‘add a couple of YouTube type videos about program analysis’.  The interactive nature of programming does lend itself to the use of OU Live, via application sharing, but on-line asynchronous tutorials are always going to be difficult and it takes a skilled facilitator to use more sophisticated functions such as on-line break out rooms.

Another perspective was that it might help the students if there was slightly more signposting to different resources.  (I understand that this is something that the module team have been working on for the new presentation).

Contact hours, tutorials and day schools

Different regions do different things when it comes to on-line tutorials and day schools.  When it comes to on-line time, the London region has given tutors the opportunity to schedule and run individual sessions.  The south region runs join sessions, as does the south east region.

When it comes to the face to face sessions, all the London groups come together to form a series of big day schools with the intention of creating a critical mass of both students and tutors.  In other regions tutors run sessions with pair of tutors.  The differences can be down to geography, both in terms of the location of the students and the location of the tutors.  One other thought from my side is that it is also important to emphasise to all students that they are encouraged to go to any of the tutorials that they might find in the tutorial finder (so they can discover evening as well as weekend events).

Some tutors use materials that are created by the module team, whereas others create their own materials.  One example is the London region tutors creating materials in structured English, with a view to trying to ‘plug a gap’ in the module materials (regarding how students new to programming might set about splitting a program into different components).

Another approach that some tutors adopt to use their allocated on-line time is to run on-line drop in sessions via OU live.  The idea for this is that students could just pop into an on-line room to have a chat with a tutor if they had any questions.  I personally find this a really compelling way of making use of the on-line rooms, particularly when students might be wishing to chat about programming.  The breaks with the formality of a one-to-one conversation of the technology, but also allows participants to see what is being displayed on a shared whiteboard.

Working with OU Live

The first tip (for tutors) was, ‘remember to switch on your microphone’.  Another thought was, ‘can we make headsets compulsory please?’  The reason for this is simple: when students use the microphone and headset that is built into a laptop, a whole group of participants can be easily distracted by feedback, making communications a whole lot more difficult.

In some respects, participating in an OU Live session can be quite intimidating and one observation was that there are lots of students who don’t want to speak at all.  Sometimes some students prefer to use the text chat window rather than using the microphone, which can then make if quite difficult for the tutor to keep on top of everything (which is why some regions share OU Live sessions between tutors).

One point was that it is useful to ‘do something’ every 20 or so seconds.  This might be asking students questions, requiring them to respond with yes/no answers.  Another thought is to use a series of polls to assess understanding of certain concepts.  (One thing that I have personally learnt from my experience with the South East of England training is to poll students using the, ‘happy face’ button, i.e. by asking the students, ‘is everyone happy?, can you click on your happy face?’  When you regularly ask this, it helps to keep the student’s attention).

Marking of code

This section of the plenary discussion echoed an earlier point, that when it comes to communicating what needed to be done with complicated TMA questions (which involve programming), could the module team produce a video about how things should run, or have been constructed (using Sense)?

I’ve learnt that there are two different ways to add comments into Sense code.  One way is to use something called a comment window.  Another is to add some in-line comments.  I made a note of a debate about the use of different types of comments and that in previous assignments a TMA question asked students to add comments.  The consensus was that comments help; they help students to reflect on the code that is being written and help tutors to understand what has been submitted.

Project retain

An interlude between the first and the second session was presented by Maggie King, our associate dean for teaching and learning.  One of Maggie’s responsibilities has been to be a part of a university wide project called ‘project retain’.  

Project Retain is intended to increase the university’s retention (and progression) of students across different levels of study.  The project has given the university a number of recommendations, which include: offer a guide to key learning points and module materials, schedule and communicate real-time contact sessions during the first two weeks of a module (ideally through a letter), open module materials and web sites before the module starts, and make it clear when assignments are coming up (so our students are not surprised when they have to submit their assignments).  The first year of study, it was argued, is absolutely crucial.

Session 2 : Retention

Terry, one of our experienced TU100 tutors facilitated the second main session of the day, which was also about retention (which is an issue that affects student satisfaction scores, recruitment and funding). 

Terry introduced us to HEFCE performance indicators.  These include dimensions such as the national student survey and other aspects such as the measurement of research performance.   Terry also introduced the difference between retention and progression.  Progression is all about moving from one level of study to another.  In some circumstances students can defer, allow them to take a bit of time out from study and enabling them to pick up a module again at a later date.

One of the biggest changes in the university in the forthcoming couple of years will be the introduction of something called student support teams.  Since more and more students will be registering with the intention of studying for a particular qualification, student support teams will play an important role in helping students with their choices along a student pathway – it is hoped will positively impact on student retention.

Terry covered a wealth of materials, including sharing with us points from a national audit office report, drop out rates, how retention in UK HEIs compare with the retention in other countries, and how the university compares with others in the national student survey.  During his session Terry asked us to consider the causes of student drop out during different stages of study, such as pre-entry, induction, on-programme and movement to the next level.  In the university both tutors, student advisors and module teams all have an important role to play.  The final question of the day was, ‘how can the university support you in the task of improving retention in your tutor groups?’  This was an exceptionally very good question to ask and is something that I’m keen to pick up on and delve into when I have a bit more time.

Session 3 : Open Session

I have to confess that I haven’t taken too many notes about this final session mainly because we ran out of time!  Everyone was very willing to share experiences and opinions throughout the day, which was one of its fundamental objectives.

Reflections

One tutor made the comment: ‘you can make a full time job of teaching TU100’.   TU100 is, without a doubt, a very big module: there is a lot of material and there are a lot of demands on the tutor’s time.  What struck me about this day was the willingness of tutors to do their utmost to help their students along their TU100 journey and their willingness to share experiences with each other.  The event had lots of energy and there was a lot of positive talking going on, yielding some very good ideas.  From my own perspective, I certainly hope to be running a similar event next year.  I’ve already had a couple of thoughts about what we might do.

I have learnt quite a few things from this session.  I’ve learnt about the opinions that tutors have about certain aspects of the module and I’ll be happy to forward these directly to the module team.  It is also clearly apparent that some students struggle with programming and the idea of producing some video material to help to explain certain concepts might be something could be useful.

Acknowledgements

Many thanks to all our TU100 associate lecturers who kindly gave up their valuable time to attend this event on a Saturday. If any of the tutors who have attended would like to add further comments, please don’t hesitate to comment below.

A couple of weeks ago I was lucky enough to be able to attend a 'formative e-assessment' event that was hosted by the London Knowledge Lab.  The purpose of the event was to disseminate the results of a JISC project that had the same title.

If you're interested, the final project report, Scoping a Vision for Formative e-Assessment is available for download.  The slides for this event are also available, where you can also find Elluminate recordings of the presentations.

This blog post is a collection of randomly assorted comments and reflections based upon the presentations that were made throughout the day.  They are scattered in no particular order.  I offer them with the hope that they might be useful to someone!

Themes

The keynote presentation had the subtitle, 'case stories, design patterns and future scenarios'.  These words resonated strongly with me.  Being a software developer, the notion of a design pattern (wikipedia) is one that was immediately familiar.  When you open the Gang of Four text book (wikipedia) (the book that defines them), you are immediately introduced to the 'architectural roots' of the idea, which were clearly echoed in the first presention.

The idea of a pattern, especially within software engineering, is one that is powerful since it provides software developers with an immediate vocabulary that allows effective sharing of complex ideas using seemingly simple sounding abstractions.  Since assessment is something that can be described (in some sense) as a process, it was easy to understand the objective of the project and see how the principle of a pattern could be used to share ideas and facilitate communication about practice.

Other terms that jumped out at me were 'case stories' and 'scenarios'.  Without too much imagination it is possible to link these words to the world of human-computer interaction.  In HCI, the path through systems can be described in terms of use cases, and the settings in which products or systems could be used can be explored through the deployment of descriptive scenarios and the sketching of storyboards.

Conversational framework

A highlight of the day, for me, was a description of Laurillard's conversational framework.  I have heard about it before but have, so far, not had much of an opportunity to study it in great detail.  Attending a presentation about it and learning about how it can be applied makes a conceptual framework become alive.  If you have the time, I encourage you to view the presentation that accompanies this event.

I'm not yet familiar enough with the model to summarise it eloquently, but I should state that it allows you to consider the role of teachers and learners, the environment in which the teacher carries out the teaching, and the space where a learner can carry out their own work.  The model also takes into account of the conversations (and learning) that can occur between peers.

During the presentation, I noted (or paraphrased) the words: 'the more iterations through the conversational model you do, the higher the quality of the learning you will obtain'.  Expanding this slightly, you could perhaps restate this by saying, 'the more opportunities to acquire new ideas, reflect on actions and receive feedback, the more familiar a learner will become with the subject that is the focus of study'.

In some respects, I consider the conversational framework to be a 'meta model' in the sense that it can (from my understanding) take account of different pedagogical approaches, as well as different technologies.

Links to accessibility

Another 'take away' note that I made whilst listening to the presentation was, 'learning theories are not going to change, but how these are used (and applied) will change, particularly with regards to technology'.

It was at this point when I began to consider my own areas of research.  I immediately began to wonder, 'how might this model be used to improve, enhance or understand the provision of accessibility?'  One way to do this is to consider each of the boxes the arrows that are used to graphically describe the framework.  Many of the arrows (those that are not labelled as reflections) may correspond to communications (or conversations) with or between actors.  These could be viewed as important junctures where the accessibility of the learning tools or environments that could be applied need to be considered.

Returning to the issue of technology, peers, for instance, may share ideas by posting comments to discussion forums.  These comments could then be consumed by other learners (through peer learning) and potentially permit a reformulation or strengthening of understandings.

Whilst learning technologies can permit the creation of digital learning spaces, such as those available through the application of virtual learning environments, designers of educational technologies need to take account of the accessibility of such systems to ensure that they are usable for all learners.

One of my colleagues is one step ahead of me.  Cooper writes, on a recent blog post,  'Laurillard uses [her framework] to analyse the use of media in learning. However this can be further extended to analyse the accessibility of all the media used to support these different conversations.'  The model, in essence, can be used to understand not only whether a particular part of a course is accessible (the term 'course' is used loosely here), but also be used to highlight whether there are some aspects of a course that may need further consideration to ensure that is as fully inclusive at it could be.

Returning to the theme of 'scenario', one idea might be to use a series of case studies to further consider how the framework might be used to reason about the accessibility status of a course.

Connections

There may be quite a few more connections lurking underneath the terms that were presented to the audience.  One question that I asked to myself was, 'how do these formative assessment patterns relate to the idea of learning designs?' (a subject that is the focus of a number of projects, including Cloudworks, enhancements to the Compendium authoring tool, the LAMS learning activity management system and the IMS learning design specification).

A pattern could be considered as something that could be used within a part of a larger learning design.  Another thought is that perhaps individual learning designs could be mapped onto specific elements of the conversational model.  Talking in computing terms, it could represent a specific instantiation (or instance).  Looking at it from another perspective, there is also the possibility that pedagogical patterns (whether e-assessment or otherwise) may provide inspiration to those who are charged with either constructing new or using existing learning designs.

Summary

During the course of the day, the audience were directed, on a number of occasions to the project Wiki.  One of the outcomes of the project was a literature review, which can be viewed on-line.

I recall quite a bit of debate surrounding the differences between guidelines, rules and patterns.  I also see links to the notion of learning designs too.  My understanding is that, depending on what you are referring to and your personal perspective, it may be difficult to draw clear distinctions between each of these ideas.

Returning to the issue of the conversational model being useful to expose accessibility issues, I'm glad that others before me have seen the same potential connection and I am now wondering whether there are other researchers who may have gone even further in considering the ways that the framework might be applied.

In my eyes, the idea of e-assessment patterns and the notion of learning designs are concepts that can be used to communicate and share distilled best practice.  It will be interesting to continue to observe the debates surrounding these terms to see whether a common vocabulary of useful abstractions will eventually emerge.  If they already exist, please feel free to initiate a conversation.  I'm always happy to learn.

Acknowlegements

Thanks are extended to Diana Laurillard who gave permission to share the presentation slide featured in this post.