Personal Blogs

On 9 November 2018 I attended a continuing professional development (CPD) course for external examiners run by the Higher Education Academy (or, AdvanceHE, as it is otherwise known). The course was facilitated by two OU colleagues: Professor Mark Brandon from the STEM faculty, and Naomi Watson from the WELLS Faculty. What follows are a set of notes that I made before and during the course. A week after the course, I edited everything together so I would have a rough sketch of what happened during the event. 

The aim of the course was to further understand the role of an external examiner, develop a deeper understanding of the nature of academic standards, and to ‘use evidence-informed approaches’ to inform judgements about ‘academic standards and the enhancement of student learning’.

The course was split up into two parts an online component (part 1) and a face to face component (part 2); participants had to complete both of these parts to complete the CPD. An interesting point was that completion was also recorded by the HEA. 

During the event, delegates were given a nicely printed A4 sized book, which had the catchy title: professional development course for external examiners. During the course we dipped into the book and completed a number of activities, writing down some personal reflections and thoughts. From time to time, I’ll refer to the book, the activities, or both. 

Preparing for the face to face session

To prepare, we all had to login to a virtual learning environment and complete a couple of activities (which I summarise below). The introductory information was useful; we were referred to the Higher Education Academy's A handbook for external examiners (PDF).

I also noted down the words: “external examiners gain oversight of assessment process, provide comment about the assessment process to say whether student learning outcomes are met and to offer informative comment on good practice”.

Activity 1: External examining of student work

We were asked to read an assessment briefing paper that described principles of feedback. The briefing paper contained information about learning outcomes, the level of study, information about the task that students had to complete, and provided a marking scheme. There were three learning outcomes: one about understanding, another about designing and a final one about taking account evidence into. Students were required to write a critical review of their own assessment practice.

We had to review three assignments and judge whether the work has been given the correct marks in coordination with the scheme and write a paragraph to give feedback regarding the academic standards of the module based on the assignments.

Activity 2: The external examiner role

This second activity emphasises that external examiners have a duty to help to maintain academic standards, ensure that institution policies and regulations are followed, that standards are comparable with those in other institutions, and to share good practice, and to be a critical friend. 

We were asked to comment on a short series of scenarios: a situation where a course leader asks for help, a situation where we had to deal with differences of opinion between examiners, and to consider a situation where there was low quality student work despite institutional staff working very hard to maintain standards.

During the face-to-face session

The face-to-face session was split into 7 short components (excluding a concluding section). These components had a mixture of listening, group work, followed by individual reflection activities. What follows is a very short summary of notes from these different sections. 

Session 1: Introduction

The first session was an introduction which referred to the QAA’s UK quality code for Higher Education, Chapter B7: External Examining (pdf) Echoing the introduction to this blog, externals are required to ensure threshold standards (QAA quality code, chapter B7, p. 4), ensure that processes are followed, and also ensure that academic standards are comparable between institutions.

During this first session we reviewed the activity 2 scenarios to further understand the role of the external examiner, and to understand the tensions, and to understand how to balance the different demands of ‘the process checker’, ‘critical friend’, and the maintainer of standards.

As a brief aside, it’s important to mention that appointments to external examining posts are made by universities. Examiners are recruited either through personal recommendations (where people are encouraged to apply), or through a national mailing list, which interested academics can sign up to.

Session 2: Variability in academic standards

In the second session we looked at another scenario and were asked to think about the challenge of how we actively score assignments. During the task I asked myself about the extent to which learning outcomes can or should be linked to assessment criteria. In the context of our scenario, a question that was asked (amongst our table) was: do students have access to the marking criteria?

This session had a second task, where we were asked about what issues might lead to variability in academic standards. I made a note of differences between people, tools and the task (nature of assessment).

Session 3: People as a source of variation

An interesting point that I noted down was an assertion that standards are socially constructed. To understand more about what this meant, and the idea of variation further, we were asked to complete another exercise that began with a question: what shapes our standards? Some points that I noted down were: from our institutions, values and beliefs, but also from specialist and professional knowledge of subjects.

Another question that we were asked was: where did you get your standards from? Some answers included: 

• Influential people and groups such as colleagues and networks
  
• Experience of working with students, carrying out assessments, and working within industry 
  
• Known and understood professional values and beliefs, such as the aims of higher education and values from a discipline or subject.
  

Sessions 4 & 5 : Tools and tasks

This activity complemented one of the online preparation activities.  We were asked to look at module specification documents and descriptions that were from different sectors, professions and institutions. We were asked to think about internal reference points, such as qualification descriptors, learning outcomes and assessment guidance and external reference points, such as subject benchmark statements. 

These discussions led to an activity: we were given a set of cards which related to different scenarios that we might observe as external examiners. We were asked to place the cards on a two dimensional grid. One axis had the title ‘internal/external’ (which related to the types of tool the care related to), and the other had the title: ‘high/low effectiveness’. After a figuring out where the cards went, we were invited to have a look at what other groups had done.

Session 6: Profession practice

This session was all about figuring out what to do in certain situations. In our groups we were given a set of different external examining scenarios, and offered 5 different choice cards. After one of the group read out the scenario, we were invited to vote on what we thought was the best course of action. After the voting, discussed why we had chosen to vote the way we did.

It was a fun exercise; there were friendly differences of opinion about what strategies to adopt. I sensed that there was no right or wrong answer, and the best course of action might depend on a combination of different factors, including the institution, subject, and the colleagues that we’re working with.

Session 7: Social moderation and collaboration of standards

The penultimate session featured a couple of videos. There was one video of exam marking of a musical performance, and another video of a project where academics from Australian universities discussed how they would mark different pieces of work. In some respects, the second video (which featured a marking exercise) very much resembled by own experience of being a project marker on a Computing and IT project module. During the co-ordination meeting different tutors would present their views and justify their marks.

There seemed to be an important point underlying this final session, which I noted down, which is: ‘you need to be talking to other people within your subject to understand what the standards are’.

Reflections

All in all, this CPD was pretty good fun! Having had some experience of being an external examiner, I found that some of the discussions directly resonated with my personal experience. 

One of the key points that I took away from the session was a differences between internal and external documents (or tools) that can help and guide the external examiners. 

Although I had an implicit awareness of the distinction, the way that it was made clear was very helpful. In my own experience, I’ve been reviewing course descriptions and marking guides (internal documents) and also having a look at different qualification outlines (external documents). I have also remembered that I have, on occasions, had a look at module descriptions from other institutions (to help me carry out a comparison of standards).

I enjoyed the interactive element and the opportunity to discuss issues with colleagues from other schools and faculties. I don’t have any suggestions about how to improve the course, since it offers a lot of tools and useful tips. The next step for me is to try my best to connect what I’ve learnt to my current external examining contract.