On Thursday 13th October 22, I attended a workshop facilitated by Sara Spencer, Head of Research Degrees, and Emeritus Prof Marian Petre from the School of Computing and Communication, that was all about examining a doctoral thesis,
The workshop was described as being intended “for research degree supervisors who are new or fairly new to the process of examining PhD and Professional Doctorate theses” and was open to both new and experienced research supervisors.
The broad aim of the session was to provide an “introduction to thesis examination at the OU”, to provide a summary of “what is involved in examining a research degree thesis”, explore the roles of different participants in the process, and to say something about expectations in terms of what takes place during the examination process, and the role of the viva and thesis.
In some ways, this event reminds me of an earlier workshop that I attended in June 22, which had the title Supporting EdD/PhD students through the thesis and the viva (OU blog). One of the differences between this session and the earlier session, is that this session provides a bit more information about the different roles.
On the topic of blogs that might be useful, this earlier blog, Doctoral research: a short introduction published in October 2022 might be also helpful for prospective students.
What follows are a set of (edited) notes that I made during Sara and Marion's workshop.
Session objectives
Participants were invited to contribute to an online document to share what they were looking for from the session. Some of the key points included: to learn from the experience of others, to understand what the overall process is, what to do if there are disagreements, to share general tips about how to approach examining a thesis, and how to provide feedback.
The facilitators shared some of their experiences, and begin to discuss some of the roles within an examination, and what happens. Some points I noted down were that supervisors will try to choose examiners who are appropriate (given the focus of their student’s research) and examiners should always endeavour to set their egos aside: it isn’t about them, it is about the student who has written a viva, and their research.
Roles and responsibilities
Marion made the following important points: the viva is a real examination and conducting it well matters. Also, if a student has a flawed dissertation, a student can strengthen their position through the viva process. Conversely, if a student has a strong thesis, but gives a weak defence, the outcome might not be as hoped.
Panel chair
Every doctoral examination has a panel chair. The job of the chair isn’t to ask questions, but to moderate the session, mediate communication between everyone, check that everything is going okay, ensure that procedures are followed, and ensure that everyone feels comfortable. The chair is someone everyone can appeal to if help or support is needed, and call for breaks, if necessary. Unlike the examiner, the chair may not be a subject specialist, but will be someone independent and experienced who understands the process. In contract to the chair, the doctoral supervisors are outside the examination process.
Examiners
The examiners assess the quality of the research. There will usually be either two or three examiners, and two of those may be external. The examiners read the dissertation thoroughly in advance, and prepare a re-viva report. Under strict confidence, the chair will then share each report with each of the examiners. Examiners are also expected to be familiar with the university’s regulations and must work with the chair to prepare an examination report. They must also be willing to provide clarifications for the student if required and assess any revisions, and conduct a re-viva if necessary.
Although there is an expectation that the viva examination process a relatively short amount of time, examiners may be employed within the process for considerably longer, especially if a student is required to carry out remedial work to their viva.
Observer
An observer is allowed to come along to the viva, and it is typically the lead supervisor. The role of the observer is very limited, and the observer doesn’t speak unless invited to do so by the chair. An observer may well take notes, to help the candidate understand what happened within the viva, and to help the candidate remember some of the detail of the discussions that took place.
Candidate
The candidate is, arguably, the most important person in the room (although it might be argued that the chair is just as important). The whole event is about the research that the candidate has carried out, and to check to see whether they have a thorough and detailed understanding of what they have done, and their subject. An important point is this: there are very few opportunities in life where we have opportunities to talk to a group of other people, at length, about a subject that we are very interested and passionate about. With this perspective in mind, and it might even be possible to think of the viva as a precious and potentially even enjoyable event. A candidate can request breaks via the chair, and always ask for clarifications to any question that is asked.
The procedures
As mentioned earlier, examiners read the dissertation, and the examiners prepares a report 5 days before the viva, which are then shared with each of the examiners via the panel chair. An examiner may form an opinion which may be expressed within the form, but this need not be fixed: “the report is not a contract; it is an initial assessment”. This assessment can change depending on what happens within the viva.
Pre-viva meeting
During the pre-viva meeting, the chair and examiners meet to discuss their view and opinions about the thesis. The report helps everyone to see if there is a consistent perspective. Using the reports, the examiners will form an approach. They will discuss a plan about how ask questions.
The exact approach will be different, depending upon the examiners, thesis, and subject. On some occasions, examiners might start with some very easy questions and then work towards points that really matter. Other examiners may choose to take turns, and some will go through a chapter at a time. Sometimes the external will lead, and the internal will follow.
Before the viva, the chair will have some idea of what is going to happen, and how the thesis will be assessed. The chair also provides and offers any necessary clarification about regulations. An important note is that every organisation is slightly different.
The Viva
This is the key meeting between the candidate, the panel chair, examiners, and any observer. Typically, the chair introduces the panel and provides an overview of what is going to happen.
A viva lasts as long as it takes. It might typically last between an hour and a half and three hours; online takes a bit longer. There should be no particular end time. A point that was made: there is no correlation between the length of the viva and the outcome. Breaks can be requested by any participant, via the chair.
Post-viva meeting
After the viva, the examiners, and the chair meet. The candidate and observer are asked to leave the room, where they discuss what has happened, and what recommendation is to be made. The duration of the post-viva meeting also takes as long as is necessary. If examiners do not agree (which very is unlikely), and there is a formal procedure to take account of this. It was emphasised during this session that this a very rare occurrence: examiners tend to agree.
Recommendation meeting
Everyone meets up again, and the recommendation is shared with the student.
During this meeting there is an opportunity for the examiners to provide some feedback. Revisions are discussed (if necessary), and the observer usually takes notes. During this meeting, the candidate may ask questions.
An important part of this process is the completion of an examination report form, which contains an outcome. The outcome is a recommendation to a university authority, and the panel offers a recommendation summarising what revisions are necessary, and why.
Outcomes
Assessment criteria for a thesis is presented on the examination report form. Points include presentation and style of the thesis (whether the candidate is able to contribute to academic debates), evidence of the work being a significant contribution to knowledge, whether the candidate show evidence of being able to carry out research in the future, and whether the thesis contain material worthy of publication.
On the point of publication, both Marion and Sara emphasised that publication is neither necessary, nor sufficient for a PhD; the thesis is a monograph, not a collection of papers.
In the OU, there are a number of different possible outcomes: the candidate is awarded the degree, the candidate is awarded the degree with minor corrections, or the candidate has to make substantial amendments. Other outcomes include: the candidate must resubmit their thesis for re-examination, a degree of MPhil is awarded subject to dissertation amendments, resubmission of thesis for re-viva for a MPhil award, and finally, a student is not awarded the degree and not permitted to be re-examined.
Outcomes will be based on the quality of the submission, and each category has a specific timeframe, i.e., minor corrections might be required to be completed within 3 months, and major correction may have to be submitted within 6 months.
How to be an effective examiner
Towards the end of the session, there was a discussion, where participants shared tips about how to be an effective examiner. I noted down the following points from a PowerPoint slide that directed the session: “the best examiners bring out the best in the student” and “there is a correlation between examiner experience and moderation/kindness”.
Marion emphasised the point: “look for the value in the work; whether it conveys a sense of confidence and contribution” Another point was: It is about people skills, as much as it about technical skills. Also, create a rapport with the candidate before asking any tough (but necessary) questions, such as: what did you enjoy, how did you come to study this in the first place? Make sure that you listen well to all answers.
Judge the work on its own merits and make sure that you don’t impose your (examiners) framework on the candidate’s work. Break down larger questions to smaller questions, and give sufficient time to allow your questions to be answered. Importantly, reflect on your own tone and way of communicating, and potentially mention this to the candidate to put them at ease. Be very mindful of how the candidate might be experiencing stress during the viva, and encourage breaks.
A really important point I noted down was: what does “good enough” look like in your discipline? In the viva, what matters is a pass. Another comment was: very few dissertations are without flaws. Always look to what is good in a thesis.
Reflections
This session made me think about to my own viva. My viva was a positive experience. At the time, I didn’t have a really thorough understanding of what everyone’s roles were. I remember the internal examiner, and the external examiner, but I can’t remember who the chair was. I do remember the close scrutiny of the work that I submitted, and a feeling of being asked some really difficult questions.
Interestingly, I also remember that the internal examiner really liked a certain aspect of my thesis, where I drew on materials from outside of my home discipline. In retrospect, I think this may have contributed to the assessment that I was capable of carrying out original research, which is such an important part of the process. The point here is that I remembered the nice bits, just as I remember the tricky bits.
In the next two months, I’m going to be an external examiner. Attending this session has helped me to strengthen my understanding of the process, and really emphasised what my role and responsibilities are going to be.
I remember another bit of advice I was given by a colleague when I was preparing to be an external for the first time. The advice was about how to approach the reading of a thesis: “Look to what happens within the methodology. The methodology is about what has been done. Does the methodology make sense, given the research questions?” Whilst this bit of advice is practical, the most important bit of advice from Sara and Marion’s session was: “make sure you’re approachable”.
Acknowledgements
The structure of this blog directly echoes the session that was designed and facilitated by Marion Petre and Sara Spencer. Many of the words within this blog also reflect points made by both Marion and Sara. I hope I’ve done justice to your excellent session!