Personal Blogs

On 14 June 2023, I attended a professional development event that was all about the previous REF, the next REF, and the sharing of tips about how to prepare what is called a 4* paper. The event was facilitated by Rachael Luck, who is a lead for a design qualification, and Toni Gladding.

The REF is the UK Research Excellence Framework, which is a controversial system that ranks UK universities in terms of the quality of the research that they carry out. It was interesting to hear that both Rachael and Toni were both on review panels for the previous REF.

The most recent REF happened in 2021, and the next one is due in 2028. The broad aim of this session was to begin to think about preparation for the 2028 REF, to begin to consider what a 4* paper looks like, and to have a discussion with colleagues.

What is a 4* paper looks like is, of course, a highly debated and contested question.  A point was: to try to increase quality for next REF, need to be thinking about it, and talking about it early.

The REF score matters. The higher the overall score, the more Quality Related (QR) income (UKRI website) an institution gains from the government. During this session, I learnt there is a multiplier effect, which means that 4* paper attracts 4 times the funding of 3* activity. A small improvement in the number of 4* papers that are published could have a large impact in QR funding. This funding is important since it is used to fund school doctoral studentships and university facilities. This, in turn, can increase the development of the university’s research capacity.

What follows is an edited set of notes I made during the session.

What happened in REF 2021?

University has a code of practice. If someone has a certain amount of research time, has to submit at least one output to REF 2021. There is a long tail of “other colleagues” who were also included. It is the quality of the output from those colleagues who might benefit from additional support. Since the rules for the 2028 REF have not yet been defined (but will be available in January 2024), there is a possibility that a wider group of staff might be included.

In the 2021 REF, 157 universities were assessed. In the OU context (if I’ve noted this down properly), 44% of staff only had 1 output, 10% produced 5 outputs.

What is a 4* output?

Each category of quality assessment has an accompanying “quality descriptor”. A three star paper is considered to be research that “is internationally excellent in terms of originality, significant and rigour but which falls short of the highest standards of excellence”. A 4* bit of research is “beyond internationally excellent”, but I’m not exactly sure what this means.

The university strategy is to maximise QR income, to increase its overall score, to aim to gain higher QR income. A strategy for an individual unit, such as a school, may be to try to gain the highest possible score with a specific assessment area.

An output could be chapters in books, design, exhibitions, journal articles, monographs, performance. This said, the largest and most significant output across universities and all units of assessment (UOA) are journal articles.

Key components of a high quality paper

This section of the session asked the question: what does a 4* paper look like? 

One way to answer this question is to consider what the characteristics of a good journal article might be. A paper should have an eye catching title that has a broad appeal, and a clear abstract which highlights clear research questions, summarises methods, and offers a clear summary of the novel contribution made to knowledge. 

Turning to the body of the article, a paper should clear clearly state research questions, clearly describe the methods that are applied (and share clear citations that offer more detail about them), explain how the research relates to theory, describe research instruments and ethical approach taken, provide details about the data samples, and conclude with acknowledgements. Not having dealt with ethical issues will limit the journals that you can submit to.

An important tip I noted down was: “know about the journal that you’re writing for”. See whether your research or research article is an argument that fits with the aims of the journal. A phrase I noted down was: “papers are not written in the abstract, they’re written with a journal in mind”.

This section of the presentation reminded me of a blog article that I wrote some years back, which had the title Getting published in Open Learning. Open Learning is, of course, a journal that I co-edit with two other colleagues. The points that were made about the clarity of papers is really important. A suggestion for anyone who is interested in writing papers, is to get involved with the peer review process in some capacity. This way you can learn more about how everything works.

During the session, the notion journal impact factors were mentioned, which are known as bibliometrics which highlight how influential a publication is within an area of research. When it comes to the REF, it isn’t just the impact factor of the journal that matters; the assessment of quality of paper is independent of the quality (or impact factor) of the journal. This is linked to something called DORA, which is an abbreviation of: Declaration On Research Assessment (DORA website), which the OU has signed up to, along with other higher education institutions (HEIs). 

What can we change?

Research can be thought of in different ways. Research is, of course, carried out by individual researchers and academics, but since much research (especially within STEM require collaboration, a wider perspective is necessary. Individual academics should develop their own research strategies (which could be shared with others), but also strategies should be defined by units, such as schools, which then should be linked to REF units of assessment. This also links to the importance of developing a research culture.

An interesting question was: what practical things could be done?  Some suggestions include working with others to ensure thar articles are thoroughly proofread before submitting to journals, perhaps setting up internal peer review processes within research groups, writing articles with co-authors, and setting up mentoring schemes with colleagues who are new to research.

The discussion of papers led onto a discussion about the challenges that accompany getting articles published in open access journals. Open access publishing means that the readers do not pay for access to research articles. Instead, researchers and writers pay. This means that authors may be asked to pay what can amount to a substantial publishing fee. This can limit the extent to which researchers are able to publish unfunded research, such as personal projects.

A suggestion is to always consider the cost of publishing fees when writing and preparing research bids, to make sure that they are considered during the budgeting. Also, if research is funded by a UK research funding council, the university library has an agreement that they will cover publication fees. 

By way of contrast, Open Learning offers a hybrid scheme. The editors of Open Learning chooses to ‘open access’ one article per issue. Individual authors can also secure open access by submitting what amounts to a publication fee, but it is typically less than journals that adopt a completely open access funding model. The advantage of this approach is that it enables institutions that do not have extensive research budgets to disseminate their research. An important point here is: pay attention to the terms and conditions that relate to the journal that you’re hoping to publish through.

One of the final comments I noted was: there are tensions between helping those currently work on 3* papers, to help them to submit 4* paper, and helping those with little research time to start publishing. A big win for the university might be to enable a wider group of staff to be included in the REF. This also links to the tension between supporting the individual researcher, and supporting the institution as a whole to gain an overall higher rating.

Reflections

I don’t like the REF for the simple reason that terms such as quality and excellence are dangerously slippery terms. Also, research is something that takes place over an extended period of time, and certainly not across a neatly defined seven year cycle. I’m also grumpy after learning that 4* papers have a multiplier effect on research income for the simple reason that universities, schools and individuals who are already doing excellent research are likely to gain even more income to do even better research, with more resources. This multiplication factor is not progressive, and it feels far from equitable. Institutions with a lower research ranking will, of course, struggle more to make an impact in comparison to, say, Russell Group universities.

I come to this topic having struggled to build a research strategy and profile. As a staff tutor, half of my time is dedicated to academic leadership and management, and the other half is dedicated to ‘academic stuff’, which means either working on module teams or working on research. If I have module team (teaching) responsibilities, this means that less of my time is available to research, and what really helps with research is building those close collaboration with colleagues, and all this takes time.

Over the last few years, and to this day, a lot of my time has been spent dealing with internal institutional changes (mainly, the new AL contract), of which a lot still remains to be figured out and understood. Whilst this does sound a little grumpy, dealing with the practicalities of my staff tutor day job leaves very little practical time to carry out disciplinary research. Whilst I’m not contractually obliged to make a REF submission, the idea of leading a submission of an article feels current unattainable, never mind the notion of producing an article that is “beyond internationally excellent”.

What are the answers? One of the obvious answers is to work on research culture. This requires the involvement of multiple stakeholders, at multiple levels. It means facilitating conversations, and learning about the interests of others. It means getting involved and encouraging others to get involved. It means joining research groups, and being patient when listening to the perspectives, challenges, and experiences of others. This is also, of course, made harder when everyone is working at a distance.

Two thoughts that I return to regularly are as follows. The first thought come from the words of a former head of school who said: “if you have two reasons to do something, there is a greater likelihood that you’ll get that thing done”. Following this though, the accompanying thought is: “try to align everything together, into a personal strategy”. It is this alignment that I’m trying to do, and this may well mean saying “no” to some things.

This post is a personal summary of a two day staff tutor conference that took place at Horwood House, Little Horwood between 5 and 6 December 2017. I’m blogging it for three reasons (1) so I can remember what CPD I’ve done when I have to do my appraisal, (2) it will help me to remember some of the important discussions that took place, and (3) it might be randomly helpful to someone!

Introduction

The conference began with a session by the outgoing associate dean of regions and nations, Steve Walker, a fellow staff tutor from the School of Computing and Communications. Steve presented a sobering description of some of the challenges that had to be addressed over the previous three years: the closure of the university regional centres, the introduction of a new tuition approach called the group tuition strategy (GTP), and the merger of faculties to create a new set of schools. A further change on the horizon is, of course, the potential introduction of a new associate lecturer contract.

Since the last staff tutor conference, I am now a home worker, there is a new student support team (SST) which is based in Manchester, and there is a venue management team that is based in Wales, and a system called the Learning Event Management system (LEM) which is (I understand) less than ideal. Also, AL development has been reconfigured (or, should I say, centralised) to create an entirely new way of working. 

Looking back to when I started in this role back in 2010, geography isn’t as important as it used to be. Support for students is instead organised in terms of curriculum teams as opposed to regional support teams. This means that the connections within schools have been strengthened and that there are more opportunities for activities such as associate lecturer development.

A number of important external drivers were emphasised:  the part time student market is receding due to government policy and fees, the student demographic is changing (they are getting younger), there are new entrants into the HE market and this means increased competition, the government has introduced the degree apprenticeship and the apprenticeship levy. There is also the Research Excellence Framework (REF), the Teaching Excellence Framework (TEF) and even talk about something called the Knowledge Exchange Framework (KEF). There has been a lot going on!

Student’s first transformation

During this period the university has defined a ‘student’s first strategy’ and has realised that it needs to address a potential income deficit due to the fall in part time numbers. In response, the ‘student’s first transformation’ programme has been set up. The programme aims to achieve a number of things: increase student retention, increase student progression and increase student satisfaction scores, whilst at the same time saving 100 million pounds (a figure that has been chosen by senior management).

To make these changes faculties have been asked to review their curriculum and other divisions of the university will be asked to review their working practices. These practices will be defined in something called a new ‘university operating model’ and the university will create a new ‘teaching framework’. Members of the university will be asked to participate in project groups and teams. Change, of course, has its own costs.

One phrase I regularly hear that is connected to the transformation programme is ‘digital by design’. A personal view is that I’m not quite sure exactly what this means; I know that students study in different ways using different technologies. Whilst I’m technologist, I’m a great believer in the usefulness of printed materials. On the point of ‘being digital’, the JISC Building digital capability project was mentioned (JISC).

During this part of the conference another point that I noted down was the term ‘sustainable academic communities’. Again, I’m not quite sure exactly what this refers to, but I did make a note of the principle that perhaps associate lecturers should be more involved and connected to schools. This discussion about associate lecturers takes us to the next part of the conference, which was about debating the purpose of tutorials.

Purpose of tutorials

Three staff tutors were asked to present perspectives about tutorials. The perspectives were: face to face tutorials, online tutorials, the use of pre-recorded tutorials and peer tutorials. I began by talking about face to face tutorials.

Face to face tutorials

Face to face classes, in my opinion, are the best and most effective way to teach. Here are some reasons why:

1. What really matters in education is, of course, people. Face to face tuition is all about putting student’s first for the simple reason that technology doesn’t get in the way.
   
2. Face to face means that learning is personalised to individual students or the group of students who are attending a tutorial.
   
3. Tutors can ‘see’ the effect of their actions; they can see the learning that takes place. To assess the effectiveness of their learning tutors can quickly ask questions, and this allows them to correct and develop understandings.
   
4. In adult learning, students can themselves become teachers. Students can arrive at class with significant and relevant real-world experience that can be discussed within the class. In some ways, students are their own resources. A skilled tutor can help to make connections between different students and topics.
   
5. Face to face gives tutors and teachers opportunities to innovate and to try new things out since they can more directly and readily understand what students feel comfortable with. Tutors can more easily facilitate role play events, facilitate discussions and create tasks that make use of the physical space of a tutorial room.
   
6. In a face to face session, a tutor can easily see if something is wrong; they can pick up on blank expressions and uncomfortable body language.
   
7. Face to face tutorials benefit all students, irrespective of whether students attend the sessions. The reason for this is simple: a tutor or teacher has to have a good command of their material. 
   
8. Following on from the above point, students will more readily question or challenge tutors during face to face sessions, helping themselves to understand how the teaching materials can be understood from different perspectives.
   
9. Face to face tutorials are real in a way that other methods are not; tutors can offer direct and personal encouragement to students.
   

I understand that there are some debates within the university about the cost of certain types of face to face tutorials, since they represent a significant proportion of the tuition budget. During this session I argued that, from an educational perspective, the university can’t afford not to provide face to face tuition.

Online tutorials

The second presentation about tuition events was by Diane Butler from the School of Health, Life and Chemical Sciences. Diane said that online tutorials, like face to face sessions, help students to understand module materials, allow students to interact socially with tutors and other students and to develop skills. Online sessions allow tutors to provide tailored support and, when travelling is difficult, help to alleviate a sense of isolation.

Diane also highlighted some of the challenges: online sessions can very lend themselves to be centred upon PowerPoint presentations, and given the complexity of driving online sessions, tutors can easily rely on the simplest tools. A common issue (and one that I regularly hear of when speaking to tutors) is that students are very reluctant to use microphones. It is a rare session that tutors present a series of activities; it can also be difficult to fully appreciate the learning needs of students.

An important point was that online tutorials don’t easily lend themselves to constructivist learning. During Diane’s presentation, I made the note: ‘we put our tutors in an environment where it makes it difficult’, and ‘[there may not be] much benefit to watching the recordings versus attending live’. A point was that perhaps the module team ought to spend more time creating recordings.

Another point was: ‘we place an unrealistic burden on our tutors to facilitate group work’, and that ‘we need to take online teaching to the next level’.

Pre-recorded tutorials

Hayley Ryder from the School of Mathematics and Statistics shared her experience of creating pre-recorded tutorials for a whole module cohort. An interesting point that Hayley made was ‘students like me to make mistakes’; this can share something about what really happens when people ‘do mathematics’. By exposing the challenges that accompanying studying maths, and demonstrating things that are ‘actually hard’ may encourage students to adopt a growth mindset: if the lecturers make mistakes, then it’s okay that I make them too!

My understanding was that the module team might have prepared a set of recordings once and then rolled them out for different presentations, but this wasn’t happens: new recordings are made every presentation; different people and different cohorts get stuck on different things. 

Hayley’s videos were fun and personable; her presentation got me thinking.

Peer tutorials

The final presentation was by Katherine Leys who is from the School of Life Health and Chemical Sciences. Peer tuition and peer support is mentioned in the new teaching framework. Peer tutorials is all about having students teach each other. Examples of these might be a group of students working together to plan an experiment in an online room or a forum. To facilitate these discussions, students might adopt roles, such as leader, deputy leader and so on, but tutors do need to mediate and some students will not be able to take part, perhaps due to the need to make reasonable adjustments to take account of disabilities or impairments.

Supporting staff tutor practice

Staff tutors offer support to associate lecturers, module teams and students. An interesting question to ask is: what can we do to help to support ourselves and fellow staff tutors? Maggie King, Staff Tutor in the School of Engineering and Innovation began this session by saying that the ‘changes to our jobs have been monumental’. In some respects, our job has changed from being a job that was primarily about people (and our academic discipline), to a job that is increasingly about administration.

We were asked a question: what could we do to make things better? Some suggestions included: the need for further staff tutor staff development, and opportunities for further sharing of practice, perhaps on a monthly basis or at different ‘regions’ or ‘regional areas’ (which could be, potentially, booked and organised through the venue management system). A personal reflection is that constant and effective communication between each other is the only way that we can continue to work well and understand the changes that are emerging through the university.

External Engagement

The title of this session was: ‘it’s good to get out’. It was facilitated by Matt Walkley, Staff Tutor from the School of Computing and Communications. Matt posed three questions that guided group discussions: what external engagement have you done? What external engagement should we be doing or be doing more of? And, what actions can we commit to?

A personal reflection was that I have always struggled to understand what was meant by the term ‘external engagement’; I was always under the impression that it had a more formal definition than it had. I was imagining external engagement as sets of targeted and strategic activities that can be used to help gain insights that could feed into module production, or to make contacts with individuals within organisations that could explicitly benefit from learning can be presented through OU programmes or modules. Although all these things do fall under the topic of external engagement, they are more directly aligned to the more formal concept of ‘knowledge exchange’.

External engagement can, as I understand it, broadly mean ‘getting out there’ to make contact with people to help us learn more about our subject and also to raise the profile of ourselves and the university. In some respects, being an external examiner and visiting the occasional British Computer Society event both represent different types of external engagement.

I learnt that external engagement is an official part of our contract. This means that we can allocate between 3 and 5 days into our work plan for external engagement activities (although there was some debate about exactly how many days we might have available). A point was if we can align our own personal and professional interests to that of our School and Faculty, then so much the better.

Another point I learnt was that each school has its own external engagement officer. I had to ask who the officer for my school was. This, in turn, made me realise that since staff tutors are scattered across the UK, the staff tutor body is at an advantage when it comes to engaging with national initiatives. A thought is that it is up to schools to develop and apply an interesting external engagement strategy; staff tutors are a resource that can be used, and used effectively (when we have the time, of course, and we’re generally pretty busy).

Improving retention and progression

The subtitle for this session that was held on the second day was: what can staff tutors do? Different staff tutors summarised different aspects of their experience. What follows is a very brief summary of each presentation.

U116 Environment: journeys through a changing world

Christine Pearson spoke about U116 and its ethos: the module assumes that students don’t know anything (that they are entirely new to the subject). I made a note that the students are changing, and also have a broad range of digital literacy skills. An interesting point was that some students were invited to take place in a focus group; a key point were that there was a need for induction or some kind of ‘fresher’s week’. Subsequently, a ‘getting started’ video was made, and students were sent a postcard to help them to get online. Another ‘bit’ was a numeracy podcast. An important point was that more and more students are doing concurrent study, which might be a side effect of the loan scheme.

Impact of presentation patterns

Bernie Clarke spoke about the impact of 22-week presentation patterns, where two modules are taught back to back (which is going to be the case for the new Computing modules TM111 and TM112). Using this presentation pattern, students gain credit at an earlier point in their studies. Another change is that in the engineering curriculum the teaching of mathematics is now done within the context of the subject, rather than students being asked to study a maths module that has been written by colleagues in a different school.

M140 early start initiative

Alison Bromley spoke about the effect of enabling students to start earlier on a module (another colleague, Carol Calvert gave a presentation about this same subject at a HEA Conference in April 2017). Over two hundred students accepted the opportunity to start M140 early. Those that started apparently really appreciated the opportunity for early tutor contact. I didn’t note down the detail of the impact, but I did note down that there was a difference between students who were studying for the very first time and students who had gained further experience through study.

Learning analytics and interventions

Nicolette Hapgood, chair of S111, reported that S111 applied an assessment approach that is known as ‘single component assessment’ (which is also going to be applied on TM111 and TM112), which means that the result is based on completing only assignments and not an end of module exam or assessment. Nicolette also described the availability of a data analytics tool that is now available to all module teams; this tool enables module teams to see differences in retention between different module presentations.

An important question to ask is: what is the main role of staff tutors when it comes to improving student retention and progression? An answer I’ve noted down is: our role is to support the associate lecturers who are closest to the students. We also represent an important link between the tutors and the module teams. Some other discussion points related to the knowledge management system that is used by the SST (which is used to offer study advice), the importance of reminding tutors about study support resources (there is an earlier blog about study skills resources), the importance of induction (which remains a mystery to me), and helping module teams to write and develop module materials. 

What will the REF mean to staff tutors?

I’ve forgotten who presented this final section of the conference, which was about the Research Excellence Framework.

There are some differences between the 2014 REF and the 2021 REF. One of the key differences lies in a statement that all staff who have significant time and resources to carry out research are to be submitted into the REF. There is a clear contractual difference between different categories of university staff: central academics are required to do research as a part of their contract, whereas staff tutors are ‘encouraged’. Subsequently, there is an ambiguity as to whether staff tutors will be included into the REF submission.

The reason why staff tutors are only ‘encouraged’ to do research is simple: workload. Time that we could have spent on research is spent supporting and developing associate lecturers and dealing with a whole host of administrative issues. Over the last year, it has more or less been a full time job keeping up with institutional changes, never mind doing institutional research.

My view is that there are two things that could be done to help to tick the research box: rather than doing discipline specific research, one possibility is to do scholarship and research into teaching and learning (since this fits closely with the role of a staff tutor), and secondly, if disciplinary research is important, another approach is to team up with central academic researchers. 

Reflections

This is the second or third staff tutor conference that I’ve attended. Typing everything up helped me to look back and to put a lot into perspective; a lot has changed. As mentioned at the start, the majority of the regional centres in England have closed and the way that tutorials are organised is now very different to how it was before. 

Put another way, I’m now doing a different job to what I was doing six years ago. I’m not going to pretend that homeworking is easy: it isn’t. 

This said, putting difficulties aside, there are some good things about this new way of working that many of us have had to embrace. A final thought is: it was really useful to spend time with so many colleagues; they are a pretty fabulous group of people to work with.