In November 2018, I had an opportunity to attend something that was called a ‘writing retreat’. The idea behind the event was simple; it was an opportunity to take a bit of time out from day to day activities and focus on writing up various bits of research that colleagues within the school had been working on. There was another reason for running the retreat: there would be a particular emphasis on writing papers that could be submitted to the 2021 REF (the Research Excellence Framework).
What follows is a brief summary of some of the points that were made during the introductory workshop which introduced the REF. Full acknowledgements are extended to Professor Jane Seale who facilitated this workshop. Many of the words here are directly from Jane’s presentation.
Introducing the REF
A key phrase that I’ve heard ever since I’ve been working at a university is: REF submission. A submission relates to particular subjects. Some universities may focus on some submission areas over others to play on their own strengths and weaknesses.
The OU makes a number of submissions, and one of the submission areas that I am connected with is education. This means that ‘education-like’ papers will be grouped together, submitted, and then assessed by an expert panel. The outcome will be a ‘research rating’, and this is directly linked to income that is received by the university. Simply put, the higher the rating, the more research income an institution receives.
I asked the question: “what does ‘education-like’ actually mean?” These will be papers that more than just describe something, like a system or a tool that has been designed (which is what some computing papers can be). Education research papers need to be firmly linked to an education context. They should also present a critical perspective on the literature, the work that was carried out, or both.
When?
The 2021 REF assesses research that has been published in the public domain that has been carried out between 1 January 2014 and 31 December 2020. Papers that are to be included are typically included within a university repository. (The OU has a repository called ORO).
What?
Every academic whose contract which has a significant research component should submit at least one paper, and there should be a REF average of 2.5 papers per academic (as far as I understand things!)
Publications should be of a 3* or a 4* quality. Three star papers means that a piece of research is of ‘international significance’. The education REF panel will accept different kinds of submissions, including journal papers, conference proceedings and book chapters, but there has been a historic bias towards journal papers for two reasons: they are peer reviewed, and are more readily cited. Books or book chapters should present summaries of research and shouldn’t be student focussed or summarise what is within a field.
In education, some journals contain practice papers or case studies. I noted down that education papers don’t have to be empirical to be considered for the REF. A paper might present a new practice or an innovation or development of an existing practice. A suggestion is to preface it with what you’re doing and why you’re doing something, and offer a thorough criticism of how and why something fits in with existing work. This means that it’s necessary to consider comparisons and contrasts. Descriptions are also necessary to contextualise the research, but the balance needs to be right: practice papers need to be generalisable.
Where?
A question to ask is: where should you publish? It was interesting to hear that the REF panel for Education isn’t particularly concerned with the impact of the journal where research is published; what matters is the research itself. Given that education research tends to be descriptive, a suggestion is to choose journals that have a generous word count. One such journal is: Open Learning, which is thoroughly excellent.
How?
How are submissions assessed? The submission chair will look at the title, abstract and reviewers and match a set of papers to an expert. Every paper will get read twice, and there will be some kind of process of random sampling. An interesting thought is: ‘don’t make the reviewers work too hard’, which is advice that I also give my students who are writing their end of module assessments or project dissertations. Institutions (including the OU) may having something called a ‘mock REF’, where they try to replicate the official REF submission to get a feel for the direction where the institution is heading.
REF criteria
Papers are judged on three key criteria: significance, originality and rigour; each of these criteria are equally important.
Significance: A paper or piece of research provides a valuable contribution to the field (this relates to the ‘so what?’ question about the purpose of the research). Also, how does the research moves the field forward. The contribution can be theoretical or empirical. A point to note is that a 3* paper contributes ideas that have a lasting influence. A 4* paper has a major influence on a field.
Originality: Is the research engaging with new or complex problems? Perhaps a paper might be using existing methods but in a new way, or challenging accepted wisdom. As a point of reference, a 2* paper contributes a small or incremental development, where as a 4* piece of research is something that is outstandingly novel in the development of concepts, techniques or outcomes.
Rigour: Papers and research offers intellectual precision or robustness of arguments. Some important questions are: is there rigour in the argument, the methods and the analysis? Also, can the readers trust the claims that are made when you ask the question, ‘how have you analysed your data?’ It is also important to include researcher reflexivity (question the role that the researcher had in the analysis of the research), and to critique the work, offering a summary of strengths and weaknesses. A 4* paper is one that is exceptionally rigorous, with the highest standards of intellectual precision.
Reflections
I only have a small amount of time in my staff tutor contract to carry research; I try to do what I can when I can, but I very often get entangled in tutor and student support issues which take me away from exploring really interesting questions and topics. Plus, increasingly, I’ve been playing a role in AL professional development. Carving out time for research is difficult balancing act due to all these competing demands.
When I started the retreat, I planned to write about some of the tutorial observation research that I’ve been doing and reacquaint myself with some of the papers that I had uncovered whilst doing this research. The introduction to the event gave me some really needed context, in terms of what a very good paper actually looks like. It also gave me a bit of direction in terms of what I actually needed to do.
One of the things that I did during the retreat was to consolidate different bits of research into a single document that forms the basis of a paper. I now have a clear and distinct structure. What I need to do now is to do some further close reading of the references to more directly position it within the literature, add a more critical twist to the analysis to broaden its appeal, and to do quite a bit more editing.