OU blog

Personal Blogs

Christopher Douce

TM470 Considering LSEPI (again)

Visible to anyone in the world
Edited by Christopher Douce, Friday, 4 Oct 2024, 16:03

This post follows on from an earlier post: TM470 Considering LSEPI. The difference between this blog and the earlier one is that this article places more emphasis on the ethics bit, specifically, how to treat participants ethically.

Before looking at anything specific, it’s useful to remind ourselves of what learning outcomes in TM470 relate to ethics:

LO10. Identify and address the legal, social, ethical and professional issues (LSEPIs) and the equality, diversity and inclusion (EDI) concerns that may arise during the development and use of computing and IT systems.

To gain a distinction for this learning outcome, you need to provide evidence to show that you have “comprehensively identified the relevant LSEPIs and EDI concerns arising during development and use and modified their project work to take these into account and behaved professionally in all aspects of [your] project work.”

At the beginning of your project it is very likely that you will be gathering requirements from stakeholders. During the middle of your project, you may well ask stakeholders for opinions about your intermediate designs, or emerging solutions; maybe asking their opinions about some prototypes. Towards the end of your project, you may (or may not) choose to carry out an evaluation. Your final project evaluation may involve stakeholders, who might be potential users of whatever product or system you have created.

Treating participants ethically

Given that you are likely to use participants throughout your project, what do you need to ensure they are treated in an ethical way?

A fellow TM470 tutor, Kawal Banga, offers the following useful summary: “you will need to consider how you are collecting data, where you are storing it, what stakeholder contact details you are storing, how you are ensuring anonymity and confidentiality, what will happen with the data on completion of the project, etc.“

Hold onto these points. 

Planning your project

TMA 1 is all about identifying a project, describing its aims, creating a plan and sharing it with your tutor. TMA 2 is all about showing that you have made some progress on your project and beginning to write about the ethical issues.

TMA01 states that:

“You will be considering in detail any legal, social, ethical and professional issues relating to your choice in TMA 02, but at this point you should consider whether these are likely to be serious enough to mean your project choice is inappropriate.”

It directs you to read some resources that have been prepared by the module team.

Before you begin writing your first TMA, do make sure you find the time to have a discussion with your tutor about your project and its aims.

In many cases, one of the first activities that you will carry out will be to establish requirements which may mean that you will need to talk to stakeholders. A stakeholder is, of course, anyone who has a vested interest in your project, or will be affected by its implementation or creation. Before you speak with anyone, you need to consider (as Kawal pointed out) how to collect data, what data you are collecting, and where you might be storing it.

TMA02 states that:

“If your project involves you working with human participants, you should include, as an appendix, your TMA02 LSEPI Form, found in the LSEPI Templates folder.” 

The point being made here is that you may need to address ethical issues before you get to your second TMA. Gathering requirements may mean working with people.

Working with stakeholders

When gathering requirements, if you need to consult people, it is important that you seek permission from those who you speak with and their line managers. Here is a suggestion about different resources that you should consider preparing before you interview anyone:

A project information sheet. This could be a single printed page, which you could then read before you go ahead with any data collection.

A consent form. This form is used to secure permission to gather data, and also to store data. More information about storing data can be seen in a related blog: Writing successful data management plans.

A set of interview questions, a set of survey questions and forms that can be used to gather responses.

Do consider sharing each of these with your tutor; they may well have some good ideas about how they might be improved.

Evidencing an ethical approach

When your examiner reads your project report, they will look for a description of what you have done and evidence that shows you have done what you have said you have done. Using a concept from creative writing, it is important to show the reader, rather than to tell the reader. 

You can show the examiner you have adopted an ethical approach by sharing evidence. You might, for example, share the following appendices:

  • Provide a copy of a project information sheet.
  • Provide samples of signed consent forms. You don’t need to provide copies of all the signed consent forms; one of each broad group of stakeholders will be enough. Make sure you hide any names and signatures. There is no reason why the examiner needs to see these.
  • Provide copies of any interview scripts or data collection forms.

Each appendix should, of course, be referenced within the body of your report.

Resources

The module website contains a number of helpful resources and pages. In particular, within the Legal, Social, Ethical and Professional issues resource, the following two sections are particularly useful:

  • Working with human participants
  • Appendix A Guidelines for conducting research with human participants

There is also a folder called LSEPI templates which can be found within the study materials section. At the time of writing, this folder contains the following files:

  • TMA02 LSEPI Form
  • consent-form-template
  • participant-information-sheet-template
  • EMA LSEPI Form

Do take the time to have a look at each of these files, and reflect on how they might be used within your project. 

When you submit your EMA you need to include a completed copy of the EMA LSEPI form as an appendix (which is something that be easily forgotten).

These above points offer some very practical advice about what you need to do to provide evidence of working with participants. This is, arguably, a very narrow treatment of the connection between your project and ethics. 

Thinking in an ethical way means that you need to consider the impact on any digital tool or product. If you are interested what this means, a good starting point is Carissa Weitz’s Oxford Handbook of Digital Ethics (Oxford University press). You might find a chapter in this handbook that connects with the aim of your project. 

There is a requirements that your write about legal, social and ethical issues within your TMA 2 submission. If you're unsure about what this mean, or how to begin, a good bit of advice is, of course, find some time to speak with your tutor.

Acknowledgements

Many thanks to Kawal Banga and the TM470 module team.

Permalink Add your comment
Share post
Christopher Douce

Writing successful data management plans

Visible to anyone in the world
Edited by Christopher Douce, Wednesday, 2 Oct 2024, 14:19

What follows is a set of notes from a professional development event by the library research support team, which took place on 8 Feb 2024 (which now feels like a long time ago).

This even was of interest for a number of reasons: I supervise some EdD research projects, I’ve been successful in gaining a small amount of seed funding, and I thought it might be useful in my role as an undergraduate project supervisor.

What is a data management plan?

A data management plan (DMP) is written at the start of the project, something that funders need to see. It describes what data you’re doing to collect, how data is going to be stored and secured, how will access be granted.

The exact requirements of a DMP may very between different funders. It is especially important if your research data is going to be uploaded into an archive. It is probably a good idea to write one even if one isn’t explicitly needed. Knowing how to do prepare a good DMP is an important research skill. Your plan can, of course, evolve as your project evolves. 

Every project will have different data requirements. Will your project gather data from interviews, focus groups, or surveys? Different types of data may have different types of management requirements. What is the volume of the data, i.e. how much? How will data be collected? Also, where will your data come from? Will it come from other sources, such as databases or third parties? To help with the preparation of a plan, there is a tool called DMP online, provided by the library.

During this session we were led to an activity and asked the question: what do you think are your main data management challenges for your research project?

What follows are some headings which highlight what elements of data collection, storage and management which need to considered.

Storage and backup

Effective backups prevent accidental losses and deletions. Even if your data is backed up, you need to ensure that backups have integrity - in the sense that if you need to access or use them, you can do so without encountering difficulties or data loss.

When thinking storage and backup consider the size and complexity of your data, and where research is carried out. If you work in “the field” (amongst participants), consider the physical security of your data and find ways to backup your data. You also need to consider how to secure your data if you are working with others.

A practical suggestion for collaborative working was to make use of tools such as Teams or Sharepoint. For individual researchers, an alternative suggestion was to make use of OneDrive (which is a part of the university infrastructure). Keep a backup copy on an external device (and make sure it is encrypted).

Ethical and legal issues

Ethics permeate research. If you’re working with human research participants on research project within the boundary of the university, such as an EdD or PhD project, or a funded research project, you need to register your project with the university’s Human Research Ethics Committee (HREC). A part of this you will need to complete a risk checklist.

You must tell your participants what will happen with the data that is collected as a part of the research, not only during the research, but also after the research project. Specifically, you should tell participants where results may be published and whether dataset may be available afterwards.

Data protection is important. If you collect personal information as a part of your research, you must add details about your project to the university Information Asset Register (IAR). 

Selection and preservation

An important question to be asked is: what data needs to be kept, and what data needs to be destroyed? The data protection legislation is important. You should ensure that any personal data that is collected is deleted at the end of a project. Some documents, such as consent forms, need to be maintained for a considerable amount of time after the project has ended (up to 10 years). How are these records going to be kept? The reason for maintaining these records is to ensure protection of both the participant and the researcher.

Data sharing

When a project has been completed, there may be a necessity to share the dataset with others. One of the reasons for wanting to do that is that fellow researchers might want to study your data to not only confirm your findings or to challenge your conclusions, but also to interrogate your dataset in different ways. The data that you collect (and share)  might be useful for the study of different research questions.

Your data should be shared using a trusted repository that is “as open as possible, but as closed as necessary”.  Sometimes, data can be shared through a research funder repository, a discipline specific repository, or an institutional repository. The OU has a repository called ORDO: OU’s Research Data repository.

An interesting resource that was mentioned was a site called re3data, which is short for Registry of Research Data Repositories. It’s also useful to note that repositories often make use of licences. Different repositories will have different licences.

Responsibilities and resources

Finally, there’s the important question of responsibilities. A key question is: who is responsible for the data management plan? In a plan, it would be useful to identify who is responsible for what data. Who, for example, be responsible for managing the upload of data to a repository. 

Reflections

A key point that underpins this post is: do contact the library team for help and support.

I have two reflections to share. The first is that the way that research data is treated has significantly changed from when I started carrying out research, and this is a good thing. Whilst all this can be perceived as an annoying administrative burden, it necessarily helpful to spell out how data is used. In turn, this can open up possibilities in terms of how data can be used by other researchers. A detailed plan can also offer helpful reassurance to participants.

The second reflection relates to my role of a TM470 project tutor. Undergraduate projects sometime requires students to carry out research that involves people – for example, potential users of software systems. Since undergraduate projects sit outside the formal university (HREC) ethics processes, students don’t have to create a data management plan, there is benefit in considering how data is collected, used, and stored. The reason is, of course, these issues speak to the ethical issues that are important to every project. A practical recommendation for TM470 students is: if you collect data, create a short appendix (it could be only one or two pages in length) that summarises your data management plan.

Acknowledgements

Many thanks to the library research support team for running this event. The headings for this blog have been derived directly from their presentation.

Permalink Add your comment
Share post
Christopher Douce

Ethics support for projects: HREC and SRPP

Visible to anyone in the world

On 24 January 2024 I attended a bit of a professional development session that shared an overview of two important points, and organisational units, which relates to research and research ethics. The session was facilitated by Alison Fox, Steven Bond, who was from the data protection team, and Bart Gamber, who was from the Student Research Project Panel (SRPP).

Introducing HREC

Research ethics is important. To help OU researchers and doctoral students, there is something called Human Research Ethics Committee (HREC) which provides services to researchers, and provides an ethical approval mechanism. 

Ethical approval needs to be taken really seriously for a number of reasons. Approval ensures the safety of researchers and safety of participants. A further check of your research aims can also improve the quality of your research. My argument is that articulating your research to others can only improve its clarity and purpose. Also, when it comes to publishing your research, some journals will insist on a detailed summary of how you have approached ethics, and some journals will directly ask for evidence of whether you have gained formal ethical approval as a part of a study.

There are, however, some projects that might not need HREC review or approval, such as an evaluative study that takes place within a course, or a study which is feeding back into a university service, for example. Also, research that is designed to inform a work practice, market research, or research with data that has already been collected (where that data set has been gathered through a process which ha been subject to its own ethical approval).

HREC offers links to other teams and groups that can offer help and advice, such as the library and information security teams (if not using core university systems). You might, for example, gather a lot of data. If you think that other researchers might want to use your data, the library will be able to offer advice and guidance about how (and where) to make that data available. Also, knowing how to secure your data is also an important part of the ethics process.

Submissions are made to HREC through something called the ethical review manager tool (which reminds me of the name of another tool: the postgraduate research manager tool).

If anyone has any questions about the process, the facilitators encouraged anyone to get in contact. To help everyone navigate through all these practical questions and challenges, it was interesting (and useful) to learn that HREC run research monthly drop in sessions, which typically take place on the 3rd Tuesday of the month.

Introducing SRPP

A related unit goes by the abbreviation, SRPP, which is short for the Student Research Project Panel. The way that I understand it, SRPP has a couple of interconnected aims. It can help to identify potential students who might be able to participate in research. Equally, it is there to make sure that students are not ‘over-research’, which means ‘contacted unnecessarily regularly’.

Like HREC, submissions to SRPP are made through a form. Some practical tips shared were: plan early, and apply early. These things can take a bit of time.

Resources

Just before the session, a PowerPoint resource was shared. After the session, I noticed that it was packed filled with useful links, many of which can be accessed externally. Here is a summary of what I took to be the most important links:

Reflections

A useful session! It was also one that was very timely since I have been awarded a small amount of funding to carry out a pilot project to explore the connection between stories, storytelling, and the professional identity of software engineers. My next action is to attend one of those drop-in sessions, and then to review all the forms. Whilst I do usually hate form filling, I do recognise that these forms relate to a process that is there to protect everyone.

Acknowledgements

Many thanks to Alison Fox, her co-facilitators, and everyone who is involved with the HREC and SRPP units.

Permalink Add your comment
Share post
Christopher Douce

TM470 Considering LSEPI

Visible to anyone in the world
Edited by Christopher Douce, Wednesday, 24 Apr 2024, 10:44

In TM470 LSEPI is an abbreviation for Legal Social Ethical and Professional Issues. A good TM470 project report should clearly address these issues to show the examiner that you have thought about how these issues have impacted on your project, and what you have done to take these into account.

LSEP issues are increasingly important in computing due to the increasing impact that computing and IT has within society. When speaking with students I often a recent example: the Volkswagen emissions scandal. In this case, there are clear environmental impacts and legal implications. It is also clear that both the engineers and leaders have to make ethical decisions.

In TM470, LSEP issues are assessed through the following learning outcome: “LO10. Identify and address the legal, social, ethical and professional issues (LSEPIs) and the equality, diversity and inclusion (EDI) concerns that may arise during the development and use of computing and IT systems.” In the marking of the EMA, this learning outcome is assessed with LO2, which is all about the aims and goals of your project.  When just looking at the number of learning outcomes, and the marks available, the LSEPI section could account for 10 marks.

To gain a top score for this learning outcome a student: “has comprehensively identified the relevant LSEPIs and EDI concerns arising during development and use and modified their project work to take these into account and behaved professionally in all aspects of their project work”. EDI being an abbreviation for equality, diversity and inclusion.

Given the importance of both LSEP and EDI, a suggestion is to include it as a top level section in your report, just before the literature review section. The justification for this is that if you identify some issues that need to be explored in greater depth, you can then go onto provide evidence of your reading.

Module materials

At the time of writing, it takes a bit of digging to find two documents that relate to both LSEP and EDI issues. From the module website, click on the Resources heading, and then click on the Study materials section.

The LSEP document contains the following key headings: working with stakeholders, working with human participants, and asking the right questions. Do review the materials that are presented under these headings and review Appendix A Guidelines for conducting research with human participants. Related to these are two template documents: a sample consent form, and a participant information sheet.

Informed consent is the process through which researchers share the aims and purpose of their research with participants, and gain their approval that they are happy to participate in a study. The accompanying information sheet is designed to offer further information under a set of familiar headings.

When working with participants, I always remember two points. The first is that participants are at liberty to leave a study at any point. The second point is related: the participants are always more important than the research that is being carried out.

The equality, diversity and inclusion section addresses “why equality, diversity and inclusion are relevant to computing and IT professionals”, introduces the concept of protected characteristics, and “unconscious bias is and how it might affect your practice as a computing and IT professional” and what mitigations might be adopted (TM470 module materials).

EDI relates to people, and differences between people, irrespective of whether they are perceived or due to physical, cognitive or sensory impairments. Since Computing and IT products are, ultimately, used by people, it is necessary to consider EDI issues. If you design an app or a website, your product should be accessible to the widest possible group of users. The motivations for doing this are twofold: firstly, there is a legal obligation to ensure that products and systems are accessible under the Equality Act, and secondly, all users are potential customers. If a product isn’t accessible or perceived negatively, a consumer might choose another service that has a more accessible, usable, or appealing interface.

Looking at this issue from a slightly different perspective, if your project uses artificial intelligence or machine learning, it is necessary to question the extent to which biases might exist within either data that informs your project, and the extent to which bias might be potentially reinforced, or even magnified.

Questions to ask

As highlighted earlier, the LSEP materials contains a section that has the title: asking the right questions. 

Go through each of these questions in turn. 

When working through these questions, do think about the stakeholders who are involved with your project. A stakeholder can be thought of anyone who is affected by your project, either directly or indirectly. Ask yourself questions about what data might need to be held and collected, and what bits of legislation might play and impact if you were ever to deploy your project. The Equality Act was mentioned above. You might want to also consider data protection and computer misuse legislation.

If your project is a research report it is important to ask: what might be the impact of my report? If something is discovered by the report, what might be the impact of disclosing the results, or not disclosing the results? The point here is that it is important to go further than just the immediate project, but also to consider wider and broader impacts.

Differences between student projects and university projects

Before university staff can carry out research that involved human participants, they must submit project proposals through a formal ethics panel. The aim of this panel is to make sure that researchers have carefully considered everything, and any potential risks to all participants (and to the university) have been mitigated.

Unlike official university projects, undergraduate and postgraduate projects are not required to go through such a rigorous process. Rather than having an ethics panel and a lot of electronic paperwork to complete, students should think of their tutor or project supervisor as a mini ethics panel.

Interacting with your tutor whilst considering your LSEP and EDI issues should be thought of as a useful and necessary part of your project. Your tutor will be able to offer some thoughts about what needs to be considered. Plus, interactions with your tutor or supervisor can be documented in an appendix of your final reports.

Further resources

A lot of good resources about ethics are available, and some of these resources are mentioned in the module materials. Here are a collection of links that might be useful:

For those that find this subject really interesting, there is a whole suggested curriculum about Society, Ethics and Professionalism on the ACM website.

Going through the ethics bit of TM470 gives you a taste of what university researchers have to go through when they plan and design studies that involve human participants. More information about what goes on behind the scenes at the OU is presented through Ethics support for projects: Which studies need review, by whom and why? (OU blog)

Reflections

I find ethics a fascinating subject. In computing it comes into play more than you might initially expect since computing touches on so many different areas of human activity. Rather than being a subject that was once on the periphery of the discipline, I now see it as a topic that has moved to the centre. It is an important and necessary part of becoming a computing professional.

It is also interesting to reflect on how ethics has developed since I was a graduate student. There is now a lot more that has to be done, but this isn’t a bad thing. Additional scrutiny along the way helps researchers to carry out better research. For TM470 students, my key bit of advice is: speak with your tutor; they are your own personal ethics panel.

Permalink Add your comment
Share post
Christopher Douce

Ethics support for projects: Which studies need review, by whom and why?

Visible to anyone in the world

On 15 June 2023, I went to another academic professional event. This one was all about ethics and ethical approval. In some respects, it directly follows on from the previous blog, which was all about how to write a 4* research paper.

The session was facilitated by Alison Fox, Chair of Human Research Ethics Committee, and Bart Gamber who is a Student Research Manager for the student research project panel (SRPP). Alison is based in ECYS, the school of Education Childhood Youth and Sport.

The aim of the session was to present an overview of ethics mechanisms for researchers who make use of human participants, and to share something about who, how and why things work. Another aim was to share something about the types of studies that may require ethical approval.

Introducing HREC

HREC is an abbreviation for the Human Research Ethics Committee. We were shown something called the Research Ethics Landing Page, which offered a whole set of links which relate to the different parts of the university which support both ethics and research. An important element of this was a flowchart which highlights what needs ethical review.

Different types of research were mentioned: evaluation research (of services, procedures and policies).; audit research; market research; research. Each of these might necessitate ethical consideration. If OU students are involved with any type of study, this necessitates a referral to the SRPP team. Also, if a study involves more than more than 30 members of staff, researchers must consult the Staff Survey Project Panel (SSPP).

An important point was made about the use of data. Audio recordings that are collected during a study is considered to be personal data, which means that it must be manged carefully. This leads us to consider the different places of support that we may need to draw upon. 

The teams we might consult, include:

  • Data projection team; to gain advice about how to record information assets.
  • Information security team; if wanted to use particular platforms.
  • Library research support team; to consult about how data is stored, how it can be retrieved.

A tip: apply to each of these in parallel.

Two ways to apply to HREC were highlighted. The first was a checklist, which is submitted to the committee. This has a 7 day turnaround time. If your project is a high risk application, a full HREC application can be submitted, then there is a longer 3 week turnaround since it is submitted to two reviewers.

Introducing SRPP

Next up was a summary of the SRPP, which is an abbreviation of the Student Research Project Panel. SRPP is the mechanism used by researchers to gather names and identifies of students you might wish to invite as participants to different kinds of studies. SRPP is important. It is a gatekeeper; you have to go through it to access any of the OU’s students. The reason why it exists is simple: it presents the same students from being approached time and time again. It’s function is pretty simple too: you give it a criteria, and it gives you back identities of those you are allowed to contact.

We were introduced to the SRPP internal website, and shown an online application form. This form contains section where you describe the methodology of your study, a description of the sample of students you wish to involve, a confirmation that you have engaged with HREC, and have taken into account information security, and data protection. 

As the site SRPP site develops, there is the intention of providing examples of previous submissions.

Tip tips

Towards the end of the session, we were given a number of top tips. What follows is an abridged (and edited) version:

  • Start early and expect a dialogue with the teams, and build this into any timescales and plans.
  • Contact teams before submitting applications to get advice.
  • If collecting personal information, you need an asset register.
  • Consider everything from a participants’ perspective; think about their concerns.
  • Consider what data is being collected, how it will be held, destroyed and how consent might be withdrawn.

If you are getting started with a design, visit the research journey landing page for help and guidance.

Undergraduate and postgraduate projects

During this session, then following important point was made: everything described earlier relates to research that is carried out either by doctoral students, or academic staff. 

Students who are carrying out projects that are a part of taught undergraduate and postgraduate qualifications do not need to make HREC submissions or interact with any of the organisational units that were mentioned earlier. This is all to do with insurance.

Students who are completing undergraduate projects, or are writing dissertations that may necessitate carrying out a small study must follow the ethical guidance that is presented within the modules that they are studying. 

Reflections

It’s been a while since I’ve been to a session about research ethics. It is interesting to see how things have developed. Notably, there is a lot more emphasis on securing and holding of data. This is, of course, a very welcome development. The advice “make sure you speak with each unit” early on in the research journey is good sound advice.

One the topic of a theoretical research journal, we were also introduced to a resource that was called “Research Journey”. This resource, a web page, was all about educational research with students, which is where SRPP comes in.

An interesting point that I have noted was: most scholarship projects won’t need to go down the road of HREC, but some academic publishers do require evidence of engagement with ethics panels before research is published. As a reviewer, I certainly look to see the extent to which ethics is mentioned within the articles that are submitted.

The themes and issues that have been mentioned have wider relevance. It was mentioned that both undergraduate and postgraduate students don’t need to submit anything to HREC. Being a tutor on an undergraduate module, where students are asked to consider ethics from a number of different perspectives has made me reflect that perhaps I ought to be sharing a bit more about the kinds of ethical issues they should consider. I feel another blog about TM470 might be needed.

Permalink Add your comment
Share post
Christopher Douce

Reviewing an academic paper for Open Learning

Visible to anyone in the world

One of the tasks I have to do pretty regularly is to review academic papers for a journal called Open Learning, which I help to co-edit. 

This blog post is intended as a summary of my own thoughts about how I approach reviewing. This post may be useful for reviewers who are new to reviewing papers for journals not too dissimilar to Open Learning.

The blog is split into three parts. The first part is about how I approach the reading (and interrogating) of a journal submission. In this first bit there are some short cuts which I tend to apply to get a feel for a paper.

The second part is about how I approach the offering of feedback to authors. The overall aim is, of course, to try to help the author of a submission to write a better paper. For this part, I should acknowledge some of the ideas of Simon Bell, a former editor of Open Learning, who put in place a really nice framework.

The final part offers an ethical perspective. This is discussed in three different ways; the ethical responsibilities of the reviewer, ethical responsibilities of an author, and the ethical perspective that must be presented through a paper.

The blog post concludes by sharing some additional resources and sharing some further reflections about the role of the reviewer.

Before beginning, an important question to ask is: why should I review? There are a few answers to this. One reason is that is gives you insight into the peer review process. It also enables you to catch sight of the kinds of papers and research that relates to a field or discipline. Also, in some respects, academics serve the discipline that they study and teach; reviewing for journals can be thought of an extension of that service. Another reason is the practice of reviewing and writing reviews develops your critical perspective. Finally, reviewing is a way to gain academic kudos and experience. If you review for a journal, this is something that you can add to your academic CV.

Reading an academic paper

One of the first things I try to do is to get a feel for the paper as a whole. 

Getting a feel for the paper

A key question to have in mind is: what kind of paper is this?

I begin with the title, then the abstract, then the introduction, and then I immediately go to the references section. My justification for this is: if I recognise some of the references, then I may be able to get a quick (and rough) understanding of the type of research that is being presented. If I don’t recognise any of the papers, then I’ll clearly have to work a lot harder than I would if some of the papers were familiar to me.

Looking at references

Whilst I’m in the references section, I look to see whether a paper has referenced any other articles from the journal that it has been submitted to. If it hasn’t referenced any papers from within the journal, this makes me ask myself the question: is this paper appropriate for the journal?

There are two reasons why references from within the paper is important. Firstly, clearly referencing from within the journal shows that the research is placed amongst and next to existing research. This means that it is likely to be following and connected to existing debates and topics. Secondly, referencing popular papers from within the journal you are submitting to is a good strategy; it enables your work to be more easily discovered by researchers. The reason for this is that many journals allow researchers to follow links between different papers. 

Gaining a critical perspective

The next thing I would do is have a quick look through all the different sections. There are always some key headings that I look for: a section that describe methods, a results section, a discussion section and then a conclusion. If any of these are missing, I would certainly be giving the paper a closer look, and asking why the article wasn’t using these headings.

Checking out the detail

When looking through all the different sections, I would also keep an eye out for any figures or graphs. I would typically ask myself a couple of things. One question would be whether there were any figures or images that were presented in colour. The reason for this is simple: printed versions of the journal are still (currently) important. Although it is unlikely that a researcher might handle a physical copy of an issue in a university library, they may well download a PDF and print a copy out. Secondly, if there were graphs, I would check to see if the axes and titles made sense.

When I’m through with looking at these aspects, I might jump from the introduction to the conclusion. Is there a consistent message between the two sections? Doing this should (ideally) give me a good feel for what the paper is all about.  

The next bit is to read through the methods section to find whether there is a clear description of the research questions, before heading onto the methods section. A key question to ask is: “does the approach make any sense?” Another question to ask is: “is there sufficient detail to enable me to get a view about the methods?”

I must confess to being more confident with assessing qualitative papers than I am with quantitative papers. If I feel that I’m not able to make sense a paper, or feel that I don’t have the appropriate expertise to make a judgement or a proper academic assessment of a submission, I tell editor to make them aware of this. This is something that I pick up on later in the ethics section.

Commenting on a paper

When my former colleague Simon Bell started as a co-editor of Open Learning, he requested that all reviewers should be sent some guidelines.

A version of his guidelines have also been published in the System Practice and Action Research Journal (Bell and Flood, 2011). Essentially, they are a set of constructive directives that are intended to create what we called “the spirit of reviewing”. 

For sake of brevity and this blog, I summarise (and paraphrase) the directives (or guidelines) as follows:

  • Always be honest but temper honesty with kindness. Ask the question: “How would I feel if I received this review?”
  • Be constructive. Articles have been developed over time and should be read with sympathy and honour. 
  • Be fair. Always comment on what I liked as well as what parts of a paper I might have had problems with.
  • Be humble and say when I do not understand something; do not present myself as a world authority on a subject. 
  • Consider myself as a co-worker who is trying to contribute to a wiser and more exciting script. 
  • To help both the editor and the author, indicate if I like the text, say whether I would publish it, and highlight what changes could be made to make the text more enjoyable and if I think the author needs to “adapt/change/re-assess the text in some more challenging manner”.

Even before I had been introduced to Simon’s guidelines, I had implicitly devised a way of providing my own feedback to authors. The approach that I take may be familiar with colleagues:

  • Highlight what you think is good about the paper and acknowledge the work that has gone into producing it.
  • Highlight areas that you might have concerns with. Explain what could be improved, giving a suggestion about how it might be improved, and describe why these improvements are important.
  • Ask whether other papers may be useful for the researcher; offer them help and pointers where you think it is appropriate.
  • Be practical; if you feel that there is a lot wrong with the paper, highlight only three points. A thought is to say something about the content, say something about the structure, and say something about how it fits in with the discipline or the journal; this will help the editor too.
  • End on a positive note. 

An ethical perspective

It is really important that reviewers carry out reviews in an ethical way. 

There are three different perspectives that need to be kept in mind: the ethical practice of the reviewer, the ethical practice of the author of an article, and the ethical practices that are followed within an article. Each of these perspectives are covered in turn.

The reviewer

Reviewers are in a position of power and privilege; their comments can influence whether an article is published. Reviewers must bear in mind the following perspectives:

Impartiality: Reviewers should be impartial. This means that they should be aware of potential biases they may have about any aspect of a submission. A reviewer should not be familiar with the work that is being reviewed.

Expertise: Reviewers should be confident in their assessment of a paper. If they lack sufficient knowledge or expertise to make a judgement, they should either state this in the review, or let an editor know that they do not have sufficient expertise to carry out a review.

Integrity: Through their articles, authors may share new ideas. These ideas are being shared, in confidence, with reviewers. Any interesting and novel research directions that are suggested through an article should remain entirely with the author of an article. Reviewers should not directly draw on or build upon the work of papers that they review.

The author

Authors of papers must not use text, data or images from unattributed sources. Quotations that are used within a paper must be correctly presented; the source of an author should be mentioned, along with accompanying page numbers. If sources are not referenced correctly and fully, authors run the risk of being accused of plagiarism, which is a term that can be applied to not only intentional copying, but also inadvertent copying.

Authors should also be mindful of a concept known as self-plagarism. This is where an author of an article might make use of their own words which might have been used (and published in) other articles. This can occur, for example, if an author writes a paper that describes their doctoral research. The words they use within a doctoral thesis must be substantially different to the words used within an academic article. The exception to this is when authors deliberately quote their earlier work, and earlier publications.

One of the ethical responsibilities of a reviewer is to make a confident judgement that an author’s submission is their own, and to the best of their knowledge, isn’t using the words of other researchers. Reviewers should also let an editor know if they find that a very similar version of a submission has been published in another journal.

In many cases, the journal editor and publisher will also be able to make use of specialist tools to carry out further checks to ensure the originality of submissions.

The research

Since Open Learning publishes education research, many research articles make use of human participants. Whenever human participants are used, reviewers must ensure that there is sufficient evidence within a paper that suggests that research is carried out in an ethical way.

Simply put, participants must be clearly told about the aims of the research they are a part of, and they should be free to leave, or to decline participation in a study at any point. This view should be clearly presented within a section that describes research methods or procedures. Reviewers should feel free to provide comments if they find that a researcher has not provided sufficient description to enable them to decide whether an ethical approach to research has been adopted.

Research ethics is a subject in its own right, and one that has a rich history. Whilst the journal is not expecting reviewers to be an expert in all aspect of research ethics, there is an expectation that reviewers always ask the question: “has this research been carried out in an ethical way?”. More information about research ethics can be found through the British Educational Research Association Ethical Guidelines for Educational Research (BERA website).

Additional resources

The publisher of Open Learning has provided a set of reviewer guidelines (Taylor and Francis website) which may also be useful.

Taylor and Frances have also published some information about their editorial policies and plagiarism (Taylor and Francis website) which may be helpful for both authors and reviewers.

Reflections

Reviewing can be an interesting and rewarding process. Although I spend most of my time reviewing papers for Open Learning, I have also reviewed papers for conferences, workshops and other journals. One of the benefits of reviewing is that is helps to maintain a connection with a discipline. It is rewarding to see how authors respond to comments, and how reviewers can directly (but implicitly) contribute to the continuing professional development of fellow academics and researchers. I would also emphasise that is isn’t necessarily something that is easy; sometimes there are some papers that are difficult to review, and the accompanying comments can be difficult to write.

To conclude, here is a concise summary of what I perceive to be the benefits of being a reviewer:

  • It helps to maintain a link with a discipline.
  • It provides a way to give academic service to a discipline, which can be highlighted on an academic CV or resume.
  • It helps to develop skills to critically assess academic writing.
  • The process of providing feedback helps to develop (and maintain) critical writing skills.
  • It helps to further understand the peer review process.
Permalink Add your comment
Share post

This blog might contain posts that are only visible to logged-in users, or where only logged-in users can comment. If you have an account on the system, please log in for full access.

Total visits to this blog: 2266878