Personal Blogs

On the evening of 13 June 2023, I attended a university wide session about critical thinking and writing, which was delivered as a part of the Student Hub Live programme. The event was facilitated by Margaret Greenhall, study skills specialist, and OU tutor in Science, and was supported by another tutor, Nicky Mee. An edited recording to this session is available through the Student Hub Live website.

Margaret presented what could be described as a “a buffet of ideas” to help students to work through ideas and concepts they are presented with during their studies. Whilst it is intended to be a starting point, the buffet presents some really useful tools that could be used by anyone who is approaching the study of a new subject. It may also be useful for anyone who needs to do a review of a topic, or to carry out a literature review.

We were given a lot of notes to download, which shared some of the key concepts which I hope to summarise through this short blog.

What follows is my own notes from the session, which are presented, of course, from my own perspective, complete with my own understandings (and misunderstandings!)

What does critical thinking mean to you?

We were asked a question, and invited to respond by giving answers in a text box. We were then led towards a suggestion, that perhaps critical thinking could be thought of a pyramid, or hierarchy, which has a number of levels, such as:

1. Content
2. Validity
3. Relevance
4. Criteria
5. Evaluation

These concepts could be unpacked further, by asking accompanying questions, such as:

1. What? What is the content?
2. Who? Who wrote it?
3. Why? Why is it relevant to you and the problem?
4. What? What is important with what you have found?
5. How? How does it connect to other things and how is it useful?

In the following sections, I summarise how each of these key points were broken down.

Content: What?

A question I noted down during the presentation as: is this all about reading the information? We were also asked a question during the session: what do you do before you read the course materials? This question led to a short activity, where we were shown sections of text.

There was an important point which was made here, which is: critical thinking starts before you read the assessments; you’re gathering criteria before you start. Critical thinking before reading helps to prime oneself with respect to what things we will be looking at.

I made a note of some tips: read the TMA first, write down your own questions, look at big picture and detail, leave things overnight before reading in detail, and then go back to review the material. I might have imagined this, but I’m pretty sure that spider diagrams were mentioned too.

Validity: Who?

In other words, where did the information come from? This connects to the source of the material. Who wrote it? Did it come from a reputable source.

The tool that we were introduced to help us think about validity was PROMPT: Provenance, Relevance, Objectivity, Method, Presentation, Timeliness. There is some accompanying OU materials about PROMPT on the OU website.

Another tool, which was gently rephrased as being ‘CAARP’ serves a similar purpose: CRAAP: Current, Relevance, Authority, Accuracy and Purpose (University of the West of Scotland)

Relevance: Why?

Simply put, why is something important? It is very well looking at an article and thinking you know something is important, but why is it really important? 

One tool is to keep continuing the ‘why’ question 5 times over. When you get an answer to the first ‘why’ rephrase it, and ask another ‘why’ question, until you create a chain of five of them. This might be the real reason why something is studied or examined. By rephrasing question, you dig deeper into the issue.

Criteria: What?

What is important? In other words, how do you make an evaluation about whether a particular article or source (or topic) is important in the context of the problem? A meta question (questions about questions) is: what questions would you ask what is important?

An interesting tool that was shared was something called the CIA Phoenix list (Wikiversity).

Three of the first questions from this list are: Why is it necessary to solve the problem?, what benefits will you receive by solving the problem?, and what is the unknown? The idea is to use this list to try to dig deeper to evaluate a problem. There is also a list of questions that relate to evaluating a plan.

One further question was asked, which was: which question (on the Phoenix list) is your favourite?

Evaluation: How?

How does everything all connect with each other. In other words, if you have found something out, how can you use it? When it comes to being a student, an evaluation is often expressed through a tutor marked assessment, which is often in the form of an essay.

One of the tools that might be useful for essay writing is something called PEEL, which is an abbreviation for: Point, Evidence, Explain, Link. In other words, you make a point, you evidence that point (with a reference), explain to the reader what it is (and why it is important in the context of an argument), and then add some words which link to the next paragraph. There could well be one PEEL per paragraph.

Another tool was introduced was PESELS, which is an abbreviation for: Point, Explain, Support with evidence, Evaluate (for or against), Link, Signpost to the next paragraph.

Reflections

I haven’t ever come across this particular pyramid before, but I do know of another (and arguably similar) pyramid, Bloom’s taxonomy (Wikipedia) which is likely to be more useful in terms of thinking about our own understanding and learning of a topic, as opposed to surveying, and reading. 

I had heard of the PEEL approach to essay writing, but I had not heard of the PESELS abbreviation; this just goes to show that there are always things to learn! Another tool that was unfamiliar to me was the Phoenix list, which looks quite useful in terms helping to reflect on what has been found about a subject. A lot was covered in a short time, and I will certainly have come round for another pass of this buffet.

The closing points were helpful, which included: critical thinking starts before reading, it takes time, and you need to spread it out over an extended period of time; it is something that can take days, since you need time to let things sink in.

It was also a quite a busy session, with up to 120 students attending. The session is, of course, one of a series on Student Hub Live. A later section will focus on the evaluation stage of the pyramid model. It was also interesting to learn about what general study skill support is available for students. There are, of course, recordings of other sessions that are available.

A final point: please don’t use Wikipedia in formal pieces of writing; always consider the validity of your sources. Formal references from the university library presented using the Harvard format, as described in CiteThemRight is always the way to go. 

Acknowledgements

Many thanks to Student Hub Live, and to Margaret Greenhall who was the presenter and facilitator of this session. The structure of this blog completely mirrors what she presented. I also have quoted from her directly when preparing these notes. I did try to find references for each of the tools that are mentioned in this blog, but I haven’t managed to track these down.

One of the important components of the TM470 EMA is your literature review.

The literature review component serves a number of purposes:

• It tells your examiner what you have read, and enables them to understand where you are coming from. In other words, what you present in a literature review enables the examiner to understand, broadly, what your project is all about. 
• It enables you to demonstrate to your EMA examiner your research and critical thinking skills. 
• It allows you to demonstrate your writing and communication skills. Just as your TM470 EMA is a narrative of your entire project, the literature review within that broader narrative (or story) presents a narrative  (or story) about your reading and your research.

The literature review can be primarily linked to the following TM470 learning outcome:

LO4: Gather, analyse and evaluate relevant information to complete the project successfully.

It can also be linked to the following learning outcomes:

LO3: Identify, list and justify the resources, skills and activities needed to carry out the project successfully. Identify and address any associated risks.

LO7: Communicate information, ideas, problems and solutions clearly

A really important rule of thumb is: if you use a resource in the body of your report, that resource should be introduced within the literature review section. A resource might be any number of different things, depending on what your project is all about: it might be some module materials, a textbook, an academic paper, or even some software. Also, if you have something in the references section, it should have been ideally in the literature review section (although it is okay to occasionally break that rule, if it helps with the writing and presentation of your project report).

What follows are a set of what I hope to be useful ideas about how best to complete a TM470 project literature review.

Starting the literature review

An important question to ask is: how do I start my literature search? The biggest tip I can offer is: begin with what you know. This might be the specifics about a project, or maybe beginning with some of the level 3 module materials that you have previously studied. If you have studied TM356 Interaction Design and the User Experience, for example, a really good place to start is the module materials, and the accompanying set text. The textbook contains a lot of references which you can look to, and you can find many of these resources in the university library.

The OU library is also a great place to start too. It contains a whole host of useful resources, such as eBooks, and hundreds of thousands of academic articles that have been published in academic journals. When starting out to look at a subject they have not looked at before, some researchers carry out searches of library databases using a systematic approach, making notes of what keywords they have used, and what they have found.

Another tip is: if you find an interesting paper in the OU library it is sometimes possible to find out how many times a paper or article has been referenced, and what papers have referenced the paper that you have found. Looking at the popularity of papers, and chains of referencing can enable you to find out what papers or bits of research have been influential in a subject area. Sometimes, it is also useful to look to see what other papers a particular author has written about.

A final tip in carrying out a literature search: ask your tutor! The TM470 module team try to match students and student projects with tutors who have a particular specialism. After having an initial discussion with your tutor about your project, it is completely okay to ask the question: do you have any suggestions?

Criticality

During the course of your TM470 project, you might look at a lot of resources. Whilst it might be tempting to show everything that you’re read or looked at whilst working on your literature review, please don’t. You need to be selective, and you need to do this to demonstrate your critical thinking skills. 

More information about what this means is available in the OU booklet about Thinking Critically.

In terms of TM470, it is important to ask: how does this resource influence, affect, or relate to my project? A good literature review will introduce some concepts or ideas, which are referenced. These concepts or ideas are then used or applied within the body of a report to solve a particular problem.

Resources

In TM470, there are a number of useful resources that you may have seen, that you should be aiming to revisit whilst you work on your project.

The two key bits of module materials that you must review have the title: Preparing a Literature Search, and Reviewing Literature. A recommendation is to get a printout of these resources (by using the “view as single page” option), and work through each of the activities. You should also have a listen to the Finding and using research podcast. 

From the Preparing a Literature Search resource, do pay particular attention to the four stages of a literature search. The Reviewing Literature resource offers a set of useful pointers in the introduction which helps you to look at resources. 

Regarding this second resource, the following bit of advice is important: “This template isn’t always applicable, not least because it can become monotonous to read. You will need to make your own decisions about which elements should be included and which omitted.” These two sentences relate to the point about criticality, and the need to write a literature review that is appropriate for your own project.

On the subject of writing, a good resource to look to is the OU’s pages about Developing academic English. I also recommend The Good Study Guide, which is available to download as a PDF. Chapters that may be particularly useful when writing the literature review (and your EMA report) are Chapter 9, Researching online, Chapter 10, Writing the way ‘they’ want, and Chapter 11, Managing the writing process.

Referencing

If you use, or write about a resource in your project report, you need to make sure that you reference it correctly. In your TMAs and EMAs, there are two key bits to think about: the first is how to reference something within the body of your report (when you’re referring to something), and the second is how to provide a reference to a resource within the references section towards the end of your EMA. Another rule of thumb is: if you are writing about a resource, you need to reference it. Similarly, if you quote from a resource, you definitely need to reference it. 

The OU makes use of the Harvard referencing system, which is both comprehensive and flexible. Using this system, you can reference just about anything. Not only can you reference books and journal articles, you can also reference art works, web pages, and software. The OU has a subscription to a web resource called CiteThemRight. If you’re unsure how to reference something, do have a look at this website. 

When referencing papers or textbooks, a firm recommendation is to make sure that you also include page numbers. The reason for this is simple. Including page numbers clearly demonstrates attention to detail, and gives your EMA examiner further evidence of your depth of reading and understanding.

Finally, do make sure that you reference (and demonstrate an understanding of) earlier OU modules you have studied. This is a really efficient way to demonstrate to your examiner what topics or subjects your project relates to. You can reference any OU module material, whether it is a module website, a PDF, or printed module block. If you’re unsure about how to reference materials from any of your earlier studies, do ask your tutor.

Common Questions

Do ask your tutor any questions that you might have whilst carrying out a literature review. Here are some answers to some common questions, which might be useful.

Q: How many references should I provide?

A: There is no hard and fast rule for this, since every TM470 is different. You should choose enough resources to demonstrate the reading that you have needed to do, to complete a project that shows technical skills and knowledge you have gained during your degree studies. If pushed, I would say that a distinction quality EMA report might reference as many as 20 resources, but these resources must be important, relevant, and applied within the body of your project. In other words, your chosen resources should have influenced the work that you have done.

Q: How much time should I spend on the literature review?

A: Again, there is no hard and fast answer to this one. Some EMA reports are all about carrying out research. In a research focussed EMA, you might spend more time doing a literature review than you would for a very practical EMA. Overall, the literature review section contributes towards 20% of the overall EMA mark, but this doesn’t mean that you should only spend 20% of the time. A suggestion is to approach the literature review iteratively. For example, whilst trying to solve a technical problem, you might have to do more reading, which means that you might have to go back and to edit your literature review section.

Q: How long should the literature review be?

I’m afraid I’m going to give you a similar answer to all the others: it depends on your project! The TM470 module guidance suggests that you should be able to write everything you need to write within the 10k word limit. Given the importance of the literature review to a number of learning outcomes, I would say that the literature review is quite a substantial section within your EMA: it sets the scene, and goes a long way to demonstrating your critical thinking and problem solving skills (through the resources that you choose). Some project will have longer literature review sections than others. It should be as long as it needs to be, given the aims and objectives of your project.

Summary

This blog has shared bits of advice (and some links) that might be useful when it comes to writing your TM470 literature review.

One of the most useful bits of advice about report writing that someone gave me was: make sure it is interesting. 

Although this bit of advice related to EU project deliverables, it is just as applicable to your TM470 EMA. 

Your TM470 EMA is a technical narrative (a story) about your project. The literature review section within your report is a narrative within a bigger narrative; it is the story of your reading. It is a story which introduces resources which you will then go onto apply later on within your report. It is an important section which demonstrates the depth of your reading, and shares what you know about with the examiner.

Other blog posts that relate to the study of TM470 can be found through the TM470 blog tag.

Good luck with your literature review, and remember to make good use of your tutor, by asking them lots of questions.

Between Friday 8 June and Sunday 10 June 2018 I attended an EdD residential weekend at the university campus in Milton Keynes. The EdD residential weekend was something that was new to me: I was attending in the capacity of a ‘co-supervisor’ (or ‘second supervisor’). 

The EdD qualification is a doctorate in education that is at the same level as a PhD, except for one fundamental difference: the research and contribution to knowledge carried out through an EdD is situated in the educational practice or context of the student who is carrying out the research. 

One of the things that I learnt from the weekend was that other institutions have their own EdD programmes. Since 1997 more than 370 students have been awarded an EdD through the OU’s EdD programme.

What follows is a transcription and summary of some of the notes that I made during the weekend. There are mostly from my perspective of a supervisor, but they might be of interest and use to EdD students or anyone who is interested in learning more about what EdD research and study entails.

Introduction

The event was introduced by Inma Alvarez, the university’s Centre for Research in Education and Educational Technology EdD programme director. Inma emphasised that the EdD is a professional doctorate that enables students to gain skills in educational research and enquiry and be able to carry out a study that contributes to professional and practice knowledge.

The EdD takes at least 3 and a half years, with a maximum of 6 years. Students are provided with two supervisors; a lead supervisor and a co-supervisor. During the first year, students are required to design and carry out a preliminary study.

Progress on the EdD is not measured through tutor marked assignments (TMAs) but by a series of progress reports. At the end of the first year, students are required to produce a report which is assessed by an academic who is neither of their supervisors. Inma made an important point that a lot of the responsibility is down to the student; an EdD should take notes of their supervision meetings and actively manage their supervisors!

All the students were given a number of useful tips, including: treat the programme guide as your ‘bible’ and subscribe to the student forums, so students can get updates of when people post messages, updates and questions. 

The first year is all about becoming an independent researcher, which includes carrying out a literature review, carrying out that initial study, submitting 4 progress reports and the end of year progress report. The final report will contain an introduction to the project, a summary of the research questions, a literature review, a section about the methodology that is adopted, a description of an initial study, outcomes, and a detailed reference section.

In the second and third year students will ‘follow a more independent and individual programme supported by their supervisors’.

Students will have access to resources, which includes access to the OU graduate school network, the EdD programme website and online doctoral training resources. Another important message that was coming through was: ‘be responsible for your own development’, and a connected thought is to start a reflective diary. This diary can be used to keep notes about what is studied and what is learnt, help to develop academic writing and creativity.

Doctoral researchers and supervisors

The aim of this next session was to enable supervisors to meet their students and members of the EdD team. Some notes that I made from this session were about “gaining confidence in plans, getting used to critical feedback, getting some research training, understanding research ethics, talking to some EdD graduates and becoming a research professional”. 

I also made a note that there was a group discussion about the question: what is theory in education? I noted that there is the concept of ‘critical theory’, but there are other approaches and theoretical tools that could be used, such as critical pedagogy and activity theory. This said, I was also mindful that the educational research notion of ‘theory’ is slightly different to a scientific understanding of what a theory is.

Day 2: Doctorate in Education Literature Review

The second day began with a presentation by Ursula Stickler in a library seminar room. The aim of the session was to learn more about ‘how to approach a literature review, the criteria, and practical considerations, such as knowing when and where to stop’. 

We were given an activity, where we were asked the question: why are you doing a literature review? Answers included: looking at themes, examining ideas and methods, examining debates, learning about academic literacies and making sure you’re not duplicating your research. Other answers also included identifying key authors and researchers and uncovering your own view of the literature and what has been done before. I also noted down some key terms that were used in the REF, the Research Excellence Framework: originality, significance and rigour.

We were then guided to another activity, where we had to answer the question: how to best go about a literature review? Other questions that were asked included: where to start, where to finish, what to include and what to leave out. It was also important to ask the question: what are the key journals and writers? It’s also important to be clear about what the main argument (or arguments) are. Another note I made was: narrow your search, find your gap (within the research) and widen your implications (which I assume relates to the impact that your research can make).

The final activity asked the question: what are the strengths of a good literature review? I didn’t make too many notes during this part of the event, except that the discussions were focussed upon an article by Boote and Beile entitled ‘Scholars before researchers: On the centrality of the dissertation literature review in research preparation’.

Session: Ethics

Ethics are important. The first session on ethics was facilitated by Alison Fox and Kris Stutchbury. I made a note that “your entire project is an ethical task” with an accompanying comment that how students choose their research projects, carry out research tasks and disseminate their research results are all ethical tasks. In this session I was introduced to an new acronym: CURD, which stands for Consequence, Ecological, Relational and Deontological.

The next ethical session was all about case studies. Duncan Banks gave a presentation that had the title: an introduction to research ethics (PDF). We were introduced to the BPS, British Psychological Society, code of ethics of human participants. Some of the key points I noted down were: respect for the autonomy and dignity of persons, that the research must have scientific value, quality and integrity, and that it must maximise benefit and minimise harm. Another dimension of ethics relate to risks, both to research participants and also to the researcher.

Day 3: Designing an initial case study

The third day of the event was organised slightly differently; we were all brought together for a plenary presentation, and then we were able to attend different parallel sessions. In some respects, the weekend turned into a mini conference! What follows is a polished and paraphrased version of the notes that I made during each of these sessions.

Opening plenary session

The opening session was presented by Felicity Fletcher-Campbell. 

Literature review

Felicity returned to one of the important topics of the weekend; the literature review. The literature review should take account of a theoretical position, the substantive area and a methodology. A literature review is ongoing, flexible, adaptable, malleable, reactive and proactive. During the literature review, students should add and remove papers, and also reconceptualise their work. It offers a means to inform your empirical work. A key phrase I noted down was “keep it slim and purposeful”, which I thought was great advice. 

The initial study

The initial study is important since students need to carry this out to complete an important assessment within the EdD. Students must write a report that must present a clearly structured framework for the whole study. In some ways, this initial study and accompanying report is used to ‘sort out’ any issues regarding theory or theoretical position. I noted that the “report shows your developing knowledge and experience of relevant theoretical traditions and literature”. It is used to critically assess where the different authors and researchers are coming from and their accompanying perspective. The report also allows students to relate the literature to their research questions. 

Felicity offered some really useful tips and pointers: “your theoretical position informs your methodology” and “buy yourself a very big box of quotation marks and inverted commas” and “be really boring and put quotes around everything and be obsessed with page numbers”. On the subject of ethics, students were told: “name your supervisors on consent forms, so they get blamed too”.

On the subject of time, “research time is different than normal time; time fills up, everything will take longer than you think they will take”. 

There’s also the need to balance everything; to balance the preparation and the doing, the data production and the data collection, and the analysis with the report writing. Also, when it comes to the writing, “the initial study will help you to explain the genesis of your main study”.

The main study

Some key points about main study were: “you need to tell a story” and to ask “what I need to tell the reader? What do they need to know? Why did you tell me this? Also, why didn’t you tell me this earlier?” From a personal perspective, I’ve internalised the point about the need to tell a story, and I’ve passed this message onto the TM470 Computing and IT project students that I help to support. The narrative that is presented to the examiner is really important. 

Parallel Session: Working with digital data

The first parallel session that I went to was by Carol Azumah who discussed the usage of digital data and resources, such as blogs and social media. Some resources, such as blogs, can be viewed as public documents. Two terms that I noted down were ‘discourse oriented online ethnography’ and ‘fast ethnography’. An important point is that ethics always need to be considered. We were directed to the Ethics pages of the Association of Internet Researchers (AoIR).

Parallel Session: Concept maps

Diane Harris’s session was about how to use concept maps in research. Concept maps were introduced as tools that can be used to hear the voice of participants and “for them to own what they have said to you”. Diane offered a specific example of how they were used to study music education in a school. Participants could own, add to and create maps. The resulting maps could then be analysed using thematic analysis or critical incident analysis. Regarding this second technique, Diane mentioned two researchers, Harrison and Lee (2011) (Taylor and Francis) who used the approach in medical education.

Closing session: the way ahead

After the parallel sessions, we returned to the plenary room, where we were offered some closing advice from Inma Alvarez. From what I remember and from what I’ve noted down, students were encouraged to work as and think of themselves as independent researchers. They should also think of their supervisors as critical friends. Students were encouraged to identify what skills are needed, reflecting earlier attention that was given to the importance of continuing professional development.

The concluding bits of advice were: “be open to the unexpected; you can modify the title of your study [if you need to]; work to deadlines and use frameworks to guide what you do, and be sure to manage your supervisors”.

Reflections

What really impressed me was how well the EdD weekend was planned. There were ample opportunities to speak with supervisors and fellow students between more formal events and activities. It struck me as being a really nice mix.

There were a couple of highlights. The first one was a presentation by a former EdD student who spoke of some of the challenges of doctoral study. The second was the talk by Felicity Fletcher-Campbell, which was packed filled with useful and practical advice and delivered in a thoroughly engaging way.

I never took place in a formal induction session when I embarked on my own doctoral studies. What really impressed me with the weekend was its emphasis on structure; the importance of the literature review, the importance of the initial study, the main study and how everything connects together. I think the weekend has also positively impacted on my own practice; only by writing this blog have I realised that I have started to pass on some of the tips mentioned during this weekend to some of the undergraduate project students that I support.