OU blog

Personal Blogs

Christopher Douce

Doctoral study: Third Party Monitoring and mentoring

Visible to anyone in the world

This post shares some notes that I made during a continuing professional development (CPD) event organised by the Research Career Development Team (RCD) that took place on 6 November 2024. The event was all about how a third party monitor supports doctoral students and doctoral level study. 

The event was pitched as being useful for “those who are new to third party monitoring”. It’s key objective and learning outcome were to help participants understand what the OU expects of third party monitors, whilst also sharing practices and experiences between participants.

Before moving sharing the key points that I took away, I would just like to share that I’m mostly writing this piece from the perspective of a third party monitor, sharing points with other potential fellow third party monitors. Although I’ve made this post widely available, some of the links that I’ve shared are very OU specific, and will not be available outside the OU.

Third party monitoring: the role

The aim of the third party monitor (TPM) is to provide pastoral support for research students outside the supervision team. A TPM is an academic member of staff who has had research degree supervision experience. They are there to provide confidential, independent, and dispassionate support. They are able to offer advice about how to deal with the challenges of doctoral research, how to deal with and handle conflict within (and possibly outside) the supervision team, and can offer perspectives about research process. A TPM can provide practical opinions about the supervision process. They may, for example, help to identify resource management difficulties; a student might meet with supervisors too frequently, or not frequently enough. A TPM may be able to offer practical advice on how to broach this with supervisors. The key point is that a TPM must always act in the best interest of the student.

In terms of meetings, there is an obligation for a TPM to meet with a student at least once a year, but a student might email the monitor at any point throughout the year. Meetings do not need to be in person; they can take place online. The first meeting should take place by the fifth month following registration. In my own practice, I tend to meet with students every quarter, but I would be happy to meet more regularly if there are any emerging or ongoing issues that need to be addressed or resolved. Meetings can be documented through a form (which is used within the WELS faculty) but this is not compulsory. All that needs to be recorded is a date of a meeting.

The TPM is described in the section 6 of the university research degree regulations. There are also third party monitoring guidelines from the graduate school network. Any member of staff who is carrying out third party monitoring should record it within their academic workload management (AWM) tool. At the time of writing I don’t have an official figure, but a guideline to work to is: one day per year, for each student you are monitoring.

What happens if things are not going well?

During the session, there was a useful discussion about what to do if things are not going very well. I noted down some important points.

The first point is: if possible, have discussions with everyone. Before even picking up the phone, gain permission from the student you are supporting; confidentiality, trust, and transparency in terms of actions are paramount.

It isn’t easy doing research. Depending on the topic, it can be sometimes lonely. It can sometimes place significant demands on reserves of intrinsic motivation. When looking at a subject in a new way, this can have a potentially destabilising way of seeing the world. All this means is that there are times when doing research can negatively affect our mental health. A TPM might be able to help just by having a chat about how everything is going. You can also signpost some resources that relate to staying mentally healthy.

Sometimes, you may well need to seek advice to answer some questions which may come up when meeting a student. Two key points of help were mentioned: the graduate school, or a group of staff within your academic school that supports postgraduate students. If something is serious, you might wish to contact your head of school, or their delegate.

Resources

A number of useful resources were shared during this session. The OU national support network that follows isn’t specific to the TPM role, but offers pointers to a whole host of different resources.

There is also something called the STEM Mentoring Scheme: Framework, which dates back to 2021.

Reflections

At the time of writing, I’m a third party monitor for two doctoral students.

When I started as a third party monitor, I have to confess that I didn’t really know what I was doing. My interaction with students began with an email introduction from a graduate student supervisor (or lead). The was the followed with organising an informal meeting between myself and the student, where we introduced each other.

My big take away from this session was the importance of active listening and relationship building. Empathy also plays an important role. As third party supervisors we all have our prior doctoral experience we can draw upon. In some cases, it can be about being practical; sometimes third party monitors need to find things out for our students. In turn, we may need be their advocate, but always with their permission.

I found the discussion about the formality of a form, and its usefulness interesting. During my own meetings, I do sometimes make notes, but it does always depend on what is being said and what needs to be done.

Given the title of the session, I was expecting to learn more about mentoring, perhaps even something about models of mentoring. At various points in my OU career, I’ve been both a mentor and a mentee. Sometimes these relationships have gone well, other times less so. To help others more effectively, I feel as if I need to know more about how to ‘mentor’ well.

Acknowledgements

The event was facilitated by Dr Liz FitzGerald, a Senior Lecturer in the OU’s Institute of Educational Technology.

Permalink Add your comment
Share post
Christopher Douce

Ethics support for projects: HREC and SRPP

Visible to anyone in the world

On 24 January 2024 I attended a bit of a professional development session that shared an overview of two important points, and organisational units, which relates to research and research ethics. The session was facilitated by Alison Fox, Steven Bond, who was from the data protection team, and Bart Gamber, who was from the Student Research Project Panel (SRPP).

Introducing HREC

Research ethics is important. To help OU researchers and doctoral students, there is something called Human Research Ethics Committee (HREC) which provides services to researchers, and provides an ethical approval mechanism. 

Ethical approval needs to be taken really seriously for a number of reasons. Approval ensures the safety of researchers and safety of participants. A further check of your research aims can also improve the quality of your research. My argument is that articulating your research to others can only improve its clarity and purpose. Also, when it comes to publishing your research, some journals will insist on a detailed summary of how you have approached ethics, and some journals will directly ask for evidence of whether you have gained formal ethical approval as a part of a study.

There are, however, some projects that might not need HREC review or approval, such as an evaluative study that takes place within a course, or a study which is feeding back into a university service, for example. Also, research that is designed to inform a work practice, market research, or research with data that has already been collected (where that data set has been gathered through a process which ha been subject to its own ethical approval).

HREC offers links to other teams and groups that can offer help and advice, such as the library and information security teams (if not using core university systems). You might, for example, gather a lot of data. If you think that other researchers might want to use your data, the library will be able to offer advice and guidance about how (and where) to make that data available. Also, knowing how to secure your data is also an important part of the ethics process.

Submissions are made to HREC through something called the ethical review manager tool (which reminds me of the name of another tool: the postgraduate research manager tool).

If anyone has any questions about the process, the facilitators encouraged anyone to get in contact. To help everyone navigate through all these practical questions and challenges, it was interesting (and useful) to learn that HREC run research monthly drop in sessions, which typically take place on the 3rd Tuesday of the month.

Introducing SRPP

A related unit goes by the abbreviation, SRPP, which is short for the Student Research Project Panel. The way that I understand it, SRPP has a couple of interconnected aims. It can help to identify potential students who might be able to participate in research. Equally, it is there to make sure that students are not ‘over-research’, which means ‘contacted unnecessarily regularly’.

Like HREC, submissions to SRPP are made through a form. Some practical tips shared were: plan early, and apply early. These things can take a bit of time.

Resources

Just before the session, a PowerPoint resource was shared. After the session, I noticed that it was packed filled with useful links, many of which can be accessed externally. Here is a summary of what I took to be the most important links:

Reflections

A useful session! It was also one that was very timely since I have been awarded a small amount of funding to carry out a pilot project to explore the connection between stories, storytelling, and the professional identity of software engineers. My next action is to attend one of those drop-in sessions, and then to review all the forms. Whilst I do usually hate form filling, I do recognise that these forms relate to a process that is there to protect everyone.

Acknowledgements

Many thanks to Alison Fox, her co-facilitators, and everyone who is involved with the HREC and SRPP units.

Permalink Add your comment
Share post
Christopher Douce

Bibliometrics, Altmetrics & DORA

Visible to anyone in the world

On 2 October I attended another of the OU’s professional development events. This time, it was an event organised by the OU library. Facilitated by Chris Biggs, Research Support Librarian, the session aimed to outline three things: “common bibliometrics, and their use and misuse”, “what are Altmetrics and where they can be found” and DORA, which is an abbreviation for the “Declaration on Research Assessment (DORA)” along with the “responsible use of metrics”.

I was particularly interested in this section since I’m a co-editor of an international journal called Open Learning. Bibliometrics are sometimes discussed during the annual editorial meetings between the editors, the members of the editorial board, and the publisher, who is Taylor and Frances. During these meetings, various numbers are shared and summarised.

When I saw the title of the event, my main thought was: “I should go along; I might pick up on a point or two”. What follows is a set of notes that I made during the session, along some of the useful weblinks that were shared. One thing that I should add is that the structure of these notes come from the facilitator, Chris, and his presentation. Towards the end of these notes, I share a set of reflections.

Citations

I missed the first couple of minutes, joining just at the point when Chris was talking about the ‘the social dimensions’ of citations. Citations are all about giving credit. A useful link to look at is the page Citing Sources: What are citations and why should I use them?

One view is that the more citations an article has, the more popular it is. Subsequently, some might associate popularity to quality. An interesting paper that was referenced had the title Citations, Citation Indicators, and Research Quality: An Overview of Basic Concepts and Theories

Retuning to the notion of the social dimension of citations, one metric I noted down was that self-citations account for 12% of citations. A self-citation is where an author references their own, or earlier work. Whilst this can be used to guide authors to earlier research, it can also be used to increase the visibility of your research.

A concept that I wasn’t familiar with but immediately understood was the notion of a citation circle or cartel. Simply put, this is a group of authors, typically working in a similar field, who regularly reference each other. This may have the effect of increasing the visibility of that group of authors. Chris shared a link to an interesting article about the notion: The Emergence of a Citation Cartel

A further notion that I hadn’t officially heard of, but was implicitly familiar with, was the notion of the honorary citation. This is where an author might cite the work of a journal editor to theoretically increase chances of their paper being accepted. As an editor, I have seen that occasionally, but not very often. On a related point, the publisher, Taylor and Francis has published some very clear ethical guidelines that editors are required to adhered to.

Something else that I hadn’t heard of is the Matthew effect. This means that if something has been published, it will continue to be cited, perhaps to the detriment of other articles. Again, we were directed to an interesting article: The Matthew Effect in Science.

It was mentioned there are interesting differences in between academic disciplines. The pace and regularity of citations in the arts and humanities can be much less than, say, a busy area of scientific research. It was also mentioned that there are differences between types of articles. For example, reviews are cited more than original research articles, and methods papers are some of the most cited papers. (It was at this point, I wondered whether there were many articles that carried out reviews of methodologies).

An interesting reflection is that articles that are considered to have societal benefit are not generally picked up by bibliometrics. This immediately reminded me about how funders require researcher to develop what is known as an impact plan. I then remembered that the STEM faculty has a couple of impact managers who are able to provide practical advice on how researchers can demonstrate the impact and the benefits of the research that they carry out.

All these points and suggestions lead to one compelling conclusion, which is that the number of citations cannot be directly considered to be a measure of the quality of an article.

An important element is all this is, of course, the peer review process. Some important points were made: that peer review can be slow, expensive, inconsistent, and prone to bias. As an editor, I recognise each of these points. One of the most frustrating elements of the peer review process is finding experienced and willing reviewers. During this session, I shared an important point: if an author holds views that are incompatible or different to the reviewers, it is okay to have a discussion with an editor. Editors are people, and we’re often happy to chat.

Bibliometrics

There are a few different sources of bibliometrics. There is Scopus, Web of Science, Dimensions, CrossRef and Google Scholar. Scopus and The Web of Science offer limited coverage for social sciences and humanities subject. In contrast, Google Scholar picks up everything, including resources that may not really be academic articles. A link to the following blog, Google Scholar, Web of Science, and Scopus: Which is best for me? was shared.

There are, of course, different types of metrics. Following on from the earlier section where citations were mentioned, there is the notion of normalised citations, percentiles, and field citation ratios. Normalised citations (if I’ve understood this correctly) is the extent of an article being over a time period. Percentiles relate to how popular, or widely cited an article is. There is, of course, a very long tail of publications. Publications that appear within, say, the top 1% or top 5% are, of course, highly popular. Finally, the field citation ratio relates to the extent to which an article is published within a particular field of research.

There is also something called the h-index, which relates to the number of publications made by a researcher. During Chris’ presentation, I made the following notes: the h-index favours people who have a consistent publication record, such established academics. For example, a h index of 19 means 19 papers that have been cited 19 times.

Moving beyond metrics that relate to individual researchers, there is also something called the journal impact factor (JIF). Broadly speaking, the more popular or influential the journal, the higher its impact factor. This has the potential to influence researchers when making decisions about how and where to publish their research findings. It was mentioned that there are two versions of a JIF: a metric that includes self-citations, and another that doesn’t.

Metrics that relate to official academic journals isn’t the whole story. Outside of ‘journal world’ (which you can access through your institutional library) there are an array of ever changing social media platforms. Subsequently, there are a number of alternatives to citation based bibliometrics. Altmetrics, from Digital Science, creates something that is called an attention score, which consolidates different ‘mentions’ of research across different platform. It can only do this if there is a reference to a a persistent digital object identifier, a DOI.

Previews of AltMetric data can be seen through articles that are published on ORO, the university’s research repository. Although I’m risking accusation of self-citation here, an interesting example is the following excellent paper: Mental health in distance learning: a taxonomy of barriers and enablers to student mental wellbeing. Scrolling to the bottom of the page will reveal a summary of tweets and citations; metrics from both Altmetric and Dimensions.

There are a couple of other alternative metrics that were mentioned: PlumX, which is from Elsevier, and Overton, which is about the extent to which research may be potentially influencing policy.

Responsible metrics, the OU and DORA

Towards the end of the event, DORA, Declaration on Open Research Assessment was introduced, of which the OU is a signatory. One of the most salient point from the DORA website is this: “Do not use journal-based metrics, such as Journal Impact Factors, as a surrogate measure of the quality of individual research articles, to assess an individual scientist’s contributions, or in hiring, promotion, or funding decisions”. There is also a case study that relates to the OU which can be found on the DORA website

This final and important part of the session, which had the title ‘The Idea of Responsible Metrics” was covered quite briefly. The topic of DORA is something that I really do need to look at in a bit more detail.

Reflections

I learnt a lot from this session. One thing that really grabbed my attention was the h-index. As soon as the session had finished, I asked myself a question: what is my h-index? It didn’t take too long to find it out.

After finding my h-index figure, I made a mistake; I wondered what the h-index for some of my immediate colleagues were. I found this unnecessarily and seductively interesting. I looked up h scores for professors, some fellow senior lecturers, and some newly recruited members of staff. Through various links, I could see who had collaborated with who. I could also see which of the professors were really high ranking professors.

I then stopped my searching. I asked myself another question, which was: “does any of these numbers really matter?”

It takes different kinds of people to run an academic department and a university. Some colleagues are excellent at research. Some colleagues are excellent at teaching, and some colleagues are even excellent at administration. In my own role as a staff tutor, I do a lot of academic administration and quite a bit of work which could be viewed as teaching. This means that I don’t have a lot of time to do any research. Broadly speaking, central academic staff have a much higher h-index metric than staff tutors, simply because they have more time. What research I do carry out is often applied research. This research can sometimes be labelled as scholarship, which can be considered to be research about the practice of teaching and learning.

It was interesting that one of the important points I took away was that societal impact can’t be directly measured through bibliometrics. I also found it interesting that different types of articles attract a greater number of citations. One of my biggest academic hits (I don’t have very many of them) has been a review paper, where I studied different ways in which a computer could be used to assess the quality of computer programming assessments. The articles that I have published that relate to pure research have certainly attracted less attention.

All this comes back to a broader question: in academia, what is valued? I think the answer is: different types of work is valued. Pure research is valued alongside effective and engaging teaching. The bit that ties the two together is, of course, scholarship. 

Bibliometrics are, in my eyes, are a set of measurements that attempt to quantify academic debate. It only ever tells a part of a bigger and much more complicated story. Metrics are not, in my opinion, surrogates for quality. Also, academic fashions and trends come and go.

Acknowledgements

Many thanks are given to Chris Biggs for running such an engaging and interesting session. Many of the links shared in this article were shared during Chris' presentation.

Permalink Add your comment
Share post
Christopher Douce

SEAD/LERO Research Conference ‘23

Visible to anyone in the world
Edited by Christopher Douce, Thursday, 20 July 2023, 09:56

I attended my first joint OU SEAD/LERO research conference, which took place between 4 July and 6 July 23. SEAD is an abbreviation for Software Engineering and Design Research Group a research group hosted within the OU’s School of Computing and Communications. The conference was joined by members of LERO, the Science Foundation Ireland Research Centre for Software, which based in Limerick.

What follows is a summary of the two days I attended. There was a third day that I didn’t attend, which was all about further developing some of the research ideas that were identified during the conference, and researcher professional development

The summary is intended for the delegates of the conference, and for anyone else who might be interested in what happens within the SEAD research group. All the impressions (and any accompanying mistakes in my note taking) are completely my own. What is summarised here isn’t an official summary. Think of it as a rough set of notes intended to capture some of the themes that were highlighted. It is also used to share some potential research directions and areas that intend to be further developed and explored.

Day 1: Introductions and research discussions

Bashar Nuseibeh kicked off the day by highlighting the broad focus of the conference: to consider the role of software in society. Although I missed the first minutes of his opening address due to traffic, there was a clear emphasis on considering important related themes, such as social justice.

The first session was an ice breaker session. This was welcome, since I was an incomer to the group, and there were many delegates who I had not met before. We were asked to prepare the answers for three questions: (1) Who you are, including where you are based and your role? (2) What is your main research area/interest?, and (3) Something you love about your research and something you dislike. (Not bureaucracy!)

Having a go to answer these myself, I work as a staff tutor. My research interests have moved and changed, depending on what role I’ve been doing. Most recently, it has been about the pedagogy of online teaching and learning. When I was a researcher on an EU funded project, I was looking at the accessibility of online learning environments and supporting students who have additional requirements. Historically, my research has been situated firmly in the area of software engineering; specifically, the psychology of computer programming, maintenance of object-oriented software, and software metrics (informed by research about human memory). I have, however, returned to the domain of software engineering, moving from the individual to communities of developers by starting to consider the role of storytelling in software engineering, working with colleagues Tamara Lopez and Georgia Losasso.

What I like about the research is that it is really interesting to discover how different disciplines can be applied to create new insights. What can be difficult is that different disciplines can sometimes use different languages.

Invited talk: navigating the divided city

Next up was an invited talk by Prof. John Dixon from the OU’s Social Psychology research group. John’s presentation was about “intergroup contact, conflict, desegregation, and re-segregation in historically divided societies”. John described how technology was used to explore human mobility preferences. Drawing on research carried out as a part of the Belfast Mobility Project. The project studies, broadly speaking, where people go when they navigate their way through spaces, and can be said to sit within an intersection between social science and geography. Technology was used by researchers to study activity space segregation and patterns of informal segregation, which can shed light on social processes. 

John also highlighted tensions that a researcher must navigate, such as the tension between open science (where data ca be made available to other researchers) and the extent to which it is ethical to share detailed information about the movement of people across a city.

There was a clear link between the talk and the theme: the connection between software and society. This talk also resonated with me personally: as a regular user of an activity tracker called Strava, I was already familiar with some of the ethical concerns that were shared. After becoming a user of Strava, I changed a couple of settings to ensure that my identity is disguised. Also, a year ago I noticed that the activity tracker has started to hide the start point and the end point of any activity that I was publicly sharing. A final point from the part of the day is that both technology and software can lead to the development of new methods and approaches.

Fishbowl: Discussing society and software

Talking of new methods and approaches, John’s talk (and a lunch break) was followed by an event that was known as the ‘fishbowl session’, which introduce a ‘conference method’ that I had never heard of before.

In some respect, the ‘fishbowl’ session was a discussion with rules. Delegates sat on one of ten chairs in the middle of the room, and have a conversation with each other, whilst trying to connect together either the main theme of the discussion (software and society) or some of the topics that emerge from the discussions.  We were encouraged to discuss “anything where software has a role to play”.

The fishbowl discussed consequences of technology, collective education, critical thinking (of users), power of automation, concentration of power (in corporations), the use of AI (such as large language models), trade-offs, and complex systems. On the subject of AI, one view I noted down was that perhaps the use of AI ought to be limited to low risk domains, and leave people to the critical thinking (but this presupposes that we understand all the risks). There was also a call to ensure that AI tools to explain their “reasoning”, but this also implicitly links back to points about skills and knowledge of users. This is linked to the question: how do we empower people to make decisions about the systems that they use?

Choices were also discussed. Choices by consumers, and by developers, especially in terms of what is developed, and what is good to develop. Also, when uncovering and specifying requirements, it is important to consider what the negatives might be (an observation which reminds me of the concept of ‘negative use cases’ which is highlighted in the OU’s interaction design module).

I noted down some questions that were highlighted: how do we present our discipline? Do we research how to “do software” and leave it up to industry? Should we focus on the evaluation of the impact of software on communities and society? An interesting quote was shared by Bashar, which was: “working in software research is working for society”.

A final reflection I noted was that societal problems (such as climate change) can be thought as wicked problems, where there is no right answer. Instead, there might be solutions that are not very right or wrong, or solutions that are better or worse than others.

It was difficult to distil everything down to a group of neat topics, but here are some headings that captured some of points that were discussed during the fishbowl session: resilience, care, sustainability, education, safety and security, and responsibility.

At the end of the session, all delegates were encouraged to join a group that reflected their research interests. I joined the sustainability group.

Group Work 1 - Expansion of themes from the fishbowl

After a coffee break it was time to do some work. The guidance from the agenda was to “to develop some proposals for future research (problem; research objectives; research questions; methods; impact)”. 

The sustainability group comprised of four members: three from SEAD, one from LERO.

After broadly discussing the link between sustainability and software engineering, we produced a sketch of a poster that shared the following points:

  • How can we make connections and causal links between different (sub)systems explicit.
  • How can we engineer software to be holistically ‘resource aware’?
  • What is the meta-language for sustainable software systems?
  • What are the heuristics for sustainable software systems?

On the surface of it, all these points are pretty difficult to understand. 

The first point relates to the link between software, economics, and society. Put another way, what needs to be done to make sure that software systems can make a positive contribution to the various dimensions of our lives. By way of further context, the notion of Doughnut Economics was shared and discussed.

The second point relates to the practice of developing software. Engineers don’t only need to consider how to develop software systems that use resources in an efficient way, they also need to consider how software teams use and consume resources.

The third point sounds confusing, but it isn’t. Put another way: how do we talk about, or describe, or even rate the efficiency, or sustainability of software systems. Going even further, could it be possible to define an ISO standard that describes what elements a sustainable software system could or should contain?

The final point also sounds arcane, but when unpacked, begins to make a bit of sense. In other words: are there rules that software engineers could or should apply when evaluating the energy use, or overall sustainability of software systems? There are, of course, some links from this topic to the topic of algorithms and data structures (which is explored in modules such as M269 Algorithms, data structures and computability) which considers efficiency in terms of time and memory. A simple practical rule might be, for example: “rather than continually polling for a check in status of something, use signals between software elements”. There is also a link to the notion of software patterns and architecture (with patterns being taught on TM354 Software Engineering).

Day 2: Ideate and prototype

The second day kicked off with summaries from the various groups. The responsibility team spoke about the role of individuals, values, and organisations. The care group highlighted motivation, engagement, older users and how to help people to develop their technical skills. The education had been discussing computing at schools, education for informed choices, critical thinking, and making sure that the right problem is addressed. The resilience group discussed support through communities, and the safety and security group asked whether safety related to people, or to process.

A paraphrased point from Bashar: “look to the literature to make sure that the questions that are being considered haven’t been answered before” also, reflecting on the earlier keynote, “consider radical methods or approaches, and consider the context when trying to understand socio-economic systems”.

Group Work 2 - ideate and prototype

Back in our groups, our task was to try to operationalise (or to translate) some of our earlier points into clearer research questions with a view to coming up with a research agenda.

Discussing each of the points, we returned to the meaning of the term sustainability, along with what is meant by resource utilisation by code, also drawing upon the UN sustainable development goals https://sdgs.un.org/goals .

We eventually arrived at a rough agenda, which I have taken the liberty of describing in a bit more detail. The first point begins from a high level. Each subsequent points moves down into deeper levels of analysis, and concludes with a point about how to proactively influence change:

  1. What types of software systems or products consume the most energy?
  2. After identifying a high energy consuming product or system, use a case study approach to holistically understand how energy used, also taking into account software development practices and processes.
  3. What are the current software engineering practices of developers who design, implement and build low energy computing devices, and to what extent can sharing knowledge about practice inform sustainable computing?
  4. What are the current attitudes, perceptions and motivations about the current generation of software engineers and developers, and how might these be systematically assessed?
  5. After uncovering practices and assessing attitudes, how might the university sector go about influencing organisations to enact change?

Relating to the earlier call to “draw on the literature”, a member of our team knew of some references that could be added to the reference section of our emerging research poster:

Lago, P. et al. (2015) Framing sustainability as a property of software quality. Communications of the ACM, Volume 58, Issue 10, pp.70–78. https://doi.org/10.1145/2714560

Lago, P. (2019) Architecture Design Decision Maps for Software Sustainability. 2019 IEEE/ACM 41st International Conference on Software Engineering: Software Engineering in Society (ICSE-SEIS), 25-31 May 2019, IEEE. https://doi.org/10.1109/ICSE-SEIS.2019.00015

Lago, P. et al. (2021). Designing for Sustainability: Lessons Learned from Four Industrial Projects. In: Kamilaris, A., Wohlgemuth, V., Karatzas, K., Athanasiadis, I.N. (eds) Advances and New Trends in Environmental Informatics. Progress in IS. Springer, Cham. https://doi.org/10.1007/978-3-030-61969-5_1 

Manotas, I. et al. (2018) An Empirical Study of Practitioners' Perspectives on Green Software Engineering. 2016 IEEE/ACM 38th International Conference on Software Engineering (ICSE). 14-22 May 2016. https://doi.org/10.1145/2884781.2884810

Wolfram, N. et al. (2018) Sustainability in software engineering. 2017 Sustainable Internet and ICT for Sustainability (SustainIT). 06-07 December 2017. https://doi.org/10.23919/SustainIT.2017.8379798

(A confession: I added the Manotas reference when I was writing up this blog, since it looked like a pretty interesting recommendation, especially have previously been interested in the empirical studies of programmers).

Conference visit: Bletchley Park

The second day concluded with a visit to Bletchley Park, which isn’t too far from the campus. It seemed appropriate to visit a place where socio-technical systems played such an important role. I had visited Bletchley Park a few times before (I also recommend the computing museum, which is situated on the same site), so I sloped off early to try to avoid the rush hour to London.

Day 3: Consolidate and plan next steps

This final day contained a workshop that had the title “consolidate and plan next steps” and also had a session about professional development. Unfortunately, due to my schedule, I wasn't able to attend these sessions.

Reflections

I really liked the overarching theme of the event: the connection between software and society. Whilst listening to the opening comments it struck me that there were some clear points of crossover between research carried out within the SEAD group, and the research aims of the OU Critical Information Studies research group.

It was great working with others in the sustainability group to try to develop a very rough and ready research agenda. It was also interesting to begin to discover how fellow researchers in other institutions had been thinking along similar lines and have already taken some of our ideas further. 

One of my next steps is to continue with reading and exploring with an aim of developing a more thorough understanding of the research domain.

It was interesting that I was the only staff tutor at the event. It is hard for us to do research, since our time split in three different ways: academic leadership and management (of part time associate lecturers), teaching, and whatever time remains can be dedicated to research. For the next few years, my teaching ‘bit’ of time will be put towards doing my best to support TM354 Software Engineering.

Looking forward, what I’m going to try to do is to integrate different aspects of my work together: integrate the teaching bit with the research bit, with the tutor management bit. I’m also hoping (if everything goes to plan) to tutor software engineering for the first time.

As well as integrating everything together, another action is to begin to work with SEAD colleagues to attempt to put together a PhD project that relates to sustainable computing.

Update 20 July 23: After doing a couple of internet searches to find more about DevOps, I discovered a new book entitled Building Green Software (O'Reilly), which is due to be published in July 24. I also found an interview with the lead author (YouTube), and learnt about something called the Green Software Foundation. I feel really encouraged by these discoveries.

Permalink
Share post
Christopher Douce

Planning and evaluating impact of a scholarship project

Visible to anyone in the world

On 23 June 23, I attended an online seminar about impact and scholarship, which was facilitated by Shailey Minocha and Trevor Collins. Shailey is the School of Computing and Communications scholarship lead, and Trever used to be a director of the university’s STEM scholarship centre, eSTEeM.

The event is summarised as follows: “we will take you through the toolkit for impact of SoTL and introduce you to various resources of the impact evaluation initiative. By the end of the event, we hope that you will feel prepared to use the resources/toolkit to plan, evaluate, and report the impact of your (past, present and future) SoTL projects and interventions.” Early on in the seminar, there was a reference to a page about impact, which can be found on the eSTEeM website.

Stories of impact

One of the most notable parts of this seminar was the amount of articles and resources that were shared. One of the first articles mentioned was: Impact of Scholarship of Teaching and Learning: A compendium of case studies. In this publication, 16 Scholarship of Teaching and Learning (SoTL) projects were analysed by something called the Impact Evaluation Framework (IEF).

Two other articles were: 

Defining impact

The UK Research Excellent Framework (REF) defines impact as “an effect on, change or benefit to the economy of society”. There is a connection here with the school research fiesta which took place earlier this year: REF impact case studies are important. In terms of SoTL, impact implies demonstrative benefits to learning and teaching that are directly attributable to a specific project.

I noted a question: what has changed (as a result of a project)? What new insights have gained (from the project)? Also, how can the institution put the outcomes into use? What are the current debates that this scholarship relates to?

Impact evaluation framework

The impact evaluation framework was mentioned, but what exactly is it? It is said to contain 12 facets (or aspects) of impact, which are spread over 4 categories. During the session, I attempted to briefly summarise what they are:

  • Learning and teaching: impact on student experience; student retention; evidence of excellence?
  • Transfer to others: an influence on discipline based teaching, research, or practice; dissemination of outcomes; extent of adoption by others?
  • Stakeholder benefits: enhanced mutual understanding; facilitated personal or professional development; recognition of project team members and other stakeholders.
  • Cultural and economic benefits: has it fostered scholarship culture; financial implications (saving of money); funding opportunities.

Relating to this framework, Shailey shared a link to her blog, Impact of scholarship of teaching and learning

This article provides links to related resources, such as an executive summary, case studies, guide for educators, and two workbooks: one about impact evaluation, and another about planning for impact

Six principles (or values) of SoTL

A particularly useful resource which relates to scholarship is a free badged open course from Open Learn: Scholarship of Teaching and Learning in STEM.

This short course has 6 sections, which emphasises what contributes to an effective study:

  1. Grounded in student learning and engagement
  2. Grounded in one or more context
  3. Rigorous and methodological sound research design
  4. Conducted in partnership with students
  5. Appropriately public for evaluation and uptake by peers
  6. Reflection, critical reflection and reflexivity.

Strategies for planning and generating impact

This section of the seminar shared some useful practical tips for anyone who was considering setting up a scholarship project, or thinking about impact. These have been paraphrased as follows:

  • Align scholarship with strategic priorities of institution, school and discipline.
  • Use social media to create community and connection; make use of YouTube channels, and other social media platforms.
  • Make sure you keep a clear record of evidence of impact.

Another thought I did have was: consider developing a scholarship team which has complementary skills.

Impact resources

Building on the section which introduced the impact evaluation framework, this section aimed to highlight resources and ideas that could be useful. A key element of this was the Theory of Change methodology (ToC). This was highlighted as a dominant image methodology which is used by the Office for Students https://www.officeforstudents.org.uk/ . Apparently, the Theory of Change helps scholars plan a project for impact, helping them to consider pathways to impact from the start of project.

Some resources that were highlighted included a ToC visual tool, a SoTL impact evaluation workbook, and the Planning for SoTL impact evaluation workbook. There was also a question driven template, which was considered to be a project management tool.

A key point highlighted in this section: know who your stakeholders are. Without stakeholders, and without influence across stakeholder communities, there is no impact.

Reflections

A question that I always return to is: what is the difference between scholarship and research?

In some respects, the answer to this question is directly linked to the notion of impact. The way that I understand it is that scholarship relates to impact on teaching practice and activities. In turn, scholarship can have a direct impact on the student experience. Research, on the other hand, has impact on an academic discipline, or field of study. There is, of course, cross over between scholarship and research, especially within the domain of education and education studies. 

Another thought I always come back to is that both scholarship and research are important, and that academics should do both: research relates to what we teach, whereas scholarship relates to how we teach. I can’t get away from the perception that due to the Research Excellence Framework (REF) that research activity is valued higher than scholarship activity. This said, there are other metrics and league tables that relate to the student experience: the student satisfaction survey, and the Teaching Excellence Framework (TEF).

This seminar was timely. I’ve just finished setting up what is called my annual Academic Workload Plan. In the forthcoming year, I’m hoping to set up a scholarship project (subject to approval, of course). An important point from this session was: build in dissemination and impact right from the start.

I thought that the tools shared during this session were potentially useful, especially the articles. The session clearly highlighted that there are challenges in planning for and generating impact: projects can often take longer than expected, and project members can become tired at the end of the project. An excellent point was made; sometimes impact could occur years after the completion of a project. This point emphasises the importance of importance of collating impact after a scholarship project has officially finished.

I once heard it said that it is very difficult to change the world by writing an academic article. I understand impact being all about what you do with either your practice or research findings. A lot of academic effort goes into finding things out and getting articles published in prestigious journals. Impact, in my eyes, is all about enabling findings to facilitate positive and constructive change.

Permalink Add your comment
Share post
Christopher Douce

Ethics support for projects: Which studies need review, by whom and why?

Visible to anyone in the world

On 15 June 2023, I went to another academic professional event. This one was all about ethics and ethical approval. In some respects, it directly follows on from the previous blog, which was all about how to write a 4* research paper.

The session was facilitated by Alison Fox, Chair of Human Research Ethics Committee, and Bart Gamber who is a Student Research Manager for the student research project panel (SRPP). Alison is based in ECYS, the school of Education Childhood Youth and Sport.

The aim of the session was to present an overview of ethics mechanisms for researchers who make use of human participants, and to share something about who, how and why things work. Another aim was to share something about the types of studies that may require ethical approval.

Introducing HREC

HREC is an abbreviation for the Human Research Ethics Committee. We were shown something called the Research Ethics Landing Page, which offered a whole set of links which relate to the different parts of the university which support both ethics and research. An important element of this was a flowchart which highlights what needs ethical review.

Different types of research were mentioned: evaluation research (of services, procedures and policies).; audit research; market research; research. Each of these might necessitate ethical consideration. If OU students are involved with any type of study, this necessitates a referral to the SRPP team. Also, if a study involves more than more than 30 members of staff, researchers must consult the Staff Survey Project Panel (SSPP).

An important point was made about the use of data. Audio recordings that are collected during a study is considered to be personal data, which means that it must be manged carefully. This leads us to consider the different places of support that we may need to draw upon. 

The teams we might consult, include:

  • Data projection team; to gain advice about how to record information assets.
  • Information security team; if wanted to use particular platforms.
  • Library research support team; to consult about how data is stored, how it can be retrieved.

A tip: apply to each of these in parallel.

Two ways to apply to HREC were highlighted. The first was a checklist, which is submitted to the committee. This has a 7 day turnaround time. If your project is a high risk application, a full HREC application can be submitted, then there is a longer 3 week turnaround since it is submitted to two reviewers.

Introducing SRPP

Next up was a summary of the SRPP, which is an abbreviation of the Student Research Project Panel. SRPP is the mechanism used by researchers to gather names and identifies of students you might wish to invite as participants to different kinds of studies. SRPP is important. It is a gatekeeper; you have to go through it to access any of the OU’s students. The reason why it exists is simple: it presents the same students from being approached time and time again. It’s function is pretty simple too: you give it a criteria, and it gives you back identities of those you are allowed to contact.

We were introduced to the SRPP internal website, and shown an online application form. This form contains section where you describe the methodology of your study, a description of the sample of students you wish to involve, a confirmation that you have engaged with HREC, and have taken into account information security, and data protection. 

As the site SRPP site develops, there is the intention of providing examples of previous submissions.

Tip tips

Towards the end of the session, we were given a number of top tips. What follows is an abridged (and edited) version:

  • Start early and expect a dialogue with the teams, and build this into any timescales and plans.
  • Contact teams before submitting applications to get advice.
  • If collecting personal information, you need an asset register.
  • Consider everything from a participants’ perspective; think about their concerns.
  • Consider what data is being collected, how it will be held, destroyed and how consent might be withdrawn.

If you are getting started with a design, visit the research journey landing page for help and guidance.

Undergraduate and postgraduate projects

During this session, then following important point was made: everything described earlier relates to research that is carried out either by doctoral students, or academic staff. 

Students who are carrying out projects that are a part of taught undergraduate and postgraduate qualifications do not need to make HREC submissions or interact with any of the organisational units that were mentioned earlier. This is all to do with insurance.

Students who are completing undergraduate projects, or are writing dissertations that may necessitate carrying out a small study must follow the ethical guidance that is presented within the modules that they are studying. 

Reflections

It’s been a while since I’ve been to a session about research ethics. It is interesting to see how things have developed. Notably, there is a lot more emphasis on securing and holding of data. This is, of course, a very welcome development. The advice “make sure you speak with each unit” early on in the research journey is good sound advice.

One the topic of a theoretical research journal, we were also introduced to a resource that was called “Research Journey”. This resource, a web page, was all about educational research with students, which is where SRPP comes in.

An interesting point that I have noted was: most scholarship projects won’t need to go down the road of HREC, but some academic publishers do require evidence of engagement with ethics panels before research is published. As a reviewer, I certainly look to see the extent to which ethics is mentioned within the articles that are submitted.

The themes and issues that have been mentioned have wider relevance. It was mentioned that both undergraduate and postgraduate students don’t need to submit anything to HREC. Being a tutor on an undergraduate module, where students are asked to consider ethics from a number of different perspectives has made me reflect that perhaps I ought to be sharing a bit more about the kinds of ethical issues they should consider. I feel another blog about TM470 might be needed.

Permalink Add your comment
Share post
Christopher Douce

Writing a 4* paper for the REF

Visible to anyone in the world
Edited by Christopher Douce, Friday, 30 June 2023, 08:26

On 14 June 2023, I attended a professional development event that was all about the previous REF, the next REF, and the sharing of tips about how to prepare what is called a 4* paper. The event was facilitated by Rachael Luck, who is a lead for a design qualification, and Toni Gladding.

The REF is the UK Research Excellence Framework, which is a controversial system that ranks UK universities in terms of the quality of the research that they carry out. It was interesting to hear that both Rachael and Toni were both on review panels for the previous REF.

The most recent REF happened in 2021, and the next one is due in 2028. The broad aim of this session was to begin to think about preparation for the 2028 REF, to begin to consider what a 4* paper looks like, and to have a discussion with colleagues.

What is a 4* paper looks like is, of course, a highly debated and contested question.  A point was: to try to increase quality for next REF, need to be thinking about it, and talking about it early.

The REF score matters. The higher the overall score, the more Quality Related (QR) income (UKRI website) an institution gains from the government. During this session, I learnt there is a multiplier effect, which means that 4* paper attracts 4 times the funding of 3* activity. A small improvement in the number of 4* papers that are published could have a large impact in QR funding. This funding is important since it is used to fund school doctoral studentships and university facilities. This, in turn, can increase the development of the university’s research capacity.

What follows is an edited set of notes I made during the session.

What happened in REF 2021?

University has a code of practice. If someone has a certain amount of research time, has to submit at least one output to REF 2021. There is a long tail of “other colleagues” who were also included. It is the quality of the output from those colleagues who might benefit from additional support. Since the rules for the 2028 REF have not yet been defined (but will be available in January 2024), there is a possibility that a wider group of staff might be included.

In the 2021 REF, 157 universities were assessed. In the OU context (if I’ve noted this down properly), 44% of staff only had 1 output, 10% produced 5 outputs.

What is a 4* output?

Each category of quality assessment has an accompanying “quality descriptor”. A three star paper is considered to be research that “is internationally excellent in terms of originality, significant and rigour but which falls short of the highest standards of excellence”. A 4* bit of research is “beyond internationally excellent”, but I’m not exactly sure what this means.

The university strategy is to maximise QR income, to increase its overall score, to aim to gain higher QR income. A strategy for an individual unit, such as a school, may be to try to gain the highest possible score with a specific assessment area.

An output could be chapters in books, design, exhibitions, journal articles, monographs, performance. This said, the largest and most significant output across universities and all units of assessment (UOA) are journal articles.

Key components of a high quality paper

This section of the session asked the question: what does a 4* paper look like? 

One way to answer this question is to consider what the characteristics of a good journal article might be. A paper should have an eye catching title that has a broad appeal, and a clear abstract which highlights clear research questions, summarises methods, and offers a clear summary of the novel contribution made to knowledge. 

Turning to the body of the article, a paper should clear clearly state research questions, clearly describe the methods that are applied (and share clear citations that offer more detail about them), explain how the research relates to theory, describe research instruments and ethical approach taken, provide details about the data samples, and conclude with acknowledgements. Not having dealt with ethical issues will limit the journals that you can submit to.

An important tip I noted down was: “know about the journal that you’re writing for”. See whether your research or research article is an argument that fits with the aims of the journal. A phrase I noted down was: “papers are not written in the abstract, they’re written with a journal in mind”.

This section of the presentation reminded me of a blog article that I wrote some years back, which had the title Getting published in Open Learning. Open Learning is, of course, a journal that I co-edit with two other colleagues. The points that were made about the clarity of papers is really important. A suggestion for anyone who is interested in writing papers, is to get involved with the peer review process in some capacity. This way you can learn more about how everything works.

During the session, the notion journal impact factors were mentioned, which are known as bibliometrics which highlight how influential a publication is within an area of research. When it comes to the REF, it isn’t just the impact factor of the journal that matters; the assessment of quality of paper is independent of the quality (or impact factor) of the journal. This is linked to something called DORA, which is an abbreviation of: Declaration On Research Assessment (DORA website), which the OU has signed up to, along with other higher education institutions (HEIs). 

What can we change?

Research can be thought of in different ways. Research is, of course, carried out by individual researchers and academics, but since much research (especially within STEM require collaboration, a wider perspective is necessary. Individual academics should develop their own research strategies (which could be shared with others), but also strategies should be defined by units, such as schools, which then should be linked to REF units of assessment. This also links to the importance of developing a research culture.

An interesting question was: what practical things could be done?  Some suggestions include working with others to ensure thar articles are thoroughly proofread before submitting to journals, perhaps setting up internal peer review processes within research groups, writing articles with co-authors, and setting up mentoring schemes with colleagues who are new to research.

The discussion of papers led onto a discussion about the challenges that accompany getting articles published in open access journals. Open access publishing means that the readers do not pay for access to research articles. Instead, researchers and writers pay. This means that authors may be asked to pay what can amount to a substantial publishing fee. This can limit the extent to which researchers are able to publish unfunded research, such as personal projects.

A suggestion is to always consider the cost of publishing fees when writing and preparing research bids, to make sure that they are considered during the budgeting. Also, if research is funded by a UK research funding council, the university library has an agreement that they will cover publication fees. 

By way of contrast, Open Learning offers a hybrid scheme. The editors of Open Learning chooses to ‘open access’ one article per issue. Individual authors can also secure open access by submitting what amounts to a publication fee, but it is typically less than journals that adopt a completely open access funding model. The advantage of this approach is that it enables institutions that do not have extensive research budgets to disseminate their research. An important point here is: pay attention to the terms and conditions that relate to the journal that you’re hoping to publish through.

One of the final comments I noted was: there are tensions between helping those currently work on 3* papers, to help them to submit 4* paper, and helping those with little research time to start publishing. A big win for the university might be to enable a wider group of staff to be included in the REF. This also links to the tension between supporting the individual researcher, and supporting the institution as a whole to gain an overall higher rating.

Reflections

I don’t like the REF for the simple reason that terms such as quality and excellence are dangerously slippery terms. Also, research is something that takes place over an extended period of time, and certainly not across a neatly defined seven year cycle. I’m also grumpy after learning that 4* papers have a multiplier effect on research income for the simple reason that universities, schools and individuals who are already doing excellent research are likely to gain even more income to do even better research, with more resources. This multiplication factor is not progressive, and it feels far from equitable. Institutions with a lower research ranking will, of course, struggle more to make an impact in comparison to, say, Russell Group universities.

I come to this topic having struggled to build a research strategy and profile. As a staff tutor, half of my time is dedicated to academic leadership and management, and the other half is dedicated to ‘academic stuff’, which means either working on module teams or working on research. If I have module team (teaching) responsibilities, this means that less of my time is available to research, and what really helps with research is building those close collaboration with colleagues, and all this takes time.

Over the last few years, and to this day, a lot of my time has been spent dealing with internal institutional changes (mainly, the new AL contract), of which a lot still remains to be figured out and understood. Whilst this does sound a little grumpy, dealing with the practicalities of my staff tutor day job leaves very little practical time to carry out disciplinary research. Whilst I’m not contractually obliged to make a REF submission, the idea of leading a submission of an article feels current unattainable, never mind the notion of producing an article that is “beyond internationally excellent”.

What are the answers? One of the obvious answers is to work on research culture. This requires the involvement of multiple stakeholders, at multiple levels. It means facilitating conversations, and learning about the interests of others. It means getting involved and encouraging others to get involved. It means joining research groups, and being patient when listening to the perspectives, challenges, and experiences of others. This is also, of course, made harder when everyone is working at a distance.

Two thoughts that I return to regularly are as follows. The first thought come from the words of a former head of school who said: “if you have two reasons to do something, there is a greater likelihood that you’ll get that thing done”. Following this though, the accompanying thought is: “try to align everything together, into a personal strategy”. It is this alignment that I’m trying to do, and this may well mean saying “no” to some things.

Permalink Add your comment
Share post
Christopher Douce

Computing and Communications: 2023 Research Fiesta

Visible to anyone in the world
Edited by Christopher Douce, Tuesday, 31 Jan 2023, 17:02

On 25 January 2022 I attended an event called the School of Computing and Communications Research Fiesta, which took place on the university campus. One of my reasons for attending the fiesta was to try to restart my research activities, having stepped away from research due to taking on a role called ‘lead staff tutor’ for the last three years. 

The last time I attended a school research fiesta was on 10 January 2019 (OU blog) which took place at the nearby Kents Hill conference centre. Following this earlier event, I shared an accompanying post about research funding (OU blog).

This event was advertised as a “… time for us to reconvene and discuss everything research. This event is aimed to help us (re-)connect with one another and understand how we can help and benefit from each other’s research expertise and outputs” and was facilitated by David Bush from Ascolto.

What follows is a summary of the Research Fiesta, in terms of what happened during the meeting, and what I felt the biggest take away points were. This blog may of be interest to anyone who was at the event, couldn’t make it to the event, or broadly interested in the process of research (whether computing research, or research that takes place within other disciplines).

Preparation

Before the event, we were asked to prepare some cards which summarised our research interests. Although I didn’t write the card in advance, I did come to the event with some ideas in mind. Here’s what I wrote down on three cards:

  • Understanding and characterising green computing: what it is, what the boundaries and problem are, and how can we embed this theme into our teaching?
  • Storytelling, soft skills, and software engineering: what role does storytelling play or could play in software engineering practice, and how might storytelling be used to develop soft skills in the next generation of computing graduates?
  • Accessibility of web technologies: how accessible are the current generation of web-based applications, and to what extent are hybrid apps accessible with assistive technology. How useful is WAI-ARIA? It is still useful? Does it have an impact?

Later during the session, I added two more cards:

  • Pedagogy of teaching programming at a distance: innovative tutorials; how to develop tutors, and how to help them to be creative, perhaps by embedding and using drama.
  • Development of writing skills across the computing curriculum. 

This final idea emerged from discussions with tutors, following some discussions with tutors, and might form the basis of a scholarship project. The university has prepared a lot of materials about writing; the question is whether the computing programme makes effective use of them, given the writing requirements from some courses.

Activity 1: Sharing research ideas

Our first activity could be loosely called “academic speed dating”. 

I’ve done this before (both the academic version, and the non-academic version). 

In this version, we were sent to various tables, where we met up with two other colleagues. Our task was to show our cards (our research ideas) and try to create a new card that combined aspects of all of our cards. When we had done this, we had to pin our cards onto the wall to share our ideas with everyone.

Activity 2: Forming research teams

After a short break, everyone was asked to form a line based on how much research experience everyone has. On one side, there were all the new PhD students, and on the other side, there were the professors and heads of existing research groups.

Approximately 6 PhD students and early researchers were asked to review the cards that had been generated from the speed dating activity, and each had to choose a card they found most interesting.  This card (represented by one of the researchers) would then form the basis of a new team of 3 or 4 researchers.

One at a time, the rest of the researchers were ushered over to speak with the new researchers. If you liked an idea, and there wasn’t already 3 or 4 researchers, you could join a team. The longer the game went on, the harder it becomes for the more experienced researchers. Instead, they would have to make use of all the powers of persuasion to try to join an existing team, or to persuade fellow researchers to create new teams.

After some discussion and reviewing cards, I joined two of my colleagues, Dhouha Kbaier and Yaw Buadu. Two project cards were combined together to create a new project. Paraphrasing our cards, our project intended to:

Develop digital technologies to enhance engagement and participation by integrating more physical computing into the computing curriculum. 

Accompanying research questions were: what are the challenges of using physical computing in a distance learning environment, and how might physical computing devices be connected to and integrated within the Open STEM labs

This final question suggests the opportunity to explore costs and trade-offs of a physical computing approach where students use their own equipment, or share equipment with other students through a platform which is accessed remotely.

What might physical computing actually mean? One answer to this is: physical hardware used by students to learn about or to solve computing problems, as opposed to using software simulations. There is a precedent of using (and sharing) physical computing devices at the university. In earlier decades, there was the Hektor computer (computinghistory.org.uk), which was once sent out to computing students (and then later returned to the university).

A more modern and smaller (and much more sophisticated) version is the Raspberry Pi computer (Raspberry Pi website) which can be used with any number of interesting computing projects.

One other aspect that we discussed were about the stakeholders, and who might need to be involved? We identified the following groups: students, tutors, module team members, and administrative university functions.  (The members of module team may include both tutors and curriculum managers, who act as a fundamental link between the academic team and operations of the university bureaucracy). 

Impact: evaluation and presentation

The next bit of the fiesta was a presentation; a double act from two colleagues from the research school, Betul Khalil, an Impact Manager, and Gareth Davies, who is a Research Evidence Impact Manager. 

They began with a question: what is impact, and can we give an example? 

Impact isn’t the same as project outcomes. They are very different things. An outcome might be a report, or some software. An impact can refer to a change that may have led to a positive long term benefit to stakeholders. In terms of the UK Research Excellent Framework (REF Impact case studies), impact could mean a change to society, the economy, and to the natural environment. Also, a measurable change might be on a local, regional or international scale.

The message to us was clear: when working on a project bid, researchers need to proactively consider impact from the outset and define impact objectives, since gathering effective evidence to show how those objectives may have been met takes time. In some respects, impact evidence gathering is a further part of the research process.  To do it well, researchers need an impact plan to accompany a research or project plan. 

We were all given a handout, from which I have noted down some useful questions that researchers need to bear in mind. These are: 

  • Who are the stakeholders, and who might be affected by the change your project may facilitate?
  • What do the stakeholders (or beneficiaries) gain from your research?
  • Why will they engage with your research?
  • How will you communicate with beneficiaries?
  • What activities might you need to run to effect change?
  • How might you evidence change? 
  • How will you connect change to your research?

Later, Gareth talked more about what it means to ‘evidence’ impact. An important note I made from Gareth’s presentation was that “upsteam planning is important” and that the analysis of impact should be rigorous. Researchers also need to consider which methods they use to enable them to find a way to observe what is changing. 

Apparently, one of the most common forms of evidence is a written testimonial (in the form of a testimonial letter). Within this assertion lies the reflection that researchers need to make sure they have the time and the means to gather evidence.

Activity 3: How will we do our project?

Our next activity was to sit around a table to figure out how were going to do to answer our research questions.

We began by asking: what might the outputs from our project be? We came up with some rough answers, which were:

  • Guidelines about how physical computing could be embedded and used within module teams. If used within a module, tutors could then be offered some accompanying guidance.
  • Recommendations about physical kit that could be used (these kits might be bought, or borrowed, or used from a distance); recommendations about the use of software; recommendations about pedagogy and use (which is an idea that can relate to the idea of useful guidelines). 

To produce these, what needs to be done? Our team offered the following suggestions (but the exact order of carrying these out could be easily debated):

  1. Examine learning outcomes within various qualifications and accompanying modules.
  2. Explore the problem space running focus groups with stakeholders to understand how the terms engagement and participation are understood.
  3. Use mixed methods: from the focus group results, carry out a survey to more thoroughly understand how a wider population understands engagement and participation.
  4. From these different information sources (and input from the learning outcomes) facilitate a number of curriculum design workshops to understand how physical computing can be brought into the curriculum.
  5. Carry out a detailed analysis of all the data that has been captured, writing up all the findings.
  6. Implement the findings.

A further reflection was each of these activities needs to be considered in terms of SMART objectives; specific, measurable, achievable, realistic and timebound. 

A new question that we were asked was: what impact will your project have? 

Given that students are key stakeholders, there might be broader impacts in terms of results to the National Students Survey. There might be further impacts both within the university, and to other organisations that provide distance learning. There might also be impacts that could be broadly described as the further development of computing pedagogy. This is all very well, but how might we go about measuring all this? It is this question which the facilitators from the research school may have wanted to encourage us to consider.

What happens next?

 After presenting our plan to all the other groups, we were asked a couple of final questions, which were: how excited are you about the project? Also, how doable (or realistic) is the project?

Given that we all have our own main research interests (which are slightly different to the new project that we have defined), we all had different levels of enthusiasm about going ahead with this project idea. That said, the key concepts of physical computing (in its broadest sense) and student engagement are important topics which other researchers may well be interested in exploring. Even if this particular team may not be in a position to take these ideas forward, the ideas are still worth exploring and studying.

Reflections

I really liked the way that we were asked to focus on trying to get things done. 

When thinking about research (and research projects) impact has always been something that has always been at the back of my mind, but I’ve always tended to consider it as something that is quite intangible and difficult to measure. The presenters from the research school made a really clear point. They emphasised that it is important to plan for impact before your project has started.

A personal reflection is that impact could be thought of as a way to reflect on the success of a project. In some respects, this should be something that researchers should be doing as a matter of course to further develop their professional skills. Of course, the extent and nature of this analysis will depend very much on the nature of the research that is carried out through a project. Given the collaborative nature of research, gathering of impact evidence is likely to be collaborative too. 

It is interesting to compare this Research Fiesta with the one that was held in 2019. One of the differences being that there were a lot fewer people attending this event. This might have been a factor due to the timing (some new module presentations were just about to begin) or a hangover from the 2019-20 pandemic (where so many colleagues switched to homeworking). 

An interesting difference related to the structure: this event was facilitated in a dynamic way, where the research themes emerged from the participants. The earlier event had more emphasis on sharing information about the research groups within the school, and more of the practicalities about how to gain funding for research. There is, of course, no right or wrong way to run a research fiesta. I appreciated the dynamic structure, but equally I’m always up for hearing about new concepts and ideas, and learning about what is happening within and across the school.

Acknowledgements

Many thanks to Amel Bennaceur for organising the event. One of the impacts has been to get to catch up with colleagues, and to learn more about them! It was a pleasure working with my fellow group members, Dhouha Kbaier and Yaw Buadu who kindly reviewed this blog article before it was published.

Permalink Add your comment
Share post
Christopher Douce

STEM Education Research Group - Mixed Methods

Visible to anyone in the world
Edited by Christopher Douce, Thursday, 26 Jan 2023, 15:48

I belong to a couple of research groups within the School of Computing and Communications; I’m trying to find my research home, after not being involved with research for a while. There’s also an informal group called the STEM Education Research Group, which explores topics that are common to some of the groups that I (occasionally) visit.

On 18 January 2023 I attended a research development event that was facilitated by Ann Grand, Senior Lecturer in Astrobiology Education, that was all about mixed methods.

An example

Ann opened with an example, which was also a research question: how are people using their allotments?

You might count how many people are growing different type of crops, or how many hours a week people are ‘using’ their allotments, or you might try to understand ‘why’ they are using their allotment. The nature of the research question might lead to you choosing different methods: you might gather numbers, or you might want to speak with people who grow things on their allotments. I made a note that there’s a difference between multi-methods to answer different research questions, and mixing of methods.

Choosing methods

I noted down a reference to Tashakari and Cresswell, where mixed methods were described as: “research in which the investigator collects and analyzes data, integrates the findings, and draws inferences using both qualitative and quantitative approaches or methods in a single study or a program of inquiry. integrate everything to produce an interpretation” (2007, p.4)

An important question is: how do decide about to use which methods to use? The answer is: It relates to the overall design of what is being studied.

An important point that I noted down was that mixed methods can often take up more time than if a researcher was only using a single method. This leads to the question: under what circumstances should we use them? What is their value?

A reflection that was made during the session is that controlling for variables in education is profoundly difficult, and therefore, it is almost inevitable that we adopt mixed methods to try to understand what the variables might be. They might also be used to mitigate against the impact of extraneous variables. Also being aware of a range of different evidence may enable you to often understand the question, before even carrying out your research.

My colleague Oli Howson made the following point: “quantitative data is lovely for drawling lines around things but humans are messy and colour/context is important”. Understanding the context can, of course which can lead to other (or related) research questions. A research project might not be about asking or understanding a sequence of questions, it may be more of a messy network of questions which exist within a wider research space.

Value of mixed methods research

Mixed methods can be used to investigate and consider bias, and add meaning to data that has been gathered. One useful quote is by Denzin, who writes: “the bias inherent in any particular data source, investigators and particularly method, will be canceled out when used in conjunction with other data sources, investigation, and methods” (p.14, 1978).

Another quote relates to the application of mixed methods, namely that a mixed approach “facilitates generalization to a wider population, especially when the qualitative sample is directly linked to the quantitative sample” (Hesse-Biber & Johnson, 2015).

There is also the importance of being aware of our own biases and being mindful or how we approach any analysis. These points are related to the subject of reflexivity, which relates to how we position ourselves in relation to any research that we do. Sometimes sharing a little bit more about us (and our position) enables us to add validity to any research that we share.

More than methods…

An important reflection is that the choice of methods is one bit of a much broader picture. Our choice of methods reflects what our research paradigm is, and can link to our philosophy of how we view truth and knowledge. In some ways, using two different sets of methods can be an attempt to bridge conceptual differences between interpretivist and positivist world views. In other words, whether truth (or reality) is subjective, or objective.

The ordering of methods is important. A researcher might use a sequential approach, applying one method after another. 

A qualitative method might be used to identify concerns held by a community, which could then be brought into a survey method to quantify, or to test the extent of concerns that might be held by a wider community. In other words, a quantitative approach could be used to validate a qualitative finding.

Looking from the other perspective, a survey (perhaps using a standard instrument) might signify some interesting or curious results. Qualitative methods could then be used to explore why a certain group of participants hold a particular perspective. In other words, a qualitative approach can be used to provide explanations to accompany quantitative findings.

During this part of the session, there was also a short discussion about the use of literature surveys. Systematic reviews can apply mixed methods. For example, a literature survey could begin with a set of themes that have been identified by a researcher. This identification of themes could be thought of as a qualitative approach. During the next step, a researcher might then to go onto identity how many papers explore or study those themes.

A further example…

Towards the end of the session, we had a look at an example of some educational research which asked the question: what impact do science shows have on attitudes to career intentions? 

I understand a science show to be an engaging demonstration or a talk. In other words, is there an effect on the career aspiration of school children who attended those shows? We had a brief look at some data captured from a research student. This included responses from questionnaires, and responses from a focus group. 

Whilst discussing the research methods applied in this study, there was a further discussion point that emerged, which was about the concept of impact. Specifically, how does ‘impact’ relate to your research questions?

Reflections

Whenever research methods are discussed, there are other broader questions which can and should be asked. These questions relate to the philosophy of research, and the nature of truth, and these discussions inform the research paradigm that you adopt. Before even getting into philosophy and paradigms, it is your research questions that should drive everything. When you know the what needs to be found out, you can then think about the how.

It was great to see Creswell mentioned. I first came across his textbook when studying for my MA in Education. Creswell presents a really detailed summary of what mixed methods research is all about and provides a lot of detail about the methods that can be used and applied.

One of the unexpected points that I took away from this session is how systematic literature reviews can both draw on the quantitative and the qualitive. Thinking back to the literature reviews that I did for my MA and other qualifications, it has struck me that I’ve been applying mixed methods research, but in an informal way. Knowing about terminology makes the informal become formal, and also goes a long way to clarifying processes and how this relates to how research is carried out.

References

Denzin, N. (1978) The research act: A theoretical introduction to sociological methods. New York: Praeger.

Hesse-Biber , S. and Johnson, R. (eds) (2015) The Oxford handbook of multimethod and mixed methods research inquiry. Oxford University Press.

Tashakkori, A. and Creswell, J. (2007) Editorial: The New Era of Mixed Methods Journal of Mixed Methods Research 1(1)3 DOI:10.1177/2345678906293042

Acknowledgements

Some of the phrases and quotes shared through this blog have directly come from Ann, who kindly shared her slides following the event. In some of the earlier sections, I’ve added some further points and reflections from other periods of study. Many thanks to Ann for running a useful session.

Permalink Add your comment
Share post
Christopher Douce

PhD project: The role of storytelling in software engineering practice

Visible to anyone in the world
Edited by Christopher Douce, Friday, 23 Dec 2022, 11:06

Here’s a very short story.

One day, when I was a computing undergraduate in the 1990s, I had to do a high stakes programming assessment. With about twenty of my peers, I went into a computing lab, where we were all give a programming task to complete. We had to write code to solve a problem, and then get a printout of the program and all the test data we had used.

I remember that it took me a couple of hours. 

Halfway through the activity, one of my peers stood up, said he had finished and proudly announced he was going to the student union bar (it was 11:00am in the morning; we didn’t have any other classes for that day). When I finally got all my code and tests printed and submitted, I noticed there were some students who were still working on their problems.

In this moment, I asked myself a question which plagued me: how come some people find programming really easy, and other people sometimes struggle? Is there something special about coding? 

I took this question with me to my postgraduate studies, and then onto doctoral studies, where I learnt about cognitive psychology and working memory. My focus on the individual programmer and their capabilities led me to create a whole new type of software metrics.

After doing all this study, I got a job as a professional software engineer. I was keen to gain some industrial experience since if I were ever to return to the higher education sector to teach programming, having some real programming experience would give be a bit of credibility.

One of the most important things I learnt in industry was that whilst the individual programmer and their abilities is important, software engineering is a team game.

Although I’m interested in software, I’m more interested in people. I see software and computing as a tool through which we can understand more about ourselves; I see the machine as a mirror in which we can see more of ourselves.

Software and software engineering has taken me on a journey. It is a journey that began with one question and has ended with another, which is: “since communication is so important in software engineering, could the idea of story telling be useful?” I’ve been on a journey that once focussed on the individual, and have moved to a place where I’m now interested in groups of people and how they work together.

Topic Description

Software is an invisible technology created by people. To design and build software, communication is a necessity. Software engineers must communicate with stakeholders to gather requirements, they must communicate with each other during the development process, and then they must communicate with the stakeholders when software is deployed.

Since software development and engineering is a human-centred activity and communication is both a necessity and imperative, one of the tools that could be used and applied by software engineers is storytelling. The aim of this project is to uncover the ways in which storytelling practice can be either discovered or used within software development communities.

Storytelling and software engineering can be considered in different ways. It could be used to help to facilitate to the discovery of requirements, help to share professional expertise between developers, but it could also be used as a way to develop the communication skills and practices of the next generation of software engineering professionals, helping to address a perceived soft skills gap amongst computer science graduates.

In this project, you will carry out background research to identify how and where storytelling can be used to either develop, understand or enhance software engineering practice. You might be required to design and carry out empirical studies that explore software develop cultures or evaluate new and innovative methods or practices.

Skills Required

Although you may have a first degree in Computer Science or Software Engineering, this project may be suitable for someone who has studied humanities and social science subjects and have completed a postgraduate conversion degree in Computing or a closely related subject. 

Ideally, you should have a firm understanding and appreciation of quantitative and qualitative social science research methods. You should also be willing to study topics and subject that relate to the humanities, and be willing to explore different conceptions of employability within the discipline of computing.

Background Reading

Iniesto, F., Sargent, J., Rienties, B., Llorens, A., Adam, A., Herodotou, C., Ferguson, R. and Muccini, H. (2021) When industry meets Education 4.0: What do Computer Science companies need from Higher Education? Ninth International Conference on Technological Ecosystems for Enhancing Multiculturality (TEEM’21), October 26–29, 2021, Barcelona, Spain. ACM.

Rainer, A. (2021) Storytelling in human–centric software engineering research. EASE 2021: Evaluation and Assessment in Software Engineering, June 2021, 241–246.

Schwabe, G., Richter, A. and Wende, E. (2019) Special issue on storytelling and information systems. Information Systems Journal, vol. 29, no. 6, 1122–1125.

Storr, W. (2020). The Science of Storytelling: Why Stories Make Us Human and how to Tell Them Better. Abrams.

Closing points

If this very broad sketch of a project sounds interesting as a doctoral research project, do feel free to get in touch. 

Alternatively, if you’re a computing researcher looking at a similar subject, feel free to drop me a line; it would be great to her from you - let's find a way to collaborate.

Permalink Add your comment
Share post
Christopher Douce

Doctoral research: a short introduction

Visible to anyone in the world
Edited by Christopher Douce, Wednesday, 26 Oct 2022, 17:22

This blog is about doctoral research, a little bit of what it entails, and the different routes that are available to students who are studying in the UK. This post might also be useful for international students too.

This post begins by asking the question: “what is a doctorate?” It then goes onto describe two different routes to doctoral study: a disciplinary route and a professional route. This is followed by a very broad sketch of what doctoral research involves. 

The post concludes by sharing some of my own experiences, and offers a summary, which includes some links to some useful resources.

One thing I should add is that I don’t work for the university graduate school, but I do supervise some doctoral students. Do always check with the OU graduate school if you need further information, or the equivalent unit that is likely to exist within your own institution.

The fundamental questions

What is a doctorate and why would I want one?

A doctorate says that you have done, and are capable of carrying out original research. It also says that you have been trained to carry out research, and you are capable of advanced critical thinking. 

A doctorate is also something that can be useful if you would like to have a career in academia. Whilst it can be considered to be useful, you can, of course, still be a lecturer, and still carry out substantial research without having a doctorate. 

A related question is: will it get me a higher salary? My answer is: don’t do it for the money; do it for your subject, and also do it for yourself. 

Another answer to this question is: it all depends. It depends on the discipline, and also depends on the job opportunities that are available. Academia is notoriously and brutally competitive, and there are never any guarantees.  

Another question to ask is: would you be prepared to work for at least 4 years on a low income? During that time, your peers may well become established in parallel careers, and have spent that time continually increasing their earning potential. As mentioned above: do it for the subject, not for the money.

Do you need a masters?

For the OU professional doctorates, applicants should “normally hold, or be expecting to obtain before the start of the degree”. I’ll say something more about professional doctorate a bit later. For disciplinary doctorates, you don’t necessarily need have to have one, but they can certainly help. It may depend on the subject and the institution that you're applying to.

Do you need to be super smart?

I used to think of people who held doctorates as being a whole other species of human. I remember my chemistry teacher at school. He had an air of cleverness about him. He regularly wore a white chemist’s coat. I assumed that everyone who was called doctor was super smart. After a fortuitous sequence circumstances, I found myself having “done some stuff” and “having discovered” some things that were deemed to be suitably original enough to be given a doctorate. 

The thing is, I’m not super smart. 

What I would say is that I was passionate and interested in what I was doing to be able to find sufficient determination (and time) to really focus on a narrow area of study. Being smart is important (as is being humble), but determination matters more. You must be motivated, and maintaining motivation over an extended period of time isn’t easy.

How big is a contribution?

Doctoral research is all about carrying out original research, which broadly means discovering something new that no one had ever discovered before. This sounds like a big deal. Another fallacy that I had when I was a kid was the view that these “doctors” must have discovered something huge during their studies; something that could change the world or the course of history.

The reality is, conversely, a lot more mundane. 

One way to think of academia is to think of it as a community in which the academics contribute to a huge set of on-going debates. Everything is mostly very polite since the academics argue with each other, through the medium of academic articles and formal presentations. Academics might, for example, argue about the role and importance of the topics that make up their discipline. Doctoral students learn how to contribute to that massive debate; they’re elbowing their way in, to say: “hey, have you thought about, looked at, or seen this?”

The contribution to one of these debates can, in fact, be really small, but it can still represent a contribution.

I am a big fan of social science methods, particularly ethnography. Ethnography is all about writing about people and communities. Ethnographers write about what people do, and how their communities operate. A doctorate in the social sciences which applies ethnography might study what happens in a particular community over a period of time. Communities (and cultures) come and go, and are influenced by the events and circumstances that surround them. The very act of writing and describing a potentially short lived community represents a contribution, which other academics can look to, study and examine. In sharing your contribution, you contribute to wider debates about societies and how they work.

In computing, my home discipline, researchers might go about building software, or combining new bits of software in a unique way to demonstrate a new concept or idea. A new software tool might be a very modest contribution, but someone else might pick it up and take it into a whole new creative direction. 

Although the phrase “making an original contribution to human knowledge” sounds pretty intimidating and very grand, the contributions that doctoral researchers make can be modest. This said, some doctoral students can also be fundamental in facilitating breakthroughs. Also, it isn’t just the output from a doctorate that is important; the process is important too.

Types of doctorates

Within the OU (and other institutions) there are, broadly, two types of doctorates: disciplinary doctorates, and professional doctorates. I’ll begin with disciplinary doctorates.

Disciplinary doctorates

Disciplinary doctorates, simply, are doctorates that take place within a discipline! There are a number of routes to a disciplinary doctorate. These differ in terms of how the research question, or problem. A disciplinary doctorate might begin with a specific problem that needs to be solved, or it might begin with a research question from a student.

Doctoral research roles

Some doctoral students may carry out research as a part of an established funded research project or programme. In some ways, gaining a doctorate this way is a bit like having a job. Programmes of this kind are usually full-time, where students get paid a modest salary (or stipend), rather than having to pay the university for registration and supervision fees.

The funding for these opportunities might come from a funding council (or research funding body), which has decided to fund a project that has been proposed by a professor or a team of academics. 

Alternatively, the funding for some doctoral jobs may come from industry. In these cases, a company or business might have a very particular research and development problem that may have never been solved before, and one way to solve it would be to set up a project which may involve doctoral students. The outcomes from the project would give the business an insight into how to solve a problem, and give a doctoral student experience of carrying out research into a technical domain, and writing a thesis.

In the UK context many of these research opportunities are advertised on a well-known academic jobs board (Jobs.ac.uk). It is quite interesting and useful to have a look at some of these to see what kinds of qualifications, experience and characteristics research groups are looking for. This board also sometimes advertises opportunities in other countries too. When I last had a look I saw positions available in Sweden, Germany and Hong Kong. For an even broader international perspective, another site that is worth visiting is Find a PhD (website). 

Doctoral scholarships

Some of the roles that you may see on those PhD job board may be quite varied. You might see positions that address a very specific research problem. On the other hand, you might sometimes see scholarships which are more loosely to a subject or a topic area.

The school in which I am affiliated with advertises a couple of PhD scholarships per year. Whilst some of these scholarships might be connected to certain industrially funded projects, the school also advertises a list of research topics (OU School of Computing and Communications). 

It is also worth looking at how a university structures their doctoral research programmes. Through wider funding schemes, which are aimed at certain subject areas and facilitating collaboration, there is also something called doctoral training partnerships (OU website) which is also worth looking at.

Choosing your own research path

Sometimes you might have a disciplinary research idea or an interest that is entirely legitimate, but doesn’t immediately fit with any advertised funded PhD role or scholarship that is currently being advertised. If this is the case, you still may well be able to become a doctoral student, but you may have to handle the financial bit from your side. 

The way I understand things, there are two broad approaches: you can either find a source of funding yourself, or you can pay your own registration fees from your own pocket, or through a doctoral loan (GOV.UK website).

I’ve recently heard of something called Commonwealth PhD Scholarships for those looking to study at UK universities. Also, individual universities, such as the OU, Kings and UCL sometimes offer scholarships for students from minority backgrounds.

Gaining funding is only a part of the story. The other part is, of course, developing an idea. A suggestion is to draft a short proposal, and then look for a supervisor: someone who shares similar research interests.

Begin with the research questions, and ask yourself: what is it that you would like to find out. Also, start to find out, using any academic library you may have access to, whether anyone has tried to answer this question before. Doing this might, potentially, lead you towards an institution and a supervisor.

If you do decide to go down this route, there are other questions that you need to answer. Are you looking to do it full time, or part time? 

In the OU context, there is a bit of advice about part-time doctoral study (OU website). Also, do have a look at the fees, and ask the questions: can I really afford this, and am I really in a place where I can generate the determination required to just focus on one thing for anything between 3 and 6 years (depending on the doctoral programme, and the intensity of your research)?

Professional doctorates

Professional doctorates are slightly different to disciplinary doctorates. The OU supports the delivery of two professional doctorates: a doctorate in Education (EdD), and a doctorate in Health and Social Care (DHSC) (OU website).

These are described as follows: “a professional doctorate provides the opportunity for you to develop your own practice-based research in a structured and supportive environment. A professional doctorate differs from a PhD in that one of its key aims is to make a contribution to practice or policy, as well as to theory.” A big difference to the disciplinary doctorate is that you’re already likely to be working in the setting that will play an influence in guiding and informing your research. In other words, “they offer you the chance to enhance your career at doctoral level, enabling you to make a unique contribution to your profession or area of practice while continuing to work and progress in your field.” (EdD/DHSC website).

The EdD programme is described as being appropriate for “professionals in education, including school leaders, teachers and trainers, but also other professionals working in any educational context in formal and non-formal settings including the public, voluntary and private sectors.”

Working within a particular setting is important, since it provides you with a context which can be explored and studied. Every educational situation is different, and this means that there is an opportunity for EdD students to find out something about it, and how it works, and the kind of educational activities which might, potentially, make a positive difference to learners or those involved in delivering education. Finding out something about your own context in a systematic and rigorous, and academically respected way enables researchers to contribute to educational academic practice and debate.

In the OU, there is quite a difference between what happens within a disciplinary PhD and an EdD. Within a PhD, students are left at the mercy of their supervisors, in the sense that they will help them to gain an understanding of what needs to be done to learn about how to do research within their particular discipline. 

The EdD, on the other hand, has a structured taught component, which helps students become aware of the different stages of academic research. This component will introduce students to the importance of research questions, the literature review, and introduce important terms, such as epistemology, ontology, and methodology. For a detailed description of what is entailed in EdD study and research, the blog post about the Components of the EdD Professional Doctorate Programme may be useful. It is also typically expected that students should have completed an MA in Education, which may have helped to explore some early research questions.

As a brief aside, the university employs associate lecturers (who are, arguably, the most important people in the university) who deliver tutorials and provide correspondence teaching. The university provides something called a module fee waiver scheme for associate lecturers, which could be used on doctoral programmes, such as the EdD. If you are an associate lecturer, and are reading this blog, and have sometimes wondered about doctoral study, do have a chat with your friendly staff tutor.

One point that is common between a disciplinary doctorate and a professional doctorate is that you need to have a clear research idea, ideally presented in the form of one or more research questions. The more specific they are, the better. It isn’t enough to say that you’re interested in doing research into a particular area: you need to be specific about what you’re going to be looking at, and have some beginning of an idea about how you might do that.

To get onto the EdD programme, you need to write a short proposal, which will then be scrutinised by a small group of potential supervisors. Those students who have written proposals that look promising will then be invited to take place in a short interview. Typically, this will be with one of the potential supervisors.

When it comes to doctorates, students find their supervisors, but on other occasions, supervisors find their students. What everyone has in common is interests in the subject, and the process of carrying out research.

PhD by Published Work

There is a final route to gaining a doctorate, and one that isn’t as common: gaining a doctorate through publication. 

This is sometimes appropriate in cases for academic staff who may have already carried out considerable amount of research over an extended period of time, and just never been in a position to enrol to a doctoral programme. 

Through this path, a body of work may be collated together, and submitted, along with a narrative that presents each of the publications (or constituent) papers as a cohesive whole. 

The act of getting published, and engaging completely with the research process can serve as significant evidence of having worked at a doctoral level. Like with other forms of doctorate, candidates who choose this approach also have go through the viva process.

In my own experience, I don’t personally know of anyone who has gone through this route, but I do know that it exists! There is a bit more information about this approach on Find a PhD.

Doing a PhD

What everyone does on a day-to-day basis is, of course, different. 

There’s going to be reading, attending of seminars (to get an idea of how everything in the academy works), perhaps doing some lab work, maybe doing some field work, perhaps even interviewing people and collecting data. In other context, you might be writing some computer code or managing data files. Essentially, you’ll be applying whatever tools you have in your own discipline to answer your research questions.

In all of this, you’ll gain skills: you’ll develop your critical thinking skills, your writing skills, and your presentation skills. 

Posters and presentations

A lot of sharing takes place at disciplinary conferences or workshops. These are great opportunities to share your work with an interested audience and to meet with academics and students who are studying a similar subject. When you’re a PhD or EdD student, there might only be a couple of people studying the same subject that you are studying. Conferences and workshops are a useful opportunity to seek those people out and network with them. If you’re looking towards a career in academic, conferences and workshops are a really good place to find potential future collaborators.

Before you get to a point where you share your results, doctoral students are sometimes able to submit what is known as a poster. A poster is exactly what you imagine it to be: it is a poster that summarises your research aims and intention. During the breaks during a conference, delegates may wander up to your poster to find out more. This is a great opportunity to share an elevator pitch about your research. 

Publishing

I’ve heard it said that a very good master’s degree project should be at a level that a version of it could be theoretically published as an academic article. The difference between a master’s and a doctorate is that of originality. When it comes to doctoral research it is a good idea to always have one eye on publication, in terms of what you might publish, and where you might publish it. After having carried out a literature review, you should have some idea about where you might be able to share your research findings.

Going through the experience of writing, submitting, and reviewing a formal article, and being able to contribute to the ongoing academic debates within your area is a part of the doctoral training experience. Although it is possible to gain a doctorate without publishing a journal article, publications certainly help. It tells the examiners that other experts (through the peer review process) have assessed the quality of your work.

Writing your thesis

The thesis is one of the most important products of doctoral research. The thesis summarises the aims of your research, the reading you have done, the methodological approaches you have adopted, and is used to present your results, and should be no longer than 100k words. In contrast, my MA dissertation was limited to 12k words.

The OU has something called a ‘writing up’ year which some students may use, during which students may pay a reduced fee. Students must submit their thesis on time. When a submission has been made, the university graduate school will organise a viva.

Viva

A viva is an oral exam. It is a bit like a really intense interview, where the subject of the interview is the research that you have carried out. There are likely to be two external examiners, and a chair. One of your supervisors is likely to be present. You’re likely to know, in advance, who the examiners are, and may well have referenced some of their work in their thesis. There is a nice article in Prospects Magazine: Five tips for passing your PhD viva.

There are a number of outcomes following the Viva, ranging from passing without changes, through to different amounts of changes that may be necessary. A supervisor will only let a student get to the viva stage if they are confident about a positive outcome.

After the doctorate

Assuming that you’ve passed, and you’ve graduated, what next?

As mentioned earlier, academia is notoriously competitive. A doctorate is an indication that you’re capable of carrying out original research. To gain experience, and to secure an academic job, doctoral researchers sometimes look for post-doctoral research posts. These are often connected to specific research projects or programmes, which may have been set up by professors or lecturers with funding gained from research councils or funding bodies.

A personal perspective

My doctorate is in an area which could be loosely called The Psychology of Computer Programming. 

Whilst I was an undergraduate, I was really interested in how come some people found computer programming easy, and others found it difficult. To learn more about this, I managed to find a MSc course which had modules from both computing and psychology.

A chance job application to the University of Manchester (which I found in Prospects Magazine) led me to meeting my future supervisor. My research interests were combined with my future supervisor’s research interests. Subsequently, my thesis topic, studying the maintenance of object-oriented software, was born.

My original contributions have been modest. After spending considerable time finding my way through cognitive psychology papers, and learning how research was done and discussed, I noticed that there were some interesting cross-overs with research that was emerging from researchers who were studying software engineering. I realised that there was a gap. 

After doing a bit of empirical work, my contribution was a new model of software code comprehension (ResearchGate). Working on this model, also led me to a small side project, where I worked on a set of software metrics (ResearchGate), which were inspired by the psychology (and neuroscience) papers that I spent a lot of time reading. This points to one of the interesting thing about doctoral research: sometimes there are surprises along the way.

All this work was compressed into quite a short period of time since I had limited amount of funding. I didn’t return to my subject until quite a few years after graduating since I later realised that I had burnt myself out. 

There’s another aspect that is important too: I found it a very lonely experience. Other doctoral students, however, might have a very different experience, especially if they work within an established community of researchers. To counteract this potential of isolation of loneliness, my advice would always be: make sure you seek out a community within the institution in which your research is situated. I do know that the School of Computing and Communications at the OU tries to create a strong research community, so students don't feel disconnected or isolated. Also, make sure you have a break from the study and research; fun stuff is important!

After working in industry for a few years, I picked up a post-doctoral post, working on an EU funded project. Although this wasn’t in the exact area that I had studied as a doctoral researcher, I was pleased I could get stuck into something interesting that would make use of some of the skills I had acquired.

Summary

A doctorate isn’t only about discovering something new in the world. It is also about developing skills, and becoming familiar with what it means to carry out research. It also means that you become a trained researcher and communicator. It can be something that is hugely rewarding, but it is also hugely demanding. It requires commitment and determination.

This blog represents a summary of different bits of information about doctoral study that I’ve picked up over the last few years whilst starting to work as a doctoral supervisor. 

There are a lot of other resources available which might be helpful. A good place to go to is the Vitae website.

Just as teaching is a skill which can be enhanced through professional development, Vitae is described as a “global leader in supporting the professional development of researchers”. To help researchers, there is something called the Vitae Researcher Development Framework (Vitae website). One article that might be of specific interest is: Are you thinking of doctoral research? (Vitae website). A further article, which can be found within Prospects Magazine: PhD Study.

Finally, if you're looking for more information about how research degrees work within the OU, you can also visit the Research Degrees website, which contains a wealth of information.

Acknowledgements

Many thanks are extended to Marian Petre, who has suggested some really helpful changes to this blog. Marion also runs a blog about PhD research: Pragmatic PhD, which has the subtitle "craft skills for students and supervisors". She has also written a book, with Gordon Rugg, entitled "The Unwritten Rules Of Phd Research" which I thoroughly recommend. I might have had an easier, and less confused journey if I had read it whilst I was studying for my own doctorate.

Permalink Add your comment
Share post
Christopher Douce

Components of the EdD Professional Doctorate Programme

Visible to anyone in the world
Edited by Christopher Douce, Tuesday, 21 June 2022, 17:38

This blog post aims to summarise aspects of the OU’s professional doctorate (EdD) programme, placing particular emphasis on the topic of education.

What has been presented here has been collated from a number of different resources. My primarily aim of preparing this post was to help me to get familiar with the new structure of the taught part of the programme. I’m also sharing it since it might be useful for either existing or prospective students, or for students who might also be studying for a disciplinary based PhD, since the EdD materials offer some helpful pointers

The programme that is roughly summarised here is different from previous years, since it contains a substantial and important taught component to help students prepare for their research that follows. Although the programme contains a number of really important residential schools, I’m highlighting the academic subjects that are explored.

Year 1

The programme guide introduces the first year as follows: “Year 1 … will focus on getting you started with your research, with a particular focus on contextual background of your research and the literature review. Year 1 includes an induction residential weekend, four modules of study with four accompanying online seminars, and the completion of two formative assignments and one summative assignment."

Module 1: Getting started

This first module is about setting the scene. Drawing on the module guide, this first module “will help you get started with your doctoral studies. The module covers what is involved in studying for a PD, time management, supervision, and the Researching Professional Development Framework.” It is intended to be studied within the first couple of weeks of starting the programme. The first section introduces the notion of the professional doctorate, and this is followed by a section about planning and managing your research project. A bit of advice (for students) that I’ve read was: “think about your doctoral studies as a project”.

Section 3 is entitled your development as a researching professional. It introduces the Researching Professional Development Framework (RPDF) (Vitae website), a tool designed to help your development during your doctoral studies.

This is followed by section about Professional Academic Communication in English (PACE), and introduces students to some useful some online resources, where students share their experiences of academic writing.

Supervision is an important element of an EdD programme, and also becoming familiar with the research process. The final section of this first module is entitled “Making the most of your supervision”. Students are directed to the Code of practice for supervisors and research students, and other resources such as the university’s research degrees handbook.

Module 2: Context for educational research

This second module will “guide you through exploring the specific context of your research, including the international, national, institutional and individual context within which your research is located. It also covers the importance of your professional identity, and the standards and principles for good quality research within your area” (EdD programme guide, p.9). 

This module is split into three sections. The first is further understanding the context for research. Students are asked to consider different perspectives of their research: macro-level, meso-level and micro-level. A further aim is to identify who the different stakeholders might be.

The next section, the professional as a researcher is all about “exploring the concepts of professional identity, agency, structure and reflexivity”. Reflection and reflexivity is explored as a key topic, which emphasises how important it is to relate our own position and identities to the research that is taking place.

The final section is entitled “standards for good practice in research”. This section is about ethics, the importance of ethical guidelines, power imbalances and how they might influence research, the student voice and co-research.

When a student has completed this section, it is roughly time to submit the first formative assessment. As well as introducing a research project, students are required to consider the context of the research, and the role of the researcher.

Module 3: Reviewing the literature

The literature review is one of the really important outcomes from doctoral research. This module, which is scheduled to begin in the new year “provides guidance to conducting and writing a literature review, including searching for literature, reviewing literature, referencing and reference management tools, and writing the literature review” (EdD programme guide, p.9). Key topics that are explored include what it means to searching for literature, review literature, reference literature, write a literature review, and to write critically. There is also a section that introduces the concept of a systematic literature review. Whilst carrying out reading within a subject, students may find a number of systematic literature review papers that offers a summary of a similar or related topic.

Module 4: Principles of research design

This final module of the first year introduces students to key terms and research concepts. It “aims to stimulate further thinking about your research design and covers topics such as ontology, epistemology and research paradigms, logics of enquiry and an introduction to quantitative and qualitative research” (EdD programme guide, p9).

Moving to year 2

During this first year, students will be required to carry out a number of assessments. During the time where there is no formal study scheduled, students will be expected to be carrying out reading and study.

Year 2

As well as having a taught section, students attend a residential weekend. In November 2021, this was hosted as an online event, where students were able to attend various sessions. Resources shared from this event, and earlier events are available online.

Module 5: Considering a research methodology

This first second year module “provides guidance about different research methodologies including experimental quantitative research, ethnography, grounded theory, case study research, action research, phenomenology and narrative inquiry”. There is a section for each of these methods, which also provides a set of resources, which can be useful to understand more about a particular method. If studying these materials, a suggestion is to only go digging for resources which you think are most appropriate for your particular project. A lot of resources are highlighted.

Module 6: Approaches to data collection

In some senses, this module follows on from the previous module about methodology, but it succinctly summarises the different approaches that could be adopted. From the programme guide, this module “will help you start thinking about the practical aspects of your research project by introducing common data collect methods and sources of data. Topics covered includes interviews, focus groups, observations, questionnaires, visual and creative methods, secondary data and documents and artefacts. As with the previous module, each section provides a very detailed references section that enables students to get a more detailed introduction and insight into different approaches.

Module 7: Professional conduct and research ethics

When it comes to EdD and PhD research, ethics is one of my favourite subjects. This module is said to “encourage your ethical thinking and assist you in developing a robust application for ethical approval for your planned research. Topics covered will include professional conduct, close to practice research, making an OU ethics application, …. and research data management.” Two sections are notable: there is a section about ethics and educational research, and ethics about health and social care research; students should choose whichever strand is most appropriate. One section that I must emphasise is the section that relates to academic and research conduct. There is also encouragement to carry out what is called a Data Protection Impact Assessment (DPIA), if personal data is kept and retained.

Module 8: Qualitative data analysis and presenting results

Qualitative data is rich non-numeric data that can be interpreted to provide meanings and explanations. This module will introduce “methods of qualitative analysis, including thematic analysis, discourse analysis, document analysis and multimodal analysis.” If a student is interested in carrying out interviews with participants to gather interests and perspectives, this section offers a really helpful guidance about how to begin to make sense of data that is collected. Such data, of course, must be made sense of in light of the reading that has been carried out, and also the perspective of the researcher. During data analysis, a student might use a tool such as NVivo to organise qualitative data.

Module 9: Quantitative data analysis

Quantitative data is all about numerical data. This final module introduces “various methods of quantitative analysing including testing for differences between means, correlation analysis, linear regression and logistic regression.” Specifically, “this module will require you to use SPSS, and you will need to download this onto your computer before starting the module”. If a student is going to be carrying our survey research, or is to be carrying out experiments to test hypothesis, this section is going to be really important. It is also important to recognise that methods can be mixed. For example, an interview study could reveal themes that could be studied in greater depth through a survey. Conversely, a survey may reveal an unexpected situation that can only be further understood by asking questions.

Moving to year 3

Year 2 marks the end of the formal study part of the EdD. Students will be invited to make a poster presentation to outline their research plans, and move to the second stage.

Years 3 and 4

The EdD program guide summarises years 2 and 3 as follows: “during stage 2 you will follow a more independent and individual programme of work with the continuing support of your supervisors. During year 3 there will be formative assessments at spaced intervals in order to help you progress and to provide formal opportunities for feedback.” (p.11).

A number of useful resources are provided on the EdD websites (there is a site for year 3, and another site for year 4). Highlights include a document that attempts to answer the question “How many qualitative interviews is enough?”, which has been prepared by Baler and Edwards, and a couple of video resources that have the title “stories from the field”. 

Reflections

Many of the themes and topics mentioned within the taught aspect of the EdD programme reminded me of themes and topics that were explored within the OU’s MA in Education programme, which offers a “lead in” to this programme. Although MA students may find some of the material familiar, I hold the view that the taught section is really useful in terms of setting the groundwork for the detailed data gathering and analysis that takes place later during the later years of study and research.

I came to the EdD programme from the discipline of computing, where I completed my own doctoral studies in the late 90’s. One thing I’m struck by is the thoroughness of the EdD programme. It is only by having gone through the OU MA in Education, and having done my own doctoral research do I really appreciate the detailed discussions about epistemology, ontology and methodology. 

It is also really interesting that the softer side of computing applies many of the methods and approaches that Education does; the commonality lies with the adoption of methods from the social sciences. A lot of computing and education research is all about people, what they do, and what they learn.

At the same time as being a supervisor for a current EdD student, I’m also a supervisor for a disciplinary PhD student. The approaches are quite different, in the sense that although there is more supervision for the PhD candidate, there is less structure, in the sense that there isn’t the taught component. One of the things that I am going to do is direct my PhD student to look at some of the materials that are exposed through the EdD modules. The cross over between the two is, of course, people.

References

The Open University (2020) Doctorate in Education (EdD) Programme Guide.

Acknowledgements

All these sections have been summarised from different resources from the EdD programme. Acknowledgements are specifically extended to Dr Carol Azumah Dennis, EdD Programme Leader and Dr Philippa Waterhouse, DHSC Programme Leader.

Permalink Add your comment
Share post
Christopher Douce

Supporting EdD/PhD students through the thesis and the viva

Visible to anyone in the world

On 14 June 22, I attended a CPD session about helping students through their doctoral studies. I attended this session since I support a couple of doctoral students; one through a PhD programme, and another through the EdD programme. More information about the EdD programme that if offered through the Well-being Education and Language Studies (WELS) faculty are be available through this blog.

This session was facilitated by Dr Sara Spencer (Head of Research Degrees, Graduate School) and Dr Sarah Sherlock (School of Physical Sciences, chair of research degrees committee). It seemed to be a relatively popular event, with 23 delegates.

The key headings for the event were: the thesis, mock viva, and post-viva support. I noted down the words, “at this session we will look at common concerns that student’s voice about thesis submission and the viva voce examination and consider possible strategies for overcoming these concerns.”

From the event description, the session had the following objectives:

  • To explore students’ expectations and concerns about completing their doctoral thesis and how they will perform during their viva voce examination
  • To share ideas and practices that can be used to support students during the writing-up phase
  • To share ideas and practices that can support students to prepare for their viva
  • To identify sources of help and support offered by the Graduate School Network and the OU Library that can support students during the writing-up phase 

The Thesis

‘Write up’ is a HESA status as well as a university status; a status that applies for one year only, which is available to students during their fourth year of study. This means that students pay a lower fee during a ‘write up’ year. If they go over the write up year, they may be liable for full fees.  An important difference is that students on Professional Doctorates (such as the EdD programme) are not eligible for writing-up fees.

During the session, I made a few notes from some of the slides.  A key point was that the thesis must meet the requirements of the research degree regulations. Interestingly, things have changed since I submitted my own thesis. Students no longer need to submit a paper copy; it can be submitted electronically. (I remember having to get mine bound by a book binder who worked in the town of Chichester!)

A key point is that a thesis is a monograph. In other words, it presents a single coherent narrative. Also, students can make their own decision about whether they wish to submit. A student doesn’t have to expressly seek permission from the supervisor (but, it is probably a good idea to check with them, just to make sure they think that a student is likely to make a worthy submission). Another important point is that if a student is funded to carry out their research, and to write their thesis, a student will no longer receive a stipend when they make their submission.

One interesting point that I did learn (which was something that I already probably implicitly knew about, but didn’t really know what it was called, since I haven’t needed to think about it) was that a thesis can also include a ‘non-book’ component. In addition to submitting a textual monograph, a student may send in other forms of material to accompany a piece of research. In computing, this might be a software artifact. In design or engineering, this might be some architectural drawings. In the arts, this might even be a video of a performance. 

Mock Viva

The assessment of a thesis was described as taking place in three phases: 1) a preliminary assessment, 2) defence of the thesis at the viva, 3) re-examination of the thesis following revision. Some students have the opportunity to take part in a mock viva which is set up by the supervision team. 

The aim of the mock is, of course, to enable students to be as prepared as they can be to be robustly questioned when they defend their work. Since the viva can be a stressful exercise, a mock can help a student get a sense of what happens in the real thing. I remember when I participated in one: the different supervisors took on different roles. One asked question about the big picture, and the other supervisor asked very specific questions about the details of the text.

An important point was made, which was that examiners can get nervous too! Mocks are also helpful for the supervisors as they are for students.

Exam Panel Nomination

A request was shared to all delegates: please think about the exam panel to ensure that nominations are submitted in good time. This suggested reminded me of something. Whilst my student was carrying out their literature review, I remember saying the following: “do look for people who are doing similar research to what you are doing; they might well become potential examiners”.

The exam panel must be approved by the research degrees committee. It was also said that allocating examiners is one of the most important things that the university does (in terms of the doctoral research process). It was noted that there needs to be a minimum of two examiners. Usually, this should be one internal, and the other should be external (in some cases, they can be both external, if there isn’t the internal knowledge within the school or department). The make-up of the whole panel is important. The experience should be distributed across the panel.

Something that I didn’t (formally) know is that a doctoral examiner works according to a contract; there needs to be an offer, this needed to be accepted, and there needs to be consideration (which means that they are paid for their work). The contract is there to avoid ambiguities, and to enable a route to resolve difficulties if they were to arise.

When an exam panel has been chosen, a good tip (for a student who is going to be examined) is to read the papers that have been written by the examiner. This may give a student some insight about what perspective they might be coming from. For example, they might prefer one set of methods over another.

The Viva

The viva begins with a pre-viva meeting with the chair and the examiners. Observers may only be asked to the pre-viva meeting if there is a specific question that the examiners may wish to ask. In the meeting, the examiners may have a discussion about what the approach is going to be, and what questions to ask.

During the viva, some candidates may be encouraged to give a short presentation of the work to the chair, the examiners, and the observers. The viva may, generally, last between 2 and 3 hours, but will depend on the subject that is being examined. A viva will go on for as long as is needed. Breaks can be requested via the chair. Different examiners may take different approaches. Some may go through a thesis a line at a time; others may take a different approach, asking more broad questions. 

A bit of advice I once gained from a colleague in terms of examining a viva was, obviously, to look to the research questions, and then look to the methodology to learn how a student had tackled a question, and justify their choices.

A comment made during this part of the event was: questions to students might explore their knowledge from across the discipline of study, not just the very specific detail of the text that is being the focus of the exam.

The next step is the post-viva meeting, which takes place between the chair and the examiners. This is where the student has to be left on their own whilst the deliberations take place. If this meeting takes a while, this may not necessarily mean a bad outcome. There is also a bit of administration to complete, such as, the completion of forms, which also includes the agreeing of corrections, and what the panel needs the student to do to pass. All this admin can take a bit of time.

The outcome from the panel is a recommendation that goes to a committee. It is also important to note that a recommendation is different from an outcome.

Post-viva support

There are a range of outcomes from a viva (which are based on the quality of a submission) ranging from student being awarded the degree, resubmission, and re-examination, getting an alternative award (such as an MPhil), through to a student not being awarded the degree and not being able to resubmit (and a couple of other options in between).

Extensions to the correction period are not possible, and students who do not submit by the deadline will fail, unless there are clear mitigating circumstances. To repeat, students are not allowed extensions, as otherwise they will fail. Corrections have to be done on time.

Reflections

Having been through this process from beginning until the end, a lot that was presented within this session that was familiar to me. I was familiar with the various phases, but I wasn’t familiar with a lot of the finer detail, such as the roles of the committees, and what observers can and cannot do. Although I think I had once heard that students are not permitted to submit their corrections late, it was good to be reminded of this!

During the discussions at the end of the session, a really helpful comment was “it [the thesis] doesn’t have to be perfect; it just has to be good enough”. This has reminded me of another bit of advice that I was given about doctoral study. I once thought a PhD was gained by uncovering ground-breaking new bits of knowledge, but this was a misunderstanding about how knowledge generation works. The aim of doctoral research is to add to the sum of human knowledge in some form, and it is certainly okay if a contribution is a small one. Contributions are built on.

Another perspective is that doctoral study represents an extended form of academic apprenticeship. It demonstrates that you can do research, and that you are capable of creating something that is original. Reflecting the above comment, research also builds on the work of others.

Acknowledgements

Very many of these words have been summarised from comments from Sara and Sarah, and the slides that they shared during their really helpful CPD session.

Permalink
Share post
Christopher Douce

Finding research time

Visible to anyone in the world
Edited by Christopher Douce, Monday, 1 Feb 2021, 17:43

After about two years of working at the former OU London regional centre in Camden, I remember a presentation for someone; a fellow staff tutor who worked in the Arts Faculty.

“He’s been given a chair!” my colleague whispered. I must have looked confused, since this assertion was immediately followed by a further explanation: “he’s become a professor!”

I had discovered that much of my role of a staff tutor was constantly spent facilitating, co-ordinating and administrating (if that is a word). I didn’t have any time to carry out any substantial or significant research. I was way too busy with “getting things done” to do any in depth thinking and reading about and into my discipline, which was originally computer science – or, more specifically, computer programming and software engineering.

“They do things differently in Arts; they have a role called a senior faculty manager” my colleague explained. “The faculty manager does a lot of the administrative work that a staff tutor does, such as organising the timetables and carrying out CDSA appraisals”.

If we had help with carrying out some of the admin work, I could see how someone who was doing my role might be potentially able to carry out research, and potentially step onto the trajectory that may lead to a chair. 

I could see it working within the Arts faculty, but I couldn’t see how that would be possible in the Faculty of Maths Computing and Technology, as it was called then.

I remember some celebrations. I remember some speeches. I’m pretty sure there would have been some prosecco.

After the celebration and the speeches, I would have returned to my email inbox to continue with whatever admin I needed to complete.

Finding the time

The School of Computing and Computing has a number of research groups. 

There’s the next generation multimedia and networking research group, the critical information studies group, an interaction design group, an AI and natural language processing group, a software engineering and design group, and a Technology and Education group (group website). 

My research ‘home’ within the school lies within the Technology and Education group, for the simple reason that I thought it easier to carve out a research niche if I allied aspects of my work (which is within education planning and organisation) to computing technology (which is a field that I used to work in). This said, my research interests also cross the interaction design group (which I used to tutor), and software engineering (where I’ve done a bit of research into, particularly theories of software comprehension and software metrics).

Earlier this month, there was a meeting which was about how staff tutors in TERG might increase their opportunities to carry out research (if this is something that they are interested in doing). Essentially, it all boils down to how to “craft out time” from our day job. This may mean thinking about accompanying issues, such as how to gain help with important elements such as writing, or research design, or statistical analysis. It also means finding ways to set boundaries.

I remember a talk by a former colleague who used to work in the School of Maths. He allocated a day per week to carry out research. I tried that for a while, trying to allocate every Friday as a research day, but it broke down; meetings kept sneaking into my diary – the need to administrate intruded in my plans.

One way to increase research capacity is to combine it with other related activities. One activity is co-supervising PhD students. This means identifying projects, and recruiting potentially interested students. Doctoral students might mean full time students based in the school, or part time students who are studying away from the university, or even EdD students (if their research area has a significant or substantial educational focus). A point to bear in mind in that a staff tutor must network to find potential co-supervisors.

An important suggestion was that it might be possible to plan what could be termed an ‘internally supported sabbatical’, perhaps in collaboration with other staff tutors or assistant staff tutors. Perhaps time could be ‘chunked’ together. Perhaps my day a week didn’t work out, since I didn’t have enough concentrated time to work dedicate to a problem. It took time and energy to get into research, and there’s an overhead in trying to repeat that same activity every week. Momentum is important.

A related theme is the importance of buddying; perhaps sharing some responsibilities maybe one approach to gain some of the important ‘headspace’ needed to facilitate the development of research capacity.

Other points that have been discussed have included the need to be completely honest about how our time is allocated and reported to the university (using something called the Academic Workload Modelling tool). Maybe there are clever ways to gain administrative support, but effective admin support needs consistency, since that too is all about understanding and solving problems.

Since I haven’t been able to carry out any disciplinary research for quite a while, I feel I would benefit from knowing how others do research. Perhaps there’s opportunity for mentoring, or the development of practical support and guidance about how to bid for projects. Again, it comes down to knowing who is good and doing what, and who might be able to help.

During our meeting, something called PACE was mentioned, which is an abbreviation for Professional academic communication in English. PACE happens in another faculty, called WELS, the Faculty of Wellbeing Education and Language Studies. The programme looks interesting, and doing a bit more digging, I’m taken to a further set of pages about face to face doctoral training.

Director of research

Following on from our TERG specific meeting, our current director of research, Robin Laney attended one of our staff tutor meeting. I noted down four important points:

  1. Make sure you find the time to network widely across the school (but also outside of the school too). Find others who share your research interests.
  2. Do you have a research agenda? If not, try to write one. If you don’t, go to different research group meetings. See point 1; networking.
  3. When it comes to finding research students, make sure that you advertise projects not only through the school website, but also through various mailing lists and other communities. See point 1; networking.
  4. Go speak with the director of research. Our director may be able to put you in contact with other colleagues in the school with similar research interests, and support activities that could lead to research, such as conference attendance or funding of pilot project. See point 1; networking.

Reflections

I’m faced with a dilemma. 

I really enjoy research, but I also enjoy seeing the end result of what some might see as ‘admin’, such as making sure that a module timetable is set up and ready to go, or the development or co-facilitation of a tutor development conference. In some respects, I’ve tried to find a middle way, which has meant carrying out some scholarship through the STEM faculty scholarship centre, called eSTEeM.

The one thing that is common to both research and scholarship is, of course, time. It may be possible to carve and craft time from an existing work plan, but there are two other common elements that are important: collaboration and planning.

This blog can be linked to two other blogs, one which is about C&C research groups and another post which offers some pointers about research funding.

Acknowledgements

Many thanks to Janet Hughes who set up a meeting to discuss research amongst the staff tutors from the TERG research group. Janet kindly reviewed an earlier version of this blog. Thanks are also extended to Robin Laney, C&C Director of Research. Acknowledgements are also extended to Karen Kear, who leads the technology and education research group.

Permalink Add your comment
Share post
Christopher Douce

C&C research fiesta: getting research funding

Visible to anyone in the world
Edited by Christopher Douce, Tuesday, 27 June 2023, 16:03

This is the second in a short series of two posts that summarises some of the highlights of a ‘research fiesta’ that has held by the School of Computing and Communications. This post summarises some of the points that were made during a panel session about research funding.

The panel comprised of four professors (if I’ve counted correctly), a research manager from the STEM faculty, and was facilitated by our director of research, Robin Laney. Although the focus was about research funding, it could have also easily had another title: how to become a professor.

Here’s a list of some really useful tips that I noted down about gaining research funding: 

  1. Think about how you might go about forming a working relationship with a funding body. This might mean keeping an eye out for different research related events that they run. Networking is important. Take time to speak to them.
  2. To develop relationships with funders, join mailing lists, check their websites and respond to calls for advice and consultation activities.
  3. Take time to understand the motivations of a funding body and what their priorities are. Simply put, the closer a research proposal or bid fits the aims and objectives of a funding body, the higher the probability of success.
  4. As well as understanding their aims and objectives, take time to understand the processes that they use, both in terms of bid submission and also in terms of how bids are evaluated. A key tip here is: talk to colleagues who have been successful and know what the procedures are.
  5. Always try to play to the strength of the university. Each institution is unique.
  6. Consider projects or proposals that are a little ‘left field’; proposals that are slightly unusual or explores an unexpected area may cause interest and intrigue.
  7. Look for new funding programmes. Getting in early might benefit both the funder and the organisation (and project) that is funded, especially as the funding programme builds up experience and finds its distinct focus.
  8. Successful bids often have components of interdisciplinarity and collaboration. Unsuccessful bids don’t present a clear story.
  9. Find collaborators who are able to work between disciplines; these are rare people who can help with the writing of project bids and proposals.
  10. Find external stakeholders who have a lot to gain from their involvement in a project. When describing this, present a clear project narrative that others can easily understand.
  11. When working with collaborators and stakeholders, make sure that you give them plenty of time to create supporting documents, such as letters of support. 
  12. Think in terms of teams. Working with a team of people means that funders might see certain bids as being less risky. Use your team to read and review your bid.
  13. Learn how everything works. Become a bid reviewer and seek out opportunities to sit on funding panels. The experience of reviewing other bids is invaluable.
  14. Speak to your university ethics committee early (and show that you have done so).
  15. Think about creating what could be described as a portfolio of ideas to work on at any one time.
  16. Smaller grants can be important; small grants can lead to large ones. Small grants can help researchers and research groups to develop their experience and expertise.

Summary

There are lot of really helpful points here. The biggest points I took away from this session was: be strategic (consider your portfolio of interests), look at what funding bodies are doing and what they are doing, and network to find collaborators, and build a team around project bids. In essence, take a collaborative approach. 

Addendum

This is late breaking edit, to share an article that was shared after the fiesta, which has the title: 25 research tips and strategies. It is worth a look.

Permalink Add your comment
Share post
Christopher Douce

Introduction to the REF

Visible to anyone in the world
Edited by Christopher Douce, Wednesday, 24 Oct 2018, 10:42

In November 2018, I had an opportunity to attend something that was called a ‘writing retreat’. The idea behind the event was simple; it was an opportunity to take a bit of time out from day to day activities and focus on writing up various bits of research that colleagues within the school had been working on. There was another reason for running the retreat: there would be a particular emphasis on writing papers that could be submitted to the 2021 REF (the Research Excellence Framework).

What follows is a brief summary of some of the points that were made during the introductory workshop which introduced the REF. Full acknowledgements are extended to Professor Jane Seale who facilitated this workshop. Many of the words here are directly from Jane’s presentation.

Introducing the REF

A key phrase that I’ve heard ever since I’ve been working at a university is: REF submission. A submission relates to particular subjects. Some universities may focus on some submission areas over others to play on their own strengths and weaknesses.

The OU makes a number of submissions, and one of the submission areas that I am connected with is education. This means that ‘education-like’ papers will be grouped together, submitted, and then assessed by an expert panel. The outcome will be a ‘research rating’, and this is directly linked to income that is received by the university. Simply put, the higher the rating, the more research income an institution receives. 

I asked the question: “what does ‘education-like’ actually mean?” These will be papers that more than just describe something, like a system or a tool that has been designed (which is what some computing papers can be). Education research papers need to be firmly linked to an education context. They should also present a critical perspective on the literature, the work that was carried out, or both.

When?

The 2021 REF assesses research that has been published in the public domain that has been carried out between 1 January 2014 and 31 December 2020. Papers that are to be included are typically included within a university repository. (The OU has a repository called ORO).

What?

Every academic whose contract which has a significant research component should submit at least one paper, and there should be a REF average of 2.5 papers per academic (as far as I understand things!)

Publications should be of a 3* or a 4* quality. Three star papers means that a piece of research is of ‘international significance’. The education REF panel will accept different kinds of submissions, including journal papers, conference proceedings and book chapters, but there has been a historic bias towards journal papers for two reasons: they are peer reviewed, and are more readily cited. Books or book chapters should present summaries of research and shouldn’t be student focussed or summarise what is within a field.

In education, some journals contain practice papers or case studies. I noted down that education papers don’t have to be empirical to be considered for the REF. A paper might present a new practice or an innovation or development of an existing practice. A suggestion is to preface it with what you’re doing and why you’re doing something, and offer a thorough criticism of how and why something fits in with existing work. This means that it’s necessary to consider comparisons and contrasts. Descriptions are also necessary to contextualise the research, but the balance needs to be right: practice papers need to be generalisable.

Where?

A question to ask is: where should you publish? It was interesting to hear that the REF panel for Education isn’t particularly concerned with the impact of the journal where research is published; what matters is the research itself. Given that education research tends to be descriptive, a suggestion is to choose journals that have a generous word count. One such journal is: Open Learning, which is thoroughly excellent. 

How?

How are submissions assessed? The submission chair will look at the title, abstract and reviewers and match a set of papers to an expert. Every paper will get read twice, and there will be some kind of process of random sampling. An interesting thought is: ‘don’t make the reviewers work too hard’, which is advice that I also give my students who are writing their end of module assessments or project dissertations. Institutions (including the OU) may having something called a ‘mock REF’, where they try to replicate the official REF submission to get a feel for the direction where the institution is heading.

REF criteria

Papers are judged on three key criteria: significance, originality and rigour; each of these criteria are equally important.

Significance: A paper or piece of research provides a valuable contribution to the field (this relates to the ‘so what?’ question about the purpose of the research). Also, how does the research moves the field forward. The contribution can be theoretical or empirical. A point to note is that a 3* paper contributes ideas that have a lasting influence. A 4* paper has a major influence on a field.

Originality: Is the research engaging with new or complex problems? Perhaps a paper might be using existing methods but in a new way, or challenging accepted wisdom. As a point of reference, a 2* paper contributes a small or incremental development, where as a 4* piece of research is something that is outstandingly novel in the development of concepts, techniques or outcomes.

Rigour: Papers and research offers intellectual precision or robustness of arguments. Some important questions are: is there rigour in the argument, the methods and the analysis? Also, can the readers trust the claims that are made when you ask the question, ‘how have you analysed your data?’ It is also important to include researcher reflexivity (question the role that the researcher had in the analysis of the research), and to critique the work, offering a summary of strengths and weaknesses. A 4* paper is one that is exceptionally rigorous, with the highest standards of intellectual precision.

Reflections

I only have a small amount of time in my staff tutor contract to carry research; I try to do what I can when I can, but I very often get entangled in tutor and student support issues which take me away from exploring really interesting questions and topics. Plus, increasingly, I’ve been playing a role in AL professional development. Carving out time for research is difficult balancing act due to all these competing demands.

When I started the retreat, I planned to write about some of the tutorial observation research that I’ve been doing and reacquaint myself with some of the papers that I had uncovered whilst doing this research. The introduction to the event gave me some really needed context, in terms of what a very good paper actually looks like. It also gave me a bit of direction in terms of what I actually needed to do.

One of the things that I did during the retreat was to consolidate different bits of research into a single document that forms the basis of a paper. I now have a clear and distinct structure. What I need to do now is to do some further close reading of the references to more directly position it within the literature, add a more critical twist to the analysis to broaden its appeal, and to do quite a bit more editing.

Permalink Add your comment
Share post
Christopher Douce

EdD residential weekend, June 2018

Visible to anyone in the world
Edited by Christopher Douce, Thursday, 21 Nov 2019, 11:21

Between Friday 8 June and Sunday 10 June 2018 I attended an EdD residential weekend at the university campus in Milton Keynes. The EdD residential weekend was something that was new to me: I was attending in the capacity of a ‘co-supervisor’ (or ‘second supervisor’). 

The EdD qualification is a doctorate in education that is at the same level as a PhD, except for one fundamental difference: the research and contribution to knowledge carried out through an EdD is situated in the educational practice or context of the student who is carrying out the research. 

One of the things that I learnt from the weekend was that other institutions have their own EdD programmes. Since 1997 more than 370 students have been awarded an EdD through the OU’s EdD programme.

What follows is a transcription and summary of some of the notes that I made during the weekend. There are mostly from my perspective of a supervisor, but they might be of interest and use to EdD students or anyone who is interested in learning more about what EdD research and study entails.

Introduction

The event was introduced by Inma Alvarez, the university’s Centre for Research in Education and Educational Technology EdD programme director. Inma emphasised that the EdD is a professional doctorate that enables students to gain skills in educational research and enquiry and be able to carry out a study that contributes to professional and practice knowledge.

The EdD takes at least 3 and a half years, with a maximum of 6 years. Students are provided with two supervisors; a lead supervisor and a co-supervisor. During the first year, students are required to design and carry out a preliminary study.

Progress on the EdD is not measured through tutor marked assignments (TMAs) but by a series of progress reports. At the end of the first year, students are required to produce a report which is assessed by an academic who is neither of their supervisors. Inma made an important point that a lot of the responsibility is down to the student; an EdD should take notes of their supervision meetings and actively manage their supervisors!

All the students were given a number of useful tips, including: treat the programme guide as your ‘bible’ and subscribe to the student forums, so students can get updates of when people post messages, updates and questions. 

The first year is all about becoming an independent researcher, which includes carrying out a literature review, carrying out that initial study, submitting 4 progress reports and the end of year progress report. The final report will contain an introduction to the project, a summary of the research questions, a literature review, a section about the methodology that is adopted, a description of an initial study, outcomes, and a detailed reference section.

In the second and third year students will ‘follow a more independent and individual programme supported by their supervisors’.

Students will have access to resources, which includes access to the OU graduate school network, the EdD programme website and online doctoral training resources. Another important message that was coming through was: ‘be responsible for your own development’, and a connected thought is to start a reflective diary. This diary can be used to keep notes about what is studied and what is learnt, help to develop academic writing and creativity.

Doctoral researchers and supervisors

The aim of this next session was to enable supervisors to meet their students and members of the EdD team. Some notes that I made from this session were about “gaining confidence in plans, getting used to critical feedback, getting some research training, understanding research ethics, talking to some EdD graduates and becoming a research professional”. 

I also made a note that there was a group discussion about the question: what is theory in education? I noted that there is the concept of ‘critical theory’, but there are other approaches and theoretical tools that could be used, such as critical pedagogy and activity theory. This said, I was also mindful that the educational research notion of ‘theory’ is slightly different to a scientific understanding of what a theory is.

Day 2: Doctorate in Education Literature Review

The second day began with a presentation by Ursula Stickler in a library seminar room. The aim of the session was to learn more about ‘how to approach a literature review, the criteria, and practical considerations, such as knowing when and where to stop’. 

We were given an activity, where we were asked the question: why are you doing a literature review? Answers included: looking at themes, examining ideas and methods, examining debates, learning about academic literacies and making sure you’re not duplicating your research. Other answers also included identifying key authors and researchers and uncovering your own view of the literature and what has been done before. I also noted down some key terms that were used in the REF, the Research Excellence Framework: originality, significance and rigour.

We were then guided to another activity, where we had to answer the question: how to best go about a literature review? Other questions that were asked included: where to start, where to finish, what to include and what to leave out. It was also important to ask the question: what are the key journals and writers? It’s also important to be clear about what the main argument (or arguments) are. Another note I made was: narrow your search, find your gap (within the research) and widen your implications (which I assume relates to the impact that your research can make).

The final activity asked the question: what are the strengths of a good literature review? I didn’t make too many notes during this part of the event, except that the discussions were focussed upon an article by Boote and Beile entitled ‘Scholars before researchers: On the centrality of the dissertation literature review in research preparation’.

Session: Ethics

Ethics are important. The first session on ethics was facilitated by Alison Fox and Kris Stutchbury. I made a note that “your entire project is an ethical task” with an accompanying comment that how students choose their research projects, carry out research tasks and disseminate their research results are all ethical tasks. In this session I was introduced to an new acronym: CURD, which stands for Consequence, Ecological, Relational and Deontological.

The next ethical session was all about case studies. Duncan Banks gave a presentation that had the title: an introduction to research ethics (PDF). We were introduced to the BPS, British Psychological Society, code of ethics of human participants. Some of the key points I noted down were: respect for the autonomy and dignity of persons, that the research must have scientific value, quality and integrity, and that it must maximise benefit and minimise harm. Another dimension of ethics relate to risks, both to research participants and also to the researcher.

Day 3: Designing an initial case study

The third day of the event was organised slightly differently; we were all brought together for a plenary presentation, and then we were able to attend different parallel sessions. In some respects, the weekend turned into a mini conference! What follows is a polished and paraphrased version of the notes that I made during each of these sessions.

Opening plenary session

The opening session was presented by Felicity Fletcher-Campbell. 

Literature review

Felicity returned to one of the important topics of the weekend; the literature review. The literature review should take account of a theoretical position, the substantive area and a methodology. A literature review is ongoing, flexible, adaptable, malleable, reactive and proactive. During the literature review, students should add and remove papers, and also reconceptualise their work. It offers a means to inform your empirical work. A key phrase I noted down was “keep it slim and purposeful”, which I thought was great advice. 

The initial study

The initial study is important since students need to carry this out to complete an important assessment within the EdD. Students must write a report that must present a clearly structured framework for the whole study. In some ways, this initial study and accompanying report is used to ‘sort out’ any issues regarding theory or theoretical position. I noted that the “report shows your developing knowledge and experience of relevant theoretical traditions and literature”. It is used to critically assess where the different authors and researchers are coming from and their accompanying perspective. The report also allows students to relate the literature to their research questions. 

Felicity offered some really useful tips and pointers: “your theoretical position informs your methodology” and “buy yourself a very big box of quotation marks and inverted commas” and “be really boring and put quotes around everything and be obsessed with page numbers”. On the subject of ethics, students were told: “name your supervisors on consent forms, so they get blamed too”.

On the subject of time, “research time is different than normal time; time fills up, everything will take longer than you think they will take”. 

There’s also the need to balance everything; to balance the preparation and the doing, the data production and the data collection, and the analysis with the report writing. Also, when it comes to the writing, “the initial study will help you to explain the genesis of your main study”.

The main study

Some key points about main study were: “you need to tell a story” and to ask “what I need to tell the reader? What do they need to know? Why did you tell me this? Also, why didn’t you tell me this earlier?” From a personal perspective, I’ve internalised the point about the need to tell a story, and I’ve passed this message onto the TM470 Computing and IT project students that I help to support. The narrative that is presented to the examiner is really important. 

Parallel Session: Working with digital data

The first parallel session that I went to was by Carol Azumah who discussed the usage of digital data and resources, such as blogs and social media. Some resources, such as blogs, can be viewed as public documents. Two terms that I noted down were ‘discourse oriented online ethnography’ and ‘fast ethnography’. An important point is that ethics always need to be considered. We were directed to the Ethics pages of the Association of Internet Researchers (AoIR).

Parallel Session: Concept maps

Diane Harris’s session was about how to use concept maps in research. Concept maps were introduced as tools that can be used to hear the voice of participants and “for them to own what they have said to you”. Diane offered a specific example of how they were used to study music education in a school. Participants could own, add to and create maps. The resulting maps could then be analysed using thematic analysis or critical incident analysis. Regarding this second technique, Diane mentioned two researchers, Harrison and Lee (2011) (Taylor and Francis) who used the approach in medical education.

Closing session: the way ahead

After the parallel sessions, we returned to the plenary room, where we were offered some closing advice from Inma Alvarez. From what I remember and from what I’ve noted down, students were encouraged to work as and think of themselves as independent researchers. They should also think of their supervisors as critical friends. Students were encouraged to identify what skills are needed, reflecting earlier attention that was given to the importance of continuing professional development.

The concluding bits of advice were: “be open to the unexpected; you can modify the title of your study [if you need to]; work to deadlines and use frameworks to guide what you do, and be sure to manage your supervisors”.

Reflections

What really impressed me was how well the EdD weekend was planned. There were ample opportunities to speak with supervisors and fellow students between more formal events and activities. It struck me as being a really nice mix.

There were a couple of highlights. The first one was a presentation by a former EdD student who spoke of some of the challenges of doctoral study. The second was the talk by Felicity Fletcher-Campbell, which was packed filled with useful and practical advice and delivered in a thoroughly engaging way.

I never took place in a formal induction session when I embarked on my own doctoral studies. What really impressed me with the weekend was its emphasis on structure; the importance of the literature review, the importance of the initial study, the main study and how everything connects together. I think the weekend has also positively impacted on my own practice; only by writing this blog have I realised that I have started to pass on some of the tips mentioned during this weekend to some of the undergraduate project students that I support. 

Permalink Add your comment
Share post
Christopher Douce

7th eSTEeM Conference: 25 and 26 April 2018

Visible to anyone in the world
Edited by Christopher Douce, Friday, 25 May 2018, 09:57

The Open University runs a centre called eSTEeM which funds research and scholarship to enhance and develop STEM education. For the last few years, the centre has run a conference that serves a number of purposes: to showcase research, to create a space to get people talking (and potentially collaborating) with each other, and to offer an opportunity for academic professional development.

What follows is my own personal summary the two days of the conference. There were a number of parallel sessions to choose from. My approach to choose them was very simple: I chose the sessions that packed a lot in. This meant that I chose the paper sessions rather than the various workshop sessions were on offer. At the end of the blog I offer some very short reflections based on my experience of the session.

Opening keynote

The conference was opened by Diane Butler, who introduced the introductory keynote speaker, Tony Bates who used to work at the OU and also the University of British Columbia. Tony has recently written an Open Text Book called Teaching in a Digital Age. I made a note that Tony opened with the observation that there is ‘a lot of change’ and this has direct implications for teaching and learning at the university. One of the key forces of change is the need for skills, i.e. IT skills that are embedded within a subject area; knowing skills that are specific to a discipline. An accompanying question was: what are employers looking for? Certain skills are really important, such as active listening, speaking and critical thinking. 

Learners need to practice and develop skills and to do this they need regular feedback from experts. I made a note about that technology isn’t perhaps the most appropriate way to develop the soft skills that Tony mentioned earlier. An interesting question was posed: what does an advanced course design look like? There were some suggestions (if my notes serve me well): perhaps there might be student generated multimedia content and assessment by e-portfolios.

Tony also spoke about trends: there are new models of delivery; there is face to face teaching on one side, and fully distance learning on the other (and everything in between). An interesting point was that every university in Canada had fully online courses, with 16% of all course enrolment being to online courses and programmes. Traditional universities are moving into the space where distance learning institutions used to dominate.

An interesting new trend was the notion of hybrid learning: looking at what is best done in the classroom and what is best done online. I made a note that Tony said there was ‘no theory about what is done face to face versus online’, which strikes me as surprising.

A significant trend is, of course, MOOCs, but it was reported that there was no MOOC mania in Canada. Other trends included open educational resources and open text books. The point is that we’re now at a point where university professors offer learning support and not content and this has implications for teaching and learning. 

Tony concluded by leaving some points for the university: that technology is continually changing, that there needs to be flexible accreditation for life-long learning, and perhaps there needs to be an agile approach to course (or module) development. Also, all universities will be or are going to be digital (in some way or another). 

Paper session: Supporting students

Lessons in retention success: using video media to influence students

Jessica Bartlett spoke about her experiences working on S282 Astronomy. There are some immediate and obvious challenges: students numbers are falling and the module contains a lot of maths. An interesting point is that 50% of the students studying this module were not from the STEM faculty (which is where all the maths is studied).

The aim of the project was retain more students and help more students to pass exams. The module uses formative tutor marked assessments (TMAs) which means that the module team can reuse questions but can’t (of course) provide model answers to students. I recognised an interesting comment: ‘students don’t often look at their mark, ignoring their feedback’. The module team made videos about how to deconstruct and approach the TMA questions. I made a note of something called ‘reviewing your TMA’ activities, which encourage students to look back at what they’ve done (which sounds like a great idea). There were also weekly videos, where the filming and editing was done by the module team.

Evidence that bootcamps can help student retention and progression

Tom Wilks also spoke about S282 Astronomy but within the context of a ‘bootcamp’ that was designed to offer addition student support. Tom recorded short tutorial sessions that covered a range of topics: basic maths and physics, general OU study skills, how to use the VLE and how to use the Adobe Connect conferencing tool. 28 Adobe Connect sessions were recorded, each lasting between 2 and 10 minutes in length. These sessions were advertised to all students, who could access a forum and an Adobe connect room. Other resources include something called an ‘are you ready for quiz’ which is also used with some computing. Tom commented that tutors can refer students to his recordings if some students were struggling with certain concepts, and he also found that students did re-engage with materials when they were approaching their TMA. 

Flexible/early start M140

Carol Calvert gave a talk on her work on introducing a Flexible or Early start to M140 Introducing Statistics. I’ve heard Carol speak about this subject before, and she always delivers a great talk. Her research is based on an earlier study where she looked at students who succeed despite the odds that are stacked against them. One of the key findings of this research that one thing can really make a difference, and that is: starting early. 

Carol’s intention was to create an ‘early start’ experience that was close to a student’s experience when it officially begins. This means they have access to materials, can access the VLE site, have access to tutors, and can access to resources such screen casts and software. 400 students were sent a message offering them an invite to start early, and 200 responded saying that they would. Tutors offered sessions on study skills and tutorials on content. Another advantage is that if students do start early, they will know sooner whether they are on the wrong module, which can be really useful, since there are significant fee implications if someone finds themselves on the wrong module. If you’re interested, more information about Carole’s scholarship is available on the eSTEeM website

Improving retention amongst marginal students

Anactorial Clarke and Carlton Wood spoke about an access module: Y033 Science Technology and Maths. Access modules are important due to the university’s commitment to widening participation. Y033 is studied by 1K students per year and students who have successfully studied this module (as far as I understand things) can apply for a fee waiver. 25% of students declare a disability and access students are offered 1 to 1 telephone tutorials since previous research has suggested that sympathetic and supportive tutoring is crucial to student success. The study that Anactoria and Carlton introduced use a mixed method. They looked at the completion of S111 Questions in Science and Y033. Students who have taken the access module are more likely to stay with the module; the point being that access level study builds confidence (and emphasises the importance of access).

Paper session: online delivery, tuition and international curriculum

Synchronous online tuition: differences between student and teacher

Lynda Cook and a number of other colleagues asked a really important question: what are online tutorials really like? An accompanying question is: do we meet our student’s expectations? Students on 2nd level modules were surveyed, recorded tutorials were studied, and students and tutors were interviewed. Students reported that very few were using microphones (which isn’t a surprise to anyone who had attended an Adobe Connect session) and an analysis of recorded tutorials suggested that lots of features were not used, with the exception of the chat box.

The interviews with tutors revealed that when the recording button goes on, students are reluctant to talk. One conclusion is that students’ value tutorial recordings but students don’t like to interact. A personal note is that there is a conflict between interactive and recorded lectures and I don’t think the university has quite some way to uncovering the pedagogic opportunities afforded by online tools such as Adobe Connect (and, in some ways, this links back to some of the themes mentioned in Tony’s keynote). 

Understanding tutorial observation practice

It was time for my session. I spoke about a short project that aimed to ask the question: ‘what is the best way to observe tutorials?’ I approached this question by doing three things: carrying out a literature review (with help from a brilliant tutor colleague), and conducting two sets of focus groups: one with tutors, and another with staff tutors (the members of the university who usually carry out tuition observations).

Some of the themes that emerged from the focus groups directly echoed some of the themes in the literature. An important issue is to understand what tuition observations are for: are they for development, or are they for management? (The answer is: they should be used, in the first instance, for development; the observers can learn a lot just by observing). An outcome from the project was to uncover a set of really useful tuition guidelines that have been used and developed by colleagues in Science. The next step in the project is to formally write everything up. 

An international comparative study of tuition models in open and distance learning universities

Ann Walshe, a colleague from the school of Computing and Communications, spoke about her visit to Shanghai Open University (SOU) where she was a part of a group of visiting scholars. Ann reported that SOU emphasises vocational and life-long learning. Whilst it does offer bachelor degrees, it doesn’t offer postgrad qualifications. It was interesting to hear that SOU ‘does its own thing’ and tries not to compete with other local and national universities. It has a particular emphasis on blended learning and face to face teaching, having 41 branch schools for both full time and part time students. Interestingly, students have to attend a mandatory F2F induction.

The visiting scholar group were from a range of different institutions, including Chongqing radio and TV university, University of South Africa, the National Open University of Nigeria, Cavendish University, Zambia, Jose Rizal University, Phillipines, and the Netaji Subhas Open University, India (which apparently has 120 study centres, with more opening). Ann’s talk emphasised the importance of distance learning and its global reach.

Unpacking the STEM students’ experiences and behaviours

Jenna Mittelmeier’s presentation was about the challenges of Online Intercultural group work. I enjoyed Jenna’s talk, since it was a very research focussed talk that asked a very specific question: are students more motivated when they study materials related to their own cultural background? In other words, what are the benefits of matching content and activities to the membership of a multi-cultural group? Jenna described a randomised control trial in the context of a Dutch business school. In an activity, students were asked to look at something called the World Bank statistics dashboard and it was found that students participated more when using content from their own background. A qualitative survey suggested that internationalisation (of a study context) did improve participation but did expose tensions. There was an important point, which is that content needs to be made relevant to student’s lives and experiences.

Paper session: supporting students - STEM practice and engagement

Using a dedicated website in the continuing evolution of a statistics community of learners

Rachel Hilliam and Gaynor Arrowsmith introduced us to something called the Maths and Stats Subject Site. Before the university restructured and closed regional centres, students could attend course choice events where they could look at module materials from the regional centre library and talk to academic support colleagues and speak with other students. In an environment that is increasingly digital, an important question is: can we recreate that in an online environment? I made the note that it is (of course) important that students feel a part of a community.  There is a Maths and Stats advice forum, maths education forum, and information about professional and subject societies. There is also advice about preparing to study, revise and refresh resources, are you ready for quizzes, and early units from some modules.

Implementing additional maths support for Health Science students

Nicola McIntyre, Linda Thomson and Gerry Golding spoke about their experiences on SDK100 Science and health: an evidence based approach. An important aspect of the talk was that a maths tutorial was replaced with 18 short videos covering mathematical concepts, such as decimals, percentages, scientific notation and powers. There were also two workshops which were advertised students by email, and two tutors selected and briefed on format of the workshop. I noted an important point: it’s not enough to only provide videos, the workshops are considered to be an essential component.

Two mathematicians and a ukulele

Hayley Ryder and Toby O’Neil are module team members for M208 Pure Mathematics. The module is run through a single ‘cluster’, which means that there is only one group of tutors who teach on the module, and it has 25 hours of tuition sessions. From what I remember, there was a view that students wanted more contact with module team. 

One way to address this is to record a series of informal online tutorial sessions where Hayley and Toby talk through different mathematical concepts and also discuss what is discussed on the module forum. The idea is to convey a sense of ‘what mathematicians do’ and to build ‘mathematical resilience’, a concept that has a number of aspects: (1) the fostering of a growth mindset, (2) that maths has personal value, (3) knowing how to learn maths, (4) knowing how to find appropriate support. The sessions focussed on the first three of these aspects. 

An important point was that the presenters can easily make mistakes when doing things ‘live’ and this shows that real mathematicians can get stuck, just like everyone else. As for the ukulele, this also connects to the concept of learning; this is an instrument that Toby is learning to play (and I understand that he plays it during sessions!)

A secondary analysis of SEAM responses for programming and non-programming modules by gender

Joseph Osunde from the school of Computing and Communication studies the issue of gender disparity in computing and IT. Joseph offered an important comment during the start of his talk: ‘reasons [for gender disparity] may include learning environment[s] that convey gender stereotypes on interests and anticipated success’. To learn more, Joseph has been looking at university Student Experience on a Module (SEaM) survey results.

As a staff tutor, I regularly get to see SEaM survey results and I have my own views about their usefulness as personal development tools and sources of useful research data. This said, Joseph found that there were no significant differences in achievements between gender for modules that required students to learn about programming and those that didn’t. Joseph (with Anton Dil) looked at M250 Object-oriented Java Programming. It turned out that for modules that contained programming, like M250, men seem to be more satisfied with them. When these were again compared with non-programming modules, the result is a bit more mixed. 

Whilst this is an interesting finding, this does suggest that there is some more research to be done. A related question is: to what extend are different people motivated by modules that contain programming? Also, just as our colleague, Gerry Golding has carried out research (which I mention later on) into ‘mathematics life histories’, I do feel that there might be an opportunity to study something that might be called ‘computing life histories’ to understand the qualitative reasons for differences in satisfaction.

Closing keynote for day 1

The closing keynote by Bart Rientes was entitled a ‘critical discussion of student evaluation scores and academic performance at the OU’. Bart began by telling us that he used to be an economics teacher where his teaching performance was regularly evaluated. Drawing on this experience, he asked a significant question: ‘did my increase in my [evaluation] score mean that I was a better teacher?’  He asked everyone who was attending a similar question: ‘are student evaluations a good proxy for teaching excellence?’ Bart directed us to an article, entitled:  Student satisfaction ‘unrelated’ to academic performance – study  that was featured in the Times Higher.

We were given another reference to some published research that was carried out on behalf of the QAA. Digging into the QAA website later took me to two reports that are both connected to the themes of learning, student satisfaction and quality assurance. The first report, entitled Modelling and Managing Student Satisfaction: Use of Student Feedback to Enhance Learning Experience was by Rienties, Li and Marsh. The second report has the title: The Role of Student Satisfaction Data in Quality Assurance and Enhancement: How Providers Use Data to Improve the Student Experience was by Williams and Mindano.

Onto a personal reflection about this (keynote presentations are, of course, intended to get us thinking!) As mentioned earlier I’m very aware of the OU SEaM surveys. In my experience as a tutor line manager, tutors only tend to receive a couple of responses for a group of twenty students, and the students who do respond often have a particular cause to do so. This observation connects back to Bart’s opening point, which is: what can these measure of performance (or satisfaction) tell us? The fact is that education can be difficult and frustrating, but it can also be transformative. Sometimes we only can truly judge an experience (or feel satisfied with it) when the effects of our experience have become clearer over an extended period of time.

Paper session: Supporting students and technologies for STEM learning 

Using student analytics with ALs to increate retention

Gerry Golding spoke about some of his own research into Maths life histories, an idea that, as far as I remember from Gerry’s talk, originated from a researcher called Cobden. Gerry interviewed people to understand how adults coped when studying advanced maths topics and touched on the importance of high school experiences and maths anxiety. Maths life histories can students help to understand the cause of their anxieties and help them to think about what affected them. In turn, these reflections can be used to build and develop self-efficacy to help them through the hard times and facilitate the development of a growth mindset. In terms of this bit of scholarship, initial contact with students is important. Also, the university virtual learning environment (VLE) is not a big deal, because students are studying using books. I have to confess, that I didn’t pick up on the main outcomes of this bit of research, since I started to think about Gerry’s idea of ‘life histories’.

Analytics for tracking student engagement: TM355 Communications Technology

Allan Jones spoke about TM355 Communications Technology, an important module in the computing and IT undergraduate programme. The module has three 10 point blocks, printed books, 3 TMAs and a final exam. It is also a module that makes extensive use of the VLE. 

Students study what is meant by communication technologies and how they work, such as how you modulate waves and signals, encode data and correct errors. The module also makes use of 30 computer aided learning packages. Data analytics are used to track the use of the online parts and comparisons are made between two presentations and students are interviewed to understand their motivations. 

It’s a bit more complicated than that: STEM OU analyse evaluation

Steve Walker asked a question that was implicitly linked to Allan’s presentation: can learning analytics help students to complete modules? The answer was: no… until something is done with the data. The reason for looking at this subject was both simple and important: retention is important and there is the need to figure out what works, for whom and in what context, and why. 

Steve introduced a term that I had never heard of: realist evaluation, and directed us towards a paper by Pawson and Tilley (PDF) which is (apparently) used in medical education. Points that I noted down that sounded important included: mechanisms, interventions, outcome and context. 7 associate lecturers (ALs) were interviewed by members of a module team using something called ‘intervention interviews’. An observation is that the term ‘analytics’ is used in different ways. I also made a note of a simple model, which has the components: identify, diagnose and intervene.

Java specification checking

Anton Dil spoke about the evaluation of a prototype tool for M250 object-oriented programming tutors. M250 students need to write some object-oriented software code. This includes creating something called ‘classes’. These classes have contain a number of ‘fields’ (or data stores) and are designed to carry out certain actions which are started (or invoked) using something called ‘methods’. Student code can be automatically evaluated in a couple of ways: you could write something called a ‘style checker’ to assess what code a student has written, or you could assess its functionality through using something called unit testing. The module team have written a tool called checkM250 that could be used by tutors.

Eight tutors were surveyed and 6 tutors were interviewed. Tutors didn’t use the tool because they didn’t know about it, didn’t have enough time, or didn’t think they needed it. If they did use it, they were likely to recommend it, but they were unsure whether it could highlight things that were missed. I made note of the quote: ‘if you asked me previously whether I missed things, I would say: of course not’. Tutors did report that it could be useful. My own take on tools for tutors is that any tool may be useful (and I write that in the context of being a tutor!) 

Digital by Design: workshop

The conference workshop was, interestingly, run by a theatre group. The key concept behind the workshop was an observation that the ‘coffee break’ discussions in conferences that can be just as useful as formal presentations. Instead of having further talks, the idea was to create a long session that is, in fact, one long coffee break where participants could move between different discussion groups.

Another important idea is that anyone can propose a topic for discussion. Whenever someone decides on a topic, a delegate chose a post-it note that indicates when the topic is going to be discussed, and where in the large room that discussion is taking place. Participants can see a summary of the topics that are being discussed at any one time and participants are, of course, encouraged to move between different groups, according to their own interests. It was a neat idea.

I proposed a topic: how do we develop and support our associate lecturers to do ‘digital’ in the best possible way?’ Examples of other topics included: how should we be using social media apps to communicate with students and each other, how do we become experts in advising students at how to study onscreen, and how do we decide when digital is appropriate and when is it not?

During our ‘coffee’ conversation, I was joined with two colleagues. I soon began to think about whether there might be something that could be very loosely called the ‘digital associate lecturer capability model’. I sketched out a model that had three levels: university systems and tools (such as Adobe Connect), module tools (such as module specific computer based learning products, like there is in TM355) and common IT systems and products (such as Word, Excel and Powerpoint).

During these discussions, I was reminded of a JISC project called Building digital capability that the OU library was connected to and involved with. This also provided a useful framework that could be used to guide AL development. In later discussions, I discovered that colleagues from the OU library were already using this framework in AL development sessions.

Reflections

Everyone’s experience at a conference is, of course, likely to be different. I had a simple objective when I was attending this eSTEeM conference, which was to attend as many presentations as I could to try to get a feel for the breadth of projects that were happening across the university. In some respects, there was one commonality that jumped out at me, and that was the use of videos or personalised recorded tutors that were customised to the needs of students. Underpinning this is, of course, the use of technology.

During the conference, I heard presentations from module teams and presentation from tutors. I also understand that some students were attending too, but I didn’t get to speak or hear from any of them. This links to an important reflection that it is really important to hear the student voice; we need to hear stories about what has worked and what hasn’t worked. This said, eSTEeM scholars are always asking students questions through surveys and module teams are always looking to figure out what works and what doesn’t.

A final thought is this: I’m still not sure is meant by ‘digital by design’ but I don’t think that really matters. We access materials, write materials, and carry out our scholarship using digital tools. What I think is really important is how we use these digital tools in combination with each other. Digital technologies in their various forms might new and seductive, but ‘digital’ tools cannot be transformative if you can’t see or understand how they might be used. There’s something else that is even more important: what really matters in education is people, not machines. It is people who can show us which digital tools can help our studies.

Permalink Add your comment
Share post
Christopher Douce

Getting published in Open Learning

Visible to anyone in the world
Edited by Christopher Douce, Wednesday, 24 June 2020, 10:05

It’s been a few months since I have taken over being the lead editor of a journal called Open Learning (Taylor and Francis website). I’m not on my own, though: there are two fabulous co-editors and an editorial assistant to help me out (thankfully!) The aim of this short blog post is to share some thoughts that might be helpful to anyone who is potentially considering making a submission to the journal. I hope this is useful!

Tip 1: Does your research fit?

The question: ‘does my research fit with the aims and objectives of the journal?’ is, perhaps, one of the most important questions that needs to be asked. This question should be applied to any kind of research that you want to share: some journals are more likely to publish your research if it is more in keeping with the aims and objectives of that journal. Another question is: who is the audience of the journal likely to be? Stop for a moment and imagine who they might be. If you can’t imagine them, or picture what kind of research they might be working on, then you need to consider whether you are looking at the right journal. 

Tip 2: Write a clear abstract

Put another way: clarity is important. Does your abstract clearly summaries the aims and objectives of the research. Also, does it present some clear research questions? I’ve seen papers that have been submitted that do not have an abstract, or have an abstract that just isn’t clear. Although academic papers sometimes be appropriately challenging to read, I’m a great believer in respecting the reader, and a way to show that an author is doing this is simple: take time to write a good abstract. 

Tip 3: Consider what has gone before

A really important tip is to be aware of the literature and debates that presented through the journal; reference earlier debates that have been published. This enables your article to be positioned amongst others. This is important, since as a researcher, as well as looking at the title, and abstract, I regularly look at the references before I even start to read a paper to see how it fits into the work of others. If I see that there are a few papers that have been published in Open Learning before, I view this as a very good thing.

Tip 4: Not too long please!

Make sure that the size of your paper is appropriate for the journal. Open Learning has a limit of seven thousand words. In my short time as editor, I have seen papers that are longer than this. Length is very important, since the publishers (and the editors) are working to a fixed number of pages per issue.

Tip 5: Practice papers are very welcome

Open Learning welcomes papers that present case studies or summaries of professional practice. Although practice papers may not be very theoretical, descriptions of teaching practice and accompanying challenges can inspire theoretical thinking and reflections amongst other researchers. As educational practitioners, always recognise what you’re doing is important and consider writing about it; this is an important aspect of your own professional development and contribution to a community.

Tip 6: Approach the editors

Don’t be afraid of the editors. They want to be helpful, so do ask them questions; they are approachable! If you are not sure whether a paper or research is appropriate, feel free to ask. Also, if you’re interested in getting more involved in a journal (it doesn’t have to be Open Learning) don’t be afraid about being cheeky. Ask to become a reviewer; introduce yourself. Any journal contributes to an academic community, so don’t be afraid to ask to become more involved in that community.

Tip 7: Be patient and engage with the process

This is a very big tip and one that I’m sharing from my own experience. Peer review sometimes feels like a brutal process. Treat the peer review as an opportunity to engage and develop, and again, do correspond with the editor if you have concerns that your own submission has been understood or interpreted by reviewers; dialogue is important. If you ever receive what you think is a negative review, try not to take things personally; they are not criticising you; they are only commenting on what they have read. After reflecting on their comments, do engage and work with the reviewers and the editors. Very often, this can lead to a much better submission than you had ever imagined. Plus, the more that you submit papers, the more experience you get.

Permalink Add your comment
Share post
Christopher Douce

Tutorials and tutorial observations: what works and what helps tutors?

Visible to anyone in the world
Edited by Christopher Douce, Wednesday, 25 Oct 2017, 14:02

As a part of an OU funded eSTEeM research project about tuition and tutorial observations, I ran two short focus groups for associate lecturers at an Open University AL development conference which took place in Leeds between 5 and 6 May 2017. 

This blog post represents a set of notes that have been expanded from comments made on flipcharts during the focus groups. Follow on research is to run a focus group with staff tutor colleagues, and then to consolidate all findings by way of internal and external publications about educational practice.

I’m sharing a summary at this early stage, since I feel that it’s important to be open in terms of the research that has been carried out. Plus, through a blog, anyone who has any opinions about the subject or the session should be free to get in contact.

Introducing tutorial observations

A tutorial observation is, as it suggests, an observation of a university learning or teaching event. It can take place either face to face, or online. 

Ever since joining the university I have been aware that different colleagues (within different departments and faculties) have done observations in slightly different ways. One colleague in one school has used a complex form which was a bit like a questionnaire. Another colleague in my school has had a really very simple form to capture a free form description of what happened during a tutorial.

My research question is: what is the best practice that helps associate lecturers? Given that the university has recently completed a faculty merger, this seems like an ideal time to ask this question. 

Accompanying questions are, of course: what are tutorial observations for? An obvious answer is: to ensure that students are given good quality tuition. Although this may be true, a more detailed answer might be a bit more complicate and nuanced.

Introducing the focus group

In order to find out more, my AL support and professional development said that I could run a workshop that gently masqueraded as a focus group. The ‘focus shop’ had the title: Tutorials and tutorial observations: what works and what helps tutors?

The workshop had the accompanying abstract: do you remember when you last observed during a tutorial? If so, what happened, and were you happy with the feedback that you received? This session is all about the concept of a tutorial observations, both on-line and face to face. Chris Douce is leading a research project that aims to learn more about different observation practices, both inside and outside the university. The research project aims to ask two very important questions: (1) what do tutors need? And, (2) how should staff tutors and faculty managers run effective observations? Other questions include: what feedback would help you the most, and do you have any thoughts about how observations should be run when you do team teaching? All welcome and all feedback appreciated; this session can help to develop and (hopefully) enhance tuition observation and develop online and face to face teaching practices.

What follows is a set of notes gathered from both focus groups.

Points captured from the focus groups

Tutors were asking the important question of: what are observations for and what it its purpose? Is it something that is done to monitor the performance of tutors? There was a view that observations shouldn’t be done in a cursory way, or be paying lip service to an administrative process. 

There are a number of different dimensions to observations: they can range from being formal to being very informal. They can also vary in terms of their participants: they can be of an individual, or they can be of a group of tutors. There are further questions: what about recordings? The question about recordings helped us to start to consider other dimensions of observation: in addition to using discussion forums some tutors have, in the past, created their own podcasts, or used tools such as Jing. A suggestion from a tutor was to ask the question: ‘which recordings would you like me to look at?’ and ‘what would you like me to look for?’

There was an awareness that observations have the potential to be negative (or, as noted, be destructive); they can negatively impact on a tutor’s confidence. There was also the point that observations can be used as a way to facilitate a dialogue between a tutor and a tutor manager; after an observation and the receipt of an observation report, tutors may be invited to offer a ‘right to reply’. Another comment was that it should be ‘a two way thing’.

An important question was: how often should observations take place? Opinions about frequency ranged from every two years to every four years, and perhaps be connected with a tutor’s appraisal (which takes place every two years). One tutor reported that they had been observed twice in ten years; another tutor reported they had been observed two times in six months. This raises an accompanying question: now that tutor line management is a lot more complex, who is actually going to carry out an observation? (We now have tuition task managers, lead line managers and cluster managers). 

So, what about the practicalities of carrying out observation? Giving a warning, or notice, was considered to be important. There was also a practice of sending tuition plans to staff tutors in advance of a tutorial so they could see what is planned; some preparatory work needed to be done.

Accompanying the details of the tutorials and the plans, there are other important questions to negotiate; one of those challenges is the extent to which a tutor may wish a staff tutor to be involved in the actual tutorial. Staff tutors might ask the question: ‘what would you like me to do?’ as a way to being negotiation about the extent of involvement. The practicalities of engaging in a tutorial can, of course, depend on the subject and its level.

Feedback was a theme that recurred a number of times. To prepare for an observation, one tutor suggested the use of the question: ‘what would you like me to look at?’ There was also a suggestion that staff tutors should look at only a few things during a tutorial. There was also an emphasis on the importance of expectations. 

A further comment is that feedback should emphasise the good bits, and this is something that could be done immediately after a tutorial. A key phrase I noted down was: ‘how do you phrase things not to be critical?’ An immediate response was to use a ‘feedback sandwich’. 

As expected, the way in which feedback was presented to tutors differed: the school of health and social care used a form, whereas in the school of maths, tutors were sent a letter.

There were a number of other really interesting points that were raised. A question was: perhaps we should ask students what they want? Also, there are opportunities to share examples of practice, activities and reflections. This raises an interesting question about the importance and use of peer observations. This is, of course, connected to the important issue of trust between the observed and the observer. Other points were made about the connection to the importance of correspondence tuition and the role of mentoring.

There was an important acknowledgement that tutorials and tutorial observations can, of course, be stressful and a recognition that personalities play a fundamental role in shaping the teaching environment in which teaching takes place.

Summary

The two focus groups were very different in their composition, but there was a lot of crossover between the themes that emerged: both suggested, for example, the idea of focusing on a selected number of aspects, and there were different experiences in terms of how frequently observations were carried out.

These notes are influenced by one very big factor: myself. I am the researcher, but I am also a tutor, as well as a line manager of tutors. This means that I am the observer as well as being the observed. All this means is that my own views have necessarily affected how I have interpreted and presented the points that have arisen from the two focus groups. This closeness to the subject will, inevitably, cause me to emphasise some points over others.

As mentioned earlier, the next steps in this project is to run a series of focus groups for staff tutors. 

Permalink Add your comment
Share post
Christopher Douce

Using the Kindle for research and studying

Visible to anyone in the world
Edited by Christopher Douce, Monday, 3 Nov 2014, 15:03

I have to confess, that when it comes to some technologies, I am a bit of a laggard.  It was only very recently that decided to get to grips with understanding the mysterious world of eReaders.  I have two excuses: the first is that there’s just so much ‘tech’ to keep on top of, which means that it’s difficult to know what to do next (which is actually a pretty lame excuse), and secondly, I’ve always been a bit sceptical about the screen quality of eReaders.

About a month ago, I requested a new book from the University library to do some preparatory reading for a new course I’m involved with.  The library turned around my request pretty quickly, but they also sent me an email that suggested that I’ve got some figuring out to do.  It was: ‘we only supply that text book in eBook format’.  No dead tree variety?  No, apparently they don’t do that anymore.

Back at home, I searched around for a box that contained a discarded Christmas present that one of my relatives had received and had then given to me after a couple of months; it was an Amazon Kindle.  After figuring out how to give it some power, the first thing I did was connect it up to my Amazon account.  I was gradually finding my way into being a ‘contemporary reader’.

This blog might be useful for anyone who has to use these eReader devices for their studies.  It might also be useful for any of my colleagues who have to battle with the mixture of convenience and frustration that accompanies the use of eReaders. 

I say ‘eReaders’, what I actually mean is ‘Kindle’, for now.  And when I say ‘Kindle’, is actually the really old ones with keyboards and black and white screen, and not any those new-fangled colour models.

The first section is all about figuring out how to read a text book.  The second section is all about how to download Open University on-line materials to your device (so you can read it on the go).  Some of the OU courses are presented entirely on line.  Two examples of this are: TT284 Web technologies, and H810 Accessible on-line learning: supporting disabled students. I describe how you might (potentially) go about downloading an on-line course to your device, so you can get ahead with your studies.

The third part is a bit of useful fun.  I asked myself the question, ‘I wonder what books I can get hold of for free?’  The answer is, ‘actually, quite a few’.  In the final section I share a few tips about how to download books that are out of copyright.  I, for one, haven’t been a great reader of the classics (I’ve been too busy messing around with computers; another lame excuse), but there are loads that are clearly available.

Working with text books

Apparently, the OU has a website called Mobile Connections, which offers some guidance about the use of mobile devices (OU website) and pointers to mobile strategy documents.  This is all very well, but how do I get a text book onto my device.

After clicking around the university library and attempting to access the text book that I wanted to ‘take out’, I was presented with the following message: "Patrons using iPads, iPhones or Android devices can download and read EBL content via the free Bluefire reader app. "  Now, I don’t have an iPad or an iPhone, and I’ve explicitly made a decision not to read any textbooks on my Android phone simply because my eyes are not up to it.  I haven’t heard about the Bluefire app, but the Bluefire website may or not be useful.

Another part of the library message was that "Downloaded EBL ebooks can also be transferred to any portable ebook reader that supports Adobe Digital Editions (ADE). There's a list of these compatible devices on the ADE website"

I had never heard of Adobe Digital Editions before but I’ve managed to find an Adobe website that offers a bit of information.  I had a good look on the ‘compatible devices’ list and my Kindle device wasn’t listed, which was pretty frustrating (to put it mildly).

All this frustration highlighted a division between two different formats: one called ePub and another called mobi.  Apparently ePub is an open standard, whereas mobi is owned by Amazon.  I soon saw that you couldn’t put ePubs on my Kindle, which was a bit rubbish.

 I asked myself two inevitable questions: ‘is it possible to convert an ePub to a mobi, and if you can, how do you do it?’  Thankfully, the internet is a wonderful thing, and I soon found a product called Calibre (website).  Calibre is described as a ‘free and open source e-book library management application developed by users of e-books for users of e-books’.  It’s a tool that you can download onto your PC, put an ePub in one side, and get a Kindle mobi book out of the other (with a bit of clicking and messing around in between).

 One thing that Calibre can’t do is take account of DRM.  DRM, or digital rights management, is used to protect media from being copied between different devices (which is why you need software like the Amazon Digital Editions).  If your ePub is protected by DRM (or, someone has said that you can’t copy it), then you can’t convert from one format to another.

For sake of argument, let’s say you’ve got a freely available text book that is useful with your module.  How do you go about transferring it to your Kindle?  In my naivety, I thought I could use the ‘old school’ technique of plugging it into the USB port of my computer and dragging files around.  Unfortunately, due to local OU system policies, staff cannot to write data to external USB devices due to an information security management policy.   As soon as I connected up my Kindle, I was presented with a message that read, ‘do you want to encrypt your device?’  If you’re ever asked that question in response to any e-reader you have, say ‘no’ straight away.  Thankfully, I did have the foresight to say no, as otherwise my Kindle would have probably been rendered useless.

Since I was unable to transfer my mobi files directly from my PC to my Kindle, how should I do it?  The answer came from a colleague: you email the books or any files that you want to read through your device to your Kindle account.  When you’ve done this, and you turn on your Kindle, magic happens, your document is downloaded.  If you’re interested, Amazon have some helpful pages (Amazon website).

Working with OU resources

More and more OU resources are being made available in Kindle and ePub formats.  This, I believe, can only be described as a ‘very good thing’ since some of the OU books can be pretty bulky.  When you’re working with an eReader, you can sometimes put all your module materials on your device.  When I go to tutorials, I tend to bring all the OU books with me – but rather than carrying them, I have them all preloaded on a Kindle.  This said, I am a great fan of paper; you can do things on paper that you can’t do with electronic devices and visa-versa, i.e. you can search for a term in an eBook, and you can scribble in your books with different coloured pens (and stick things between pages).

Not long after starting to mess around with my Kindle I realised I could do exactly the same with the other module materials I need to work with from time to time. I quickly realised that there would be a problem: things would start to get pretty confusing if I had all the different eBooks in one place on my Kindle.  Thankfully, there is a concept of a category.

After emailing a load of different mobi books to my Kindle, I noticed that my ‘TT284 category’ (I thought it was a good idea to group resources based on module code) became quickly overloaded, and I noticed that the default display order was the order in which the books were downloaded in.  Although this was useful, I got myself into a bit of a muddle with the download sequence.  I soon realised that it’s possible to change the ordering according to the title which made for a really nice sequence of module materials.

I’ve now got categories for all of the different modules I have downloaded resources for: H810, TT284 and M364 Fundamentals of Interaction Design.  For M364, I have a mobi version of the assignment booklet, and PDF copies of the four blocks.  I don’t, however, have a copy of the set text. 

The M364 set text is huge, and it’s a real pain to carry around, and students have regularly asked whether there are electronic versions that they could download.  Unfortunately, publishers are only just beginning to catch up with the new ways in which institutions and students consume their materials.  For now, we’ve got to battle on with a mixture of paper text books and OU materials which can be provided in a digital format.

Free books!

After months of it being in a box on my shelf, I’ve finally figured out how to use my Kindle.  Now that it’s jam packed with learning resources and I’m getting used to its screen (which isn’t too bad), I started to think about how I might use it to read stuff ‘for fun’, i.e. using it to read novels and non-fiction.

I quickly remembered Project Gutenberg which was a project dedicated to digitising books that were out of copyright.  I took another quick look at this and discovered that they now had books in eBook format, which was great news.  A quick look around took me to an interesting page called the Best Books Ever Listings (Project Gutenberg) I also discovered all these different ‘bookshelves’ organised by topic.  I really recommend that you have a good look around.

Another really good source of free (or really cheap) books is Amazon.  Within minutes of looking around I found a number of classics that I had never read before.  I clicked on a ‘buy’ button, and these new books were delivered to my device.  (Plus, since an eBook doesn’t have a cover, you can download some particularly racy books and read them when you’re on the train and no one would be any the wiser…!)

And finally…

As I said earlier, it sometimes takes me a while to get on top of a technology; I used to be someone who always wanted to mess around with the latest technologies and gadgets.

I don’t really know why it’s taken me so long to get to grips with eReaders.  I’m someone who likes the feel and smell, and flexibility of physical books.  This said, I’ve come to see that eReaders can give learners a flexibility that they never had before; an ability to carry everything around easily, and the ability to search for terms and phrases.  When a lot of material has moved ‘on-line’, eReaders can help us to access content in a convenient way without being always tied to a computer.  I think this is a really good thing.

I’m someone who loves to make notes.  One thing that you can’t do (very easily) is make scribbly notes on eBook pages, but that is okay: I’ll just have to figure out some new study strategies.

The more that you look at something, the more you think about different possibilities.  Looking at the Kindle has caused me to ask myself a further question, which is: how might you create an eBook from scratch?

Permalink 2 comments (latest comment by Leticia Briscoe, Tuesday, 29 Oct 2019, 21:44)
Share post
Christopher Douce

Scholarship for Staff Tutors

Visible to anyone in the world
Edited by Christopher Douce, Thursday, 6 Mar 2014, 16:37

I haven't really blogged about 'internal events' before.  I think this is my first one.  Although I've written this post mainly for an internal audience, it might be useful for a wider audience too, although I'm not yet sure whether I'll click on the 'make available to the world' box in our VLE blogging tool.

About a week or so ago I was lucky enough to attend what is called a Staff Tutor Staff Development event that was all about scholarship.  It was all about how we (as staff tutors) might fit scholarship into our day job.

The SD4ST scholarship event was hosted by the Open University office in Gateshead, a part of the country I had never explicitly visited before (other than passing through either in the car or on the train).  The Gateshead office is fabulous (as is the architecture in Newcastle).  The office presented us with a glorious view of the millennium bridge and the imposing Baltic contemporary art gallery.  I'm digressing before I've even begun, so, without further ado, on to describing the event.

Introducing scholarship

The first day kicked off (in the early afternoon) by asking the question of, 'what exactly counts as scholarship?'  An underlying theme was how to contribute to research that might be used as a part of the university REF submission (which is, of course, used to assess how well universities compare to each other in terms of their research output).

A number of different types of scholarship were defined, drawing on a paper that had recently been circulated (or published) through senate.  The paper also included explicit examples, but I won't go through them here.  Here's my attempt at summarising the different types:

  • Institutional scholarship (about and for the institution)
  • Scholarship of teaching (the investigation of one's own, or teaching by others)
  • Scholarship for teaching, or outputs that contribute to teaching materials in its different forms
  • Research that relates to and can inform professional practice (in whatever form this might take)
  • Discipline based scholarship, investigative research which can be understood in terms of adding to the knowledge of a particular subject or area.

The output of scholarship may be presented within journal or conference papers, chapters in books or be in reports.  Blogs can also be considered as scholarship too, but this is a rather difficult category, since it has to be a 'rated blog'.  In essence, an output should be something that can be judged as excellent by a peer, capable of use by others and have impact.

I thought about which of these categories I could most readily contribute to.  I came up with a couple of answers.   The first was that I might be able to carry out some discipline based scholarship, perhaps building on some of the accessibility or computing research I have been previously involved with.  Another idea might be to do some research that might inform the different course teams I'm involved on.  An example of this might have been an earlier blog post on mobile technologies that fed into course team discussions.  Also, given my current duties of supporting course presentations I could also see how I might be able to (potentially) contribute to the scholarship of teaching in a number of different ways.

How to find the time

Although I'm relatively new to the role of a staff tutor (or regional academic), I am beginning to feel that we have to be not only good at juggling different things, but also be able to put on a good balancing act too! 

The reason for this is that our time is split down the middle.  On one hand we have regional responsibilities (helping our tutors to do their job as effectively and as efficiently as possible, and doing a lot of other mysterious stuff, like marketing) which accounts for fifty percent of our time.  The other fifty percent of our time is spent on 'faculty' work.  This means that we are able to contribute to course teams, offering useful academic input and ensuring that our associate lecturers are fully taken into consideration during the course design phases.  We can also use this fifty percent slice to carry out scholarship in its different forms.

Given the different pulls from course teams and regional responsibility there is a significant question which needs to be asked, namely: 'how is it possible to do scholarship when we've got all this other stuff to do?'  The second section in the day aimed to answer this exact question through presentations by two staff tutors who seem to be successfully balancing all their different responsibilities.

The first presentation was by Dave McGarvie, Science Staff Tutor in Scotland.  Dave gave a cracking presentation about his research, which was all about volcanoes (I'm sure he will be able to provide you with a better description!)  What struck me about Dave's presentation was that he was also came across as being a bit of a 'dab hand' at media stuff too, being called upon as an 'expert' to talk about Icelandic volcano eruptions.  Dave talked about how he used his study leave (he uses all of it), and said that it is possible to ask for research leave too (which was something that I hadn't heard about).

The second presentation was by Gareth Williams, Maths and Stats Staff Tutor (or MCT), in Manchester.  Gareth told us about how he managed to carve out (and protect) a 'research day' which he used to speak (and work with) with other academics in his subject area.

I noted down a really important part of Gareth's presentation which summarised the reasons for doing research: that it is something that we're passionate about, that it's fun, it can help us to maintain knowledge, it can be exciting, it can help with networking (and recruitment of good ALs), and help to introduce and advertise the work of the university to a wider audience.

One fundamental point was echoed by both presenters, namely, that research can take a lot of time, can (and probably will) eat into our personal time.  Gareth offers some practical advice, urging us to be realistic, develop multiple strategies (or routes to publication), prioritise workload carefully and, importantly, to have fun.

The final talk of the day was by Ian Cook, who spoke about the universities eSTeEM initiative which replaces earlier funding mechanisms.  eSTeEM lets individuals or group of researchers to bid for funding for projects that may able to benefit the university or help to further understand and promote teaching and learning.

Designing a scholarship project

The next part of the day (and a part of the following day) was spent 'in the deep end'.  We were put into groups and asked to work towards creating cross faculty scholarship project which could help us to collectively understand our knowledge of Open University teaching and learning (perhaps through the use of technology).  Following the group discussions, we then had to devise an eight minute presentation to everyone in the room to try to 'win' a pot of imaginary funding.  Here's a rough list of the titles of the various projects that were proposed:

  • Group 1: Can on-line forums enhance students learning?
  • Group 2: What constitutes useful monitoring for associate lecturers?
  • Group 3: Investigate if text messaging can improve TMA (assignment) submission and retention
  • Group 4: Why do students attend (or not attend) synchronous on-line tuition?
  • Group 5: A system for the sharing of media resources between tutors

I have to confess I was involved in group five.

This activity reminded me that different people can understand research (and, subsequently, scholarship) in different ways.   In my 'home discipline' of computer science, research can be considered in terms of 'building stuff'.  This 'stuff' might be a new software system, tool or environment.  The 'stuff' might even be a demonstration of how different technologies may be connected together in a new or novel ways.   I also must confess that my discipline background emerged through our brainstorming activities.

In the end, there were two winners, and it interesting to learn that one of the winning project ideas (the use of text messaging) was the subject of an existing project.  It just goes to show that old adage that good ideas can emerge independently from different (independent) sources!

I enjoyed this activity.  I remember a lot of discussion about dissemination and how to evaluate whether a project had succeeded.  Referring back to the earlier notions of scholarship and Gareth's multiple routes to publication, dissemination can, of course, have a range of different forms, from internal presentations, workshops, focus groups, through to formal internal reports and REFable publications, such as conference and journal papers.

Final presentations

The event was rounded off by two presentations.  Celia Popovic gave a presentation about SEDA, which is an abbreviation for the 'Staff and Educational Development Association', which is a non-profit organisation which aims to facilitate networking and sharing of resources.  Celia begins by asking the question, 'what do you need [to enable you do to your scholarship and research stuff]?' and talked us through a set of different resources and the benefits of being a SEDA fellow.  The resources included books, a magazine, and a number of scholarly journals.

The final presentation, entitled 'Getting Started, Overcoming Obstacles' was by Karen Littleton.  Karen is currently the director of CREET which is a cross-faculty research grouping which comprises of Education and the Institute of Educational Technology (and some others too, I am sure!) 

A couple of things jumped out at me, namely, her advice to 'be pragmatic'.  I am personally guilty of 'thinking big' in terms of research ideas.  I once had this idea to perform some kind of comparison of different virtual learning environments, but it was something that I have never managed to get around to doing, perhaps because my heart sinks when I see all the obstacles that lay ahead of me.

Karen advises us to consider working on a series of smaller projects which have the potential to contribute towards a main goal.  She also mentions the important issue of time and the need to ring fence and guard it carefully, a point that was echoed throughout the two days of the event.

Summary

I'm only just starting to appreciate the different demands on my work time.  I have been wondering, for quite a while now, how to do 'research' within my role as a staff tutor.  What this event told me was that it is possible, but you need to do it with a high level of determination to succeed.

It strikes me that the best way to do research is to make sure that your research activities are aligned, as closely as possible, to some of the other duties of the role.  Of course, it might be possible to do other research, but if your 'job role dots' are not connected together, seeking permission and making cases to go ahead and do your own scholarship is likely to be so much harder.

A feeling that I have always had is that through research there are likely to be opportunities.  An example of this can be finding stuff out that can inform course production, or, connecting to Gareth's example, making contacts may help with recruitment of associate lecturers.  I've also come to the conclusion that network is important too.  Networking might be in the form of internal contacts within the university, or external contacts within either other higher education institutions or in industry.

A really important point that jumped out at me is that you really do need to be passionate about the stuff that you're finding out about.  The word 'fun' was mentioned a number of times too.

As a result of the event I've been thinking about my own scholarly aspirations.  Before changing roles I had some quite firm ideas about what I wanted to do, but this has changed.  As mentioned before, I think it's a good idea to try to align different pieces of my role together (to align the fifty percent of regional work with the fifty percent of 'other stuff').  I hope I'm making some progress in figuring out how to make the best contribution to both courses and research.  I hope to continue to blog about some of the stuff that I'm getting up to whilst on this journey.

I'm also hoping there is a follow up session next year which might ask the question of, 'how is your scholarship coming along, and what practical things could be done to help you do more of it?'

All in all, a really enjoyable event.  Many thanks to the organisers!  For those who can access internal OU sites (and might be staff tutors), some of the presentations have been uploaded to the VLE STLG workspace.

Permalink 1 comment (latest comment by Jonathan Vernon, Friday, 1 July 2011, 18:15)
Share post

This blog might contain posts that are only visible to logged-in users, or where only logged-in users can comment. If you have an account on the system, please log in for full access.

Total visits to this blog: 2374832