OU blog

Personal Blogs

Christopher Douce

STEM Education Research Group - Mixed Methods

Visible to anyone in the world
Edited by Christopher Douce, Thursday, 26 Jan 2023, 15:48

I belong to a couple of research groups within the School of Computing and Communications; I’m trying to find my research home, after not being involved with research for a while. There’s also an informal group called the STEM Education Research Group, which explores topics that are common to some of the groups that I (occasionally) visit.

On 18 January 2023 I attended a research development event that was facilitated by Ann Grand, Senior Lecturer in Astrobiology Education, that was all about mixed methods.

An example

Ann opened with an example, which was also a research question: how are people using their allotments?

You might count how many people are growing different type of crops, or how many hours a week people are ‘using’ their allotments, or you might try to understand ‘why’ they are using their allotment. The nature of the research question might lead to you choosing different methods: you might gather numbers, or you might want to speak with people who grow things on their allotments. I made a note that there’s a difference between multi-methods to answer different research questions, and mixing of methods.

Choosing methods

I noted down a reference to Tashakari and Cresswell, where mixed methods were described as: “research in which the investigator collects and analyzes data, integrates the findings, and draws inferences using both qualitative and quantitative approaches or methods in a single study or a program of inquiry. integrate everything to produce an interpretation” (2007, p.4)

An important question is: how do decide about to use which methods to use? The answer is: It relates to the overall design of what is being studied.

An important point that I noted down was that mixed methods can often take up more time than if a researcher was only using a single method. This leads to the question: under what circumstances should we use them? What is their value?

A reflection that was made during the session is that controlling for variables in education is profoundly difficult, and therefore, it is almost inevitable that we adopt mixed methods to try to understand what the variables might be. They might also be used to mitigate against the impact of extraneous variables. Also being aware of a range of different evidence may enable you to often understand the question, before even carrying out your research.

My colleague Oli Howson made the following point: “quantitative data is lovely for drawling lines around things but humans are messy and colour/context is important”. Understanding the context can, of course which can lead to other (or related) research questions. A research project might not be about asking or understanding a sequence of questions, it may be more of a messy network of questions which exist within a wider research space.

Value of mixed methods research

Mixed methods can be used to investigate and consider bias, and add meaning to data that has been gathered. One useful quote is by Denzin, who writes: “the bias inherent in any particular data source, investigators and particularly method, will be canceled out when used in conjunction with other data sources, investigation, and methods” (p.14, 1978).

Another quote relates to the application of mixed methods, namely that a mixed approach “facilitates generalization to a wider population, especially when the qualitative sample is directly linked to the quantitative sample” (Hesse-Biber & Johnson, 2015).

There is also the importance of being aware of our own biases and being mindful or how we approach any analysis. These points are related to the subject of reflexivity, which relates to how we position ourselves in relation to any research that we do. Sometimes sharing a little bit more about us (and our position) enables us to add validity to any research that we share.

More than methods…

An important reflection is that the choice of methods is one bit of a much broader picture. Our choice of methods reflects what our research paradigm is, and can link to our philosophy of how we view truth and knowledge. In some ways, using two different sets of methods can be an attempt to bridge conceptual differences between interpretivist and positivist world views. In other words, whether truth (or reality) is subjective, or objective.

The ordering of methods is important. A researcher might use a sequential approach, applying one method after another. 

A qualitative method might be used to identify concerns held by a community, which could then be brought into a survey method to quantify, or to test the extent of concerns that might be held by a wider community. In other words, a quantitative approach could be used to validate a qualitative finding.

Looking from the other perspective, a survey (perhaps using a standard instrument) might signify some interesting or curious results. Qualitative methods could then be used to explore why a certain group of participants hold a particular perspective. In other words, a qualitative approach can be used to provide explanations to accompany quantitative findings.

During this part of the session, there was also a short discussion about the use of literature surveys. Systematic reviews can apply mixed methods. For example, a literature survey could begin with a set of themes that have been identified by a researcher. This identification of themes could be thought of as a qualitative approach. During the next step, a researcher might then to go onto identity how many papers explore or study those themes.

A further example…

Towards the end of the session, we had a look at an example of some educational research which asked the question: what impact do science shows have on attitudes to career intentions? 

I understand a science show to be an engaging demonstration or a talk. In other words, is there an effect on the career aspiration of school children who attended those shows? We had a brief look at some data captured from a research student. This included responses from questionnaires, and responses from a focus group. 

Whilst discussing the research methods applied in this study, there was a further discussion point that emerged, which was about the concept of impact. Specifically, how does ‘impact’ relate to your research questions?

Reflections

Whenever research methods are discussed, there are other broader questions which can and should be asked. These questions relate to the philosophy of research, and the nature of truth, and these discussions inform the research paradigm that you adopt. Before even getting into philosophy and paradigms, it is your research questions that should drive everything. When you know the what needs to be found out, you can then think about the how.

It was great to see Creswell mentioned. I first came across his textbook when studying for my MA in Education. Creswell presents a really detailed summary of what mixed methods research is all about and provides a lot of detail about the methods that can be used and applied.

One of the unexpected points that I took away from this session is how systematic literature reviews can both draw on the quantitative and the qualitive. Thinking back to the literature reviews that I did for my MA and other qualifications, it has struck me that I’ve been applying mixed methods research, but in an informal way. Knowing about terminology makes the informal become formal, and also goes a long way to clarifying processes and how this relates to how research is carried out.

References

Denzin, N. (1978) The research act: A theoretical introduction to sociological methods. New York: Praeger.

Hesse-Biber , S. and Johnson, R. (eds) (2015) The Oxford handbook of multimethod and mixed methods research inquiry. Oxford University Press.

Tashakkori, A. and Creswell, J. (2007) Editorial: The New Era of Mixed Methods Journal of Mixed Methods Research 1(1)3 DOI:10.1177/2345678906293042

Acknowledgements

Some of the phrases and quotes shared through this blog have directly come from Ann, who kindly shared her slides following the event. In some of the earlier sections, I’ve added some further points and reflections from other periods of study. Many thanks to Ann for running a useful session.

Permalink Add your comment
Share post
Christopher Douce

eSTEeM Conference – Milton Keynes, May 2014

Visible to anyone in the world
Edited by Christopher Douce, Tuesday, 20 May 2014, 09:50

On 5 May 2014, I was at Milton Keynes again.  I had something called a module team meeting in the morning.  In the afternoon I attended an OU funded conference that had the title (or abbreviation) eSTEeM (project website).

eSTEeM is an initiative to conduct research into STEM education.  STEM is an abbreviation for Science Technology Engineering and Mathematics.  Since I have some connections with some computing modules, which can cross the subjects of Engineering and Mathematics, I decided to submit a proposal that had the objective of learning more about the tutor’s experience of teaching computer programming.  The aim was simple: if we learn more about the tutor’s challenges, we can support them better, and subsequently they will be able to support the students better too.

I have been lucky enough to receive a small amount of funding from the university.  This, of course, is great news, but it also means that I’ve got even more work to do!  (But I’m not complaining - I accept that it’s all self-inflicted, and it’s work that will allow us to get at some insights).  If you’re interested, here’s some further information about the project (eSTEeM website).

A 'pilot' project

The ‘understanding the tutors and the students when they do programming’ project is a qualitative study.  In this case, it means that I’ll be analysing a number of interviews with tutors.  I’ll be the first to admit that it’s been quite a while since I have done any qualitative research, so I felt that I needed to refamiliarise myself with what I needed to do by, perhaps, running a pilot study.

It wasn’t long before I had an idea that could become a substantive piece of research in its own right. I realised that there was an opportunity to run a ‘focus group’ to ask tutors about their experience of tutoring on another module: T320 Ebusiness Technologies (OU website).  The idea was that the outcome from this study could feed directly into discussions about a new module.

During my slot at the conference, rather than talking about my research about programming (which was still at the planning stage), I talked about T320 research, that was just about finished.  I say finished, when what I actually mean is ‘transcribed’; there is a lot more analysis to do.  What has struck me was how generous tutors are with both their opinions and their time.  Their views will really help when it comes to designing and planning the future module that I have a connection.

Final thoughts and links

In case you’re interested, here’s a link to the conference programme.

What struck me was how much ‘internal research’ was there was going on; there are certainly a lot of projects to look through.  From my perspective, I’m certainly looking forward to making a contribution to the next conference and sharing results from the web technologies and programming research project with colleagues.   The other great thing about getting my head into research again is that when you have one idea about what to look at, you suddenly find that get a whole bunch of other ideas.

Permalink Add your comment
Share post

This blog might contain posts that are only visible to logged-in users, or where only logged-in users can comment. If you have an account on the system, please log in for full access.

Total visits to this blog: 2260947