I belong to a couple of research groups within the School of Computing and Communications; I’m trying to find my research home, after not being involved with research for a while. There’s also an informal group called the STEM Education Research Group, which explores topics that are common to some of the groups that I (occasionally) visit.
On 18 January 2023 I attended a research development event that was facilitated by Ann Grand, Senior Lecturer in Astrobiology Education, that was all about mixed methods.
An example
Ann opened with an example, which was also a research question: how are people using their allotments?
You might count how many people are growing different type of crops, or how many hours a week people are ‘using’ their allotments, or you might try to understand ‘why’ they are using their allotment. The nature of the research question might lead to you choosing different methods: you might gather numbers, or you might want to speak with people who grow things on their allotments. I made a note that there’s a difference between multi-methods to answer different research questions, and mixing of methods.
Choosing methods
I noted down a reference to Tashakari and Cresswell, where mixed methods were described as: “research in which the investigator collects and analyzes data, integrates the findings, and draws inferences using both qualitative and quantitative approaches or methods in a single study or a program of inquiry. integrate everything to produce an interpretation” (2007, p.4)
An important question is: how do decide about to use which methods to use? The answer is: It relates to the overall design of what is being studied.
An important point that I noted down was that mixed methods can often take up more time than if a researcher was only using a single method. This leads to the question: under what circumstances should we use them? What is their value?
A reflection that was made during the session is that controlling for variables in education is profoundly difficult, and therefore, it is almost inevitable that we adopt mixed methods to try to understand what the variables might be. They might also be used to mitigate against the impact of extraneous variables. Also being aware of a range of different evidence may enable you to often understand the question, before even carrying out your research.
My colleague Oli Howson made the following point: “quantitative data is lovely for drawling lines around things but humans are messy and colour/context is important”. Understanding the context can, of course which can lead to other (or related) research questions. A research project might not be about asking or understanding a sequence of questions, it may be more of a messy network of questions which exist within a wider research space.
Value of mixed methods research
Mixed methods can be used to investigate and consider bias, and add meaning to data that has been gathered. One useful quote is by Denzin, who writes: “the bias inherent in any particular data source, investigators and particularly method, will be canceled out when used in conjunction with other data sources, investigation, and methods” (p.14, 1978).
Another quote relates to the application of mixed methods, namely that a mixed approach “facilitates generalization to a wider population, especially when the qualitative sample is directly linked to the quantitative sample” (Hesse-Biber & Johnson, 2015).
There is also the importance of being aware of our own biases and being mindful or how we approach any analysis. These points are related to the subject of reflexivity, which relates to how we position ourselves in relation to any research that we do. Sometimes sharing a little bit more about us (and our position) enables us to add validity to any research that we share.
More than methods…
An important reflection is that the choice of methods is one bit of a much broader picture. Our choice of methods reflects what our research paradigm is, and can link to our philosophy of how we view truth and knowledge. In some ways, using two different sets of methods can be an attempt to bridge conceptual differences between interpretivist and positivist world views. In other words, whether truth (or reality) is subjective, or objective.
The ordering of methods is important. A researcher might use a sequential approach, applying one method after another.
A qualitative method might be used to identify concerns held by a community, which could then be brought into a survey method to quantify, or to test the extent of concerns that might be held by a wider community. In other words, a quantitative approach could be used to validate a qualitative finding.
Looking from the other perspective, a survey (perhaps using a standard instrument) might signify some interesting or curious results. Qualitative methods could then be used to explore why a certain group of participants hold a particular perspective. In other words, a qualitative approach can be used to provide explanations to accompany quantitative findings.
During this part of the session, there was also a short discussion about the use of literature surveys. Systematic reviews can apply mixed methods. For example, a literature survey could begin with a set of themes that have been identified by a researcher. This identification of themes could be thought of as a qualitative approach. During the next step, a researcher might then to go onto identity how many papers explore or study those themes.
A further example…
Towards the end of the session, we had a look at an example of some educational research which asked the question: what impact do science shows have on attitudes to career intentions?
I understand a science show to be an engaging demonstration or a talk. In other words, is there an effect on the career aspiration of school children who attended those shows? We had a brief look at some data captured from a research student. This included responses from questionnaires, and responses from a focus group.
Whilst discussing the research methods applied in this study, there was a further discussion point that emerged, which was about the concept of impact. Specifically, how does ‘impact’ relate to your research questions?
Reflections
Whenever research methods are discussed, there are other broader questions which can and should be asked. These questions relate to the philosophy of research, and the nature of truth, and these discussions inform the research paradigm that you adopt. Before even getting into philosophy and paradigms, it is your research questions that should drive everything. When you know the what needs to be found out, you can then think about the how.
It was great to see Creswell mentioned. I first came across his textbook when studying for my MA in Education. Creswell presents a really detailed summary of what mixed methods research is all about and provides a lot of detail about the methods that can be used and applied.
One of the unexpected points that I took away from this session is how systematic literature reviews can both draw on the quantitative and the qualitive. Thinking back to the literature reviews that I did for my MA and other qualifications, it has struck me that I’ve been applying mixed methods research, but in an informal way. Knowing about terminology makes the informal become formal, and also goes a long way to clarifying processes and how this relates to how research is carried out.
References
Denzin, N. (1978) The research act: A theoretical introduction to sociological methods. New York: Praeger.
Hesse-Biber , S. and Johnson, R. (eds) (2015) The Oxford handbook of multimethod and mixed methods research inquiry. Oxford University Press.
Tashakkori, A. and Creswell, J. (2007) Editorial: The New Era of Mixed Methods Journal of Mixed Methods Research 1(1)3 DOI:10.1177/2345678906293042
Acknowledgements
Some of the phrases and quotes shared through this blog have directly come from Ann, who kindly shared her slides following the event. In some of the earlier sections, I’ve added some further points and reflections from other periods of study. Many thanks to Ann for running a useful session.