In the discussion of task A11 (pp. 279–81) the account of the students’ utterances is plausible, but why is transcript data to be preferred to the video data for such a visual task?
It is words that are being analysed, use of spoken rather than visual language - however important we know facial expression and body language to be.
A criticism sometimes made of quantitative research is that it uses preconceived categories rather than letting findings ‘emerge’ from the data. The ‘Commentary’ on task A11 (pp. 280–1) is qualitative rather than quantitative, but it could be argued that it also uses preconceived categories.
You have to tell readers what you did and why so that they can draw their own conclusions.
Do you think it would be possible to avoid the use of preconceived categories when analysing this data?
Yes, if it is accepted that an exploratory and iterative form of observation and analysis is valid.
When you read the claim on page 281, did you ask yourself if the researchers had looked at whether this was also true of the control group?
Which is why video is necessary compared to audio, that you need all the information that was available to the participants to decide how they would behave. They’d have had to be blind to be acting on words alone.
Are you convinced that the study effectively demonstrates the authors’ case
Whilst I don’t want to be dismissive of all research because of the bias and problems as long as there is an understanding of this then such research needs to be carried out.
What does the computer add to the analysis?
A new way of doing things and the beginning of ways to analyse ‘big data’ to look for patterns and meaning that was difficult to do before the advent of computers.
What is the status of computer-based text analysis 16 years on? Spend 20 minutes trying to answer this question by searching the web.
Wegerif in 2009 undertaking extensive study of talk in maths teaching - Data collected through baseline standardised tests, diagnostic tasks, video recordings of group work, summaries of teacher meetings, teacher interviews and evaluations.
Talking Counts: An intervention programme to investigate and develop the role of exploratory talk in young children’s arithmetic.
The second strand is to analyse changes in the children's talk. Whole lessons and group interactions are analysed to identify the relationship between talk and children's learning in mathematics.
Mercer, N. and Sams, C. (2006) “Teaching Children How to Use Language to Solve Maths Problems”, Language and Education, 2
The methodology for making this kind of comparison, as described in more detail in Wegerif and Mercer (1997) and Mercer (2004), combines a detailed qualitative analysis of language used by each group of children in specific episodes of joint activity with a quantitative computer-based analysis of the whole corpus of recorded group talk.
Our grateful thanks also to Open University colleagues Dr Martin Le Voi (for his expert assistance in completing the statistical analysis) and Dr Frank Monaghan (for his critical commentary on this paper).
Mercer, N 2010, 'The analysis of classroom talk: Methods and methodologies', British Journal Of Educational Psychology, 80, 1, pp. 1-14, Academic Search Complete, EBSCOhost, viewed 14 February 2013.
• It is difficult to use these methods to handle large sets of data, because they are so time consuming. It is commonly estimated that transcribing and analysing 1 h of talk using such methods will take between 5 and 12 h of research time;
• it can be difficult to use such analyses to make convincing generalizations, because only specific illustrative examples can be offered; and
• researchers are open to charges of selecting particular examples to support their arguments.
• Actual talk, as data, may be lost early in the analysis. A researcher works only with predefined categories, and so new insights which might be gained from repeated considerations of the original data will be missed;
• the use of pre-determined categories or other target items can limit analysts' sensitivity to what actually happens; and
• coding which depends on the decontextualized identification of language features cannot handle the ways that the meaning of any utterance will depend on its history within the observed dialogue and perhaps in previous encounters between participants.
• An efficient way of handling a lot of data; a researcher can survey a lot of classroom language relatively quickly and analyse a representative sample of events;
• enable numerical comparisons to be made across and within data samples, which can then be subjected to a statistical analysis.
• Any transcribed talk remains throughout the analysis (rather than being reduced to
categories at an early stage) and so the researcher does not have to make initial judgments about meanings which cannot be revised;
• any categories emerging are generated by the analysis, not by codings based on prior assumptions;
• in research reports, examples of talk and interaction can be used to show concrete
illustrations of your analysis: researchers do not ask readers to take on trust the validity of abstracted categorizations;
• the development of joint understanding, or the persistence of apparent
misunderstandings or different points of view, can be pursued through the continuous data of recorded/transcribed talk; and
• because the analytic scheme is not established a priori, the analysis can be expanded to include consideration of any new aspects of communication that emerge in the data.
(Strengths and weaknesses above from Mercer)