OU blog

Personal Blogs

H809: Activity 6.2: Effects of audience on research

Visible to anyone in the world

H809: Activity 6.2: Effects of audience on research (1 hour)

In the light of Activity 6.1, look again at the research question you chose for TMA01.

  • What kinds of audience were you assuming for the research findings?
  • How might this research question, and/or the methods you chose, be different for different audiences?

Do first year undergraduate Earth Science students at Keele University exhibit a discrepancy between their actual and perceived core technological literacy skills?


SQ1:   What are the students' perceptions of their levels of expertise in word processing, spreadsheets and presentation packages?

SQ2:    How do students perform in analysis of their skills in word processing, spreadsheets and presentation packages?

SQ3:    Is there a discrepancy between perceived and actual core technological literacy skills?

SQ4:    What are students' opinions on their use of word processing, spreadsheets and presentation packages?


I wrote about this fairly extensively in the TMA as I was assuming the audience to be lecturers from the Faculty of Natural Sciences. I commented that it affected the research methodology greatly as 'good science' is regarded as that from quantitative research and so I chose three out of the four sub-questions with the aim of analysing them quantitatively. I also wanted some more in depth analysis to inform later work so I slid in a fourth, qualitative, sub-question!

There are other, qualitative ways in which the central research question could be investigated. For example the same question aimed for presentation to the Faculty of Humanities and Social Sciences would be better received with the depth and detail provided with qualitative analysis. Qualitative analysis can also create openness when the respondents are encouraged to expand on their responses which may open up new topic areas that the researcher had not previously considered.


Permalink Add your comment
Share post

H809: Activity 6.1: Audiences podcast (2 hours)

Visible to anyone in the world

H809: Activity 6.1: Audiences podcast (2 hours)

Listen to the podcast, and consider how the issues raised might be reflected (or not) in the Block 1 readings.


ESRC concerned in user engagement and that effects how you turn your research idea into something they will fund

Policy-directed initiatives want specific and timely reporting; want guidelines; focused on one area; restrict language i.e. personalise it;

Academics can follow the money so influences research

Research paper for journal - no fixed audience so report on everything

Different types of journals - some have general audience; some specialist

Conference - draw more widely on work you have done; can be seen as stepping stone to journal article

Books - more interactive to write chap of book;

Heavy peer review for journal but lots of informed feedback for book chapter

Static writing of complex social situations is hard to portray; easier in conferences with video, photo and audio.

Can have censoring of policy directed research before allowed to cite elsewhere - may occur after contracting

Interdisciplinary nature of educational technology - can publish in wide area


Hiltz & Meinke, 1989

Starr Roxanne Hiltz:

Ph. D. in Sociology, Columbia University, June 1969

M. A. in Sociology, Columbia University, June 1964

A. B., Vassar College, June 1963, magna cum laude, Major in Sociology, Minor in Economics

Robert Meinke:?

Teaching Sociology (TS), published quarterly, provides articles, notes, and reviews intended to be helpful to the discipline's teachers. Articles range from experimental studies of teaching and learning to broad, synthetic essays on pedagogically important issues. The general intent is to share theoretically stimulating and practically useful information and advice with teachers.

Impact Factor: 0.582
Ranked: 75/114 in Sociology and 87/139 in Education & Educational Research


New Jersey Institute of Technology / Upsala College (now closed after financial problems in 1980s)

Annenberg/CPB Project funded Virtual Classroom at NJIT.

Annenberg Foundation is private foundation established in 1989, supports non-profit organisations

Wegerif & Mercer 1997

Rupert Wegerif - Dialogical approaches to teaching and learning with ICT

Neil Mercer - BSc (Hons) in Psychology, University of Manchester
PhD in Psycholinguistics, University of Leicester
Chartered Psychologist

Language & Education provides a forum for the discussion of recent topics and issues in the language disciplines which have an immediate bearing upon thought and practice in education. Articles draw from their subject matter important and well-communicated implications for one or more of the following: curriculum, pedagogy or evaluation in education.

The task of the Journal is to encourage language specialists and language in education researchers to organise and present their material in such a way as to highlight its educational implications, thereby influencing educational theorists and practitioners and therefore educational outcomes for individual children.

Open University

Roschelle 1992

Jeremy Roschelle:

  • Ph.D., Education/Cognitive Science, University of California, Berkeley, 1991
  • M.A., Education/Cognitive Science, University of California, Berkeley, 1989
  • B.S., Computer Science, Massachusetts Institute of Technology, 1985

The Journal of the Learning Sciences provides a multidisciplinary forum for the presentation of research on learning and education. The journal seeks to foster new ways of thinking about learning that will allow our understanding of cognition and social cognition to have impact in education. It publishes research articles that advance our understanding of learning in real-world situations and of promoting learning in such venues, including articles that report on the roles of technology can play in promoting deep and lasting learning. The Journal of the Learning Sciences promotes engaging and thoughtful participation in learning activities, and articles reporting on new methodologies that enable rigorous investigation of learning in real-world situations.

2009 Impact Factor: 1.767
Ranking: 9/44 in Social Science, Psychology, Educational
Ranking: 15/139 in Social Science, Education & Educational Research

The Institute for Research on Learning (IRL) was a research group in Palo Alto, California founded by George Pake in 1986 through a grant from the Xerox Foundation, It was a non profit research organization that looked at learning in a wide variety of settings, including schools, workplaces, and informal settings, using collaborative, multidisciplinary terms. Research questions were based in real-world problems and settings defined in collaboration with the institutions who hired IRL. IRL had a significant impact on education and knowledge management (among many other fields) not only in the US but globally because of the development of the community of practice idea. Aims to construct an integrated approach to collaboration and conceptual change



Permalink Add your comment
Share post

H809: Activity 4.4: Podcast

Visible to anyone in the world


Activity 4.4: Podcast on examining impact (30 minutes)

Listen to the Week 4 podcast. Peter Twining and Gráinne Conole discuss some of the challenges associated with examining the impact of ICT in education. Peter also describes his research into Second Life.


Research methods for looking at impact of ICT in Education

Conole: Comparison not necessarily

Small scale in depth are fairly easy but not easily transferable

Europe: contextual, mixed methods

States: large scale evidence-based

Twining: driven by US Gov.

Too complex, tiny change can have dramatic impact

Conole: dynamics of group, differences in group, tutor interactions

Twining: measuring the wrong things

e.g. BECTA impact of ICT on student's results in exams - not much as kids are not using ICT in lessons and tests. The things that they learn in ICT are not the things tests measure

Stephen Heppel - 1900s everyone rides on horses and suddenly car invented. Gov wants measure of impact car is having. Take people from cars put them back on horses and measure effect on results in gymkhana

Conole: Surgeon from 1800s in modern hospital would be lost but teacher from 1800s would be fine in modern classroom

Twining: Second Life -

People embedded within current system and cannot think of anything radically different.

Stick them in 3D virtual environment

150 kids 13-17 and stuck them on island. Provide support but them to take responsibility for own learning. Do not know age, gender, cultural background.

Chat log everything

Forum/Wiki for document analysis

Informal interviews/questionnaires

Interviews in world

Is there evidence for in world skills development (geometry, coordinates, measurements) script in world (C-programming)

Knowledge- age skills, collaborative working etc.

Looking at what they are doing, how they progress in dealing with each other in forum etc. Asking teachers and children for ideas on progress

Feed back to staff on team and kids and see if they agree

Results may be 3D environment or may be because they feel safe in this environment.

Conole: complex, exciting area. Look at sociology, globalisation, ICT research.

Methodological challenges are huge and how we translate up to policy makers and down to practitioners is also so important.



Permalink Add your comment
Share post

H809: Activity 4.2: Reading Roschelle (1992)

Visible to anyone in the world


Activity 4.2: Reading the Roschelle (1992) paper

Roschelle, J. (1992) 'Learning by collaborating: convergent conceptual change', Journal of the Learning Sciences, vol. 2, no. 3, pp. 235-76

We suggest that you divide up your reading of the paper in the following way:

  1. Read pp. 235-41 (up to the heading 'The Case Study').

Both 'collaborative learning' and 'conceptual change' are areas of research. As you might guess, 'collaborative learning' is about learning together but it is used by different authors in a variety of ways. (You may remember Hiltz and Meinke (1989) provided their own definition.) 'Conceptual change' refers to changes in people's understanding of concepts. There is a great deal of research into conceptual change in science learning. Roschelle gives his own definition of conceptual change on p. 237.

You might also want to note that 'CA' generally refers to Conversation Analysis. However, 'CA' is often referred to in the paper in the context of 'conversational actions' so that overall we cannot be clear as to Roschelle's exact meaning.

Theory - relational, situated view of learning

As you read consider the following questions:

    • What argument is being made?

That students working collaboratively need to converge to develop a shared meaning and this process is characterised by four features:

Production of a deep features situation
Interplay of physical metaphors
Interactive cycles of conversational turn-taking
Progressively higher standards of evidence for convergence

    • How does it relate to the kind of research being reported?

Students were working in pairs on a computer simulation to refine their concept of acceleration. The pair of students analysed were considered by the author to have achieved convergent conceptual change.

    • How does the research question interact with the research work being undertaken?

Research question: How can the students converge on a deep new conception with only figurative, ambiguous and imprecise language and physical interactions at their disposal?

The study analysed the interactions between the students and between the students and the computer simulation.

    • What kinds of evidence are relevant to the research question?

Body language
Tasks performed on the computer
Individual post-study interviews to determine whether the two students share the same conception.

  1. Read pp. 241-63 (up to the heading 'Evaluation of the Claims').

Read this section through fully before considering the following questions:

    • How are the results/data presented? Compare with how Wegerif and Mercer presented their results in Reading 2.

Extracts from conversations shown in each paper however Roschelle emphasises the context and interactions with the computer and the student's body language and gestures which fits well with the research argument concerning the interactions.

    • What does this say about the kinds of material that count as 'evidence' for the claims being made?

Conversation is not enough to analyse group interactions. He has added in body language, gesture and also physical metaphors such as Carol's demonstration on her hands.

    • What does the author consider relevant about the context or setting for the study?

Private, urban high school
past knowledge: algebra
intellectual ability: average students but struggling with science
Relationship: close friends who had worked together on previous occasions
Participation: voluntary
Timing: 2X one hour sessions after school
Analysis timing: after 12 problems where they had not yet developed an explanation that corresponded to a scientific understanding

  1. Read the remainder of the paper.
    • What is the nature of the claims made in the evaluation?

A shared conceptual change occurred which was compatible with a scientific interpretation of velocity and acceleration.

Gradual convergence towards a shared  understanding.

    • How is the previously reported data used to support these claims?

Data from results section used to illustrate each part of the claims

    • In what ways does the author relate the reported results to the wider literature?

Restates the basis of the problem with reference to literature
Majority of literature uses basis of contructivism but states that few theories account for convergent constructions in face of tendencies to diverge.
Tendency to diverge is strong in science (not backed by reference)

Discussion on how computer simulation can be truly constructive and situated or misused by teacher. How relational theory can assist this.

Differences from Vygotsky and Piaget and how this work supplements it

    • Are the theoretical recommendations justified by the reported research?

'A case study cannot prove or disprove a theory, but it can clarify the meaning and import of a set of ideas'

'Mainstream conceptual change research on science learning should focus attention on convergence'

I believe that this is justified by the evidence produced here from one case study. The case study shows some good indications on how the two students collaborate to produce shared knowledge and it does need further work to see if this case is reproduced for other successful cases and also the difference with unsuccessful cases.


Permalink Add your comment
Share post

H809:Activity 3.8: Reflecting on frameworks

Visible to anyone in the world


Listen to the Week 3 podcast. Peter Twining and Gráinne Conole talk about the various frameworks that have been used to examine the impact of information and communication technology (ICT) on education.


Rhetoric- reality gap: little impact on practice
More joined up in FE/HE
Assumption that education needs to prepare people technologically
Education different from other fields, complex
Disenfranchisement - learners feel school is not relevant, does not use the tools that they use in their everyday lives.

Confusion in research - different terminology to describe the same things
Identified 5 types of frameworks:

  • achievement - measuring people's progress in terms of their learning;
  • cognitive - impact of an individual in terms of their thinking;
  • software - type of software, drill & skill etc or in terms of role of software - technologically determined (positive)
  • pedagogical - e.g. Squires & McDougal - relationship between student, teacher, tool
  • evolutionary - how technology has been rolled out or changed over time

Danger - can be misused: pretty diagram can simplify complexities

Each has their own methodological foundations; give different lenses

Laurillard's conversational framework - dialogic emphasis, not good at wider contextual aspects

Activity theory - wider context

Pedagogy transfers across locations i.e. school, FE, HE

Adapting technology to current practice

  1. In the light of the podcast and this week's work, consider how you might revise the way in which you are making notes on studies. Do the questions from Activity 1.4 need elaborating?

I think it is important to note the date of the research (which may not be the same as the date of publication) and relate that to the context of the time.

Before quoting something form the paper it is important to go back to the source if possible.

Looking at the author/journal to detect any influences and, if possible, relating this to the methodology/framework they have used.


  1. Look back at Reading 1 and consider the questions that were asked in that research. Do you think they represent a dominant 'paradigm' for research in any particular period? Are the research questions and methods still relevant today?

I think I may have partially answered this a little earlier this week so put it here as well so I don't lose it!

The Hiltz & Meinke paper was written in 1989 which correlates with the end of phase two, the stand alone systems, described by Conole et al. (2007). They suggest that stage two shows 'increased activity in terms of multimedia functionality but that it is still content driven and focused on the interactive tutorial paradigm'. Hiltz & Meinke describe methods of teaching that are mainly behaviourist and mainly transmissive with e-lectures although there are some indications of self-determination and learner presentations are assessed. Other parts of the paper suggest a more social and participatory approach. This mixture of methods would link in with the end of stage two and the start of stage three which is characterised by the shift away from the individual and a move to more situated learning as described by Conole et al.

Looking at the research questions:

1) Is the Virtual Classroom a viable option for educational delivery? (On the whole, are outcomes at least as good as those for traditional face-to-face courses?)

2) What variables are associated with especially good and especially poor outcomes in this new teaching and learning environment?

'educational delivery'- emphasising the transmission of information in a behaviourist manner with operant conditioning

Both questions infer that the outcome of teaching depends on the method of delivery and the features that it uses - behaviourist approach.



Permalink Add your comment
Share post

H809: Activity 3.6: Oliver et al. (2007)

Visible to anyone in the world


http://learn.open.ac.uk/file.php/6758/Readings/ebook_h809_reading5_oliver_etal_e1i1_l3.pdf [Accessed 20th February 2011]

Read the Oliver et al. chapter, but in particular concentrate on the section headed 'Methodology' (pp. 30-7). Consider the following questions:

  1. What do each of the various approaches listed highlight?

Action Research Approach - 'technical' - instrumentalist view i.e. use pre-formed models.

'practical' - models provide guidance but practitioner must develop own practice - wider application

'emancipatory' - critical theory - educators recognise constraints- some relation to socio-cultural perspective - establish more egalitarian practices - wider application

Behaviourist - operant conditioning can guide instructional materials and can compare cohorts over time

Activity theoretic perspective - useful to analyse situations to identify problems/conflicts. Used to analyse change.

Socio-cultural perspective - investigating power manifestations

  1. How, if at all, are specific methods (interviews, surveys, focus groups, observation, etc.) and methodological approaches related?


The historical, cultural and political context influences the researcher's interests and thus their choice of topics to study and their methodological approach. Specific methods have been developed on the basis of methodological approaches and affect the types of conclusions that they are willing to draw.

For example, in an interview or questionnaire, the types of questions asked may reflect the culture/interests of the moderator or a difference in culture may cause the respondent to have a different understanding of the questions from the moderator.


Permalink Add your comment
Share post

H809: Activity 3.5: Laurillard (1994)

Visible to anyone in the world
Edited by Lynn Hunt, Sunday, 20 Feb 2011, 12:08


How Can Learning Technologies Improve Learning?

Diana Laurillard, Open University

This paper was presented at the Higher Education 1998: Transformed by Learning Technology, Swedish-British Workshop 14-17 May 1993, University of Lund, Sweden.


  1. What is the main argument?

Funding bodies are asking whether learning technologies improve learning but current studies only demonstrate that they have the potential to do so.

  1. The title to the Laurillard paper poses the question 'How can learning technologies improve learning?' She could also have written a paper about how learning technologies can 'support' or 'change' learning. What does the use of the word 'improve' imply for how elearning technologies are evaluated?

'Improve' suggests that learning is a process that is set and that technologies are an add-on to learning. In this way the evaluation is often carried out as a comparison to using the same methods without the technology that is being evaluated. This may not be the most appropriate or best use of the technology. For example, using online conferencing to deliver the same lecture as delivered to a F2F audience of university students achieves the objective of transmitting the information but is a poor use of the tool which can be much better utilised for more interactive learning. Different teaching methodologies are required. The implications for research are that assessing the learning process with and without a particular form of technology may not be the most appropriate form of research.


Following on from the previous activities, I looked at the UK elearning timeline (Conole et al., 2007). Laurillard's conference paper was presented in 1998 which equates to the end of phase three: networking technologies where they report a move towards more holistic and joined up thinking and suggest there is evidence of more linking of development to strategy and policy. The Laurillard paper shows her interest in this linkage between the research work and how it effects policy in education.

However the Laurillard paper reports on previous work that took place between 1984 and 1993 which corresponds to phase two where there was increased activity in multimedia but teaching practice was still content driven and focused on the interactive tutorial paradigm. This can be seen in the way that Laurillard reports on technologies with comments such as 'computer based learning', 'improve learning' and 'materials should be implemented' with their implications of technology as an add-on to 'normal' teaching.

I was concerned to see that six of the nine papers cited had Laurillard as an author. This can happen naturally if the field of research is small but should not happen in a wide ranging review of educational technology assessment. It is generally considered bad practice to cite yourself to this extent and certainly biases the report. Citation rate is used to assess journals and also, to some extent, authors and so there is pressure to use this technique. Most universities discourage the use of self-citation to this extent and the use of citation clubs is generally considered academic misconduct.


Permalink Add your comment
Share post

H809: Activity 3.4: Comparison of First Two Readings with the Timeline

Visible to anyone in the world

The Hiltz & Meinke paper was written in 1989 which correlates with the end of phase two, the stand alone systems, described by Conole et al. (2007). They suggest that stage two shows 'increased activity in terms of multimedia functionality but that it is still content driven and focused on the interactive tutorial paradigm'. Hiltz & Meinke describe methods of teaching that are mainly behaviourist and mainly transmissive with e-lectures although there are some indications of self-determination and presentations are assessed. Other parts of the paper suggest a more social and participatory approach. This mixture of methods would link in with the end of stage two and the start of stage three which is characterised by the shift away from the individual and a move to more situated learning as described by Conole et al.

The Wegerif & Mercer paper was written in 1997 and describes research published in 1996. According to Conole et al. this time period fits towards the middle of phase three, networking technologies, where we are 'beginning to see more emphasis on the wider contextual issues (skills, strategy, importance of embedding and integration). Also a shift away from the emphasis on the individual to the concept of situated learning'. There is also evidence of 'linking of development to strategy' as the intervention described by Wegerif & Mercer was planned to coach young children in exploratory talk with the aim of improving group problem solving. According to Conole et al., this is more indicative of the growing awareness of collaborative activities which comes into evidence in phase four (after 2000) and so may suggest that the timeline needs adjusting or that this was school utilising innovative techniques.


In our tutor group timeline we have the introduction of social technologies such as the forum, podcasting and the wiki, in 1994 with blogging coming later in 1998. This seems to suggest that, although the technologies may not have been in widespread use in education, they were available in the market place and there would have been a growing awareness of collaborative work. I believe that this would have been likely to effect the attitudes of more innovative teachers at the time and encourage the form of forward thinking that suggested this intervention.


In the Wegerif & Mercer research, qualitative analysis suggested key words which were analysed by a concordance program which illustrates another aspect of phase three where higher education lecturers are using ICT as a 'tool of the trade'.


Permalink Add your comment
Share post

H809: Activities 3.1, 3.2, 3.3: Academic Search Engines

Visible to anyone in the world

H809: Activities 3.1, 3.2, 3.3: Academic Search Engines

ISI Web of Knowledge

Search for Hiltz AND Meinke AND 1989

1. Ridener, L. R. (1999) 'The Good, the Bad, and the Ugly in Cyberspace: Ups and Downs of the Dead Sociologists' Society' Social Science Computer Review 17: 445

'Sociologists have used technological innovations in the teaching of a variety of courses' general reference to Hiltz & Meinke paper

2. Jaffee, D. (1997) 'Asynchronous learning: Technology and pedagogical strategy in a distance learning course', Teaching Sociology, vol. 25, Iss. 4, pp. 262-277

One general reference at start of paper

3. Marttunen, M. (1997) 'Teaching argumentation skills in an electronic mail environment', Innovations in Education and Training International, vol. 34, iss. 3, pp 208-218

No full text available

4. Persell, C. H. (1992) 'Bringing PCs Into Introductory Sociology Courses - 1st Steps, Missteps, And Future-Prospects',  Teaching Sociology, vol. 20, iss. 2, pp. 91-103

In references but could not find in text!!

Search for Wegerif AND Mercer AND 1997

Search found no records

Google Scholar

Search for Hiltz AND Meinke AND 1989

17 results, one of which was the paper itself; some were unobtainable; and two were irrelevant.  Really like the function to set it to find at unis where I have a subscription

1. Hacker, K. L. & Wignall, D. L. (1997) 'Issues in Predicting User Acceptance of Computer-Mediated Communication (CMC) in Inter-University Classroom Discussion as an Alternative to Face-to-Face Interaction', Communication Reports, Vol. 10, No. 1.

'As Hiltz and Meinke (1989) point out, educational networks need to be studied extensively in terms of student and faculty needs, uses, and satisfaction.'

2. Hiltz, S.R. & Turoff, M. (1990) 'Teaching computers and society in a virtual classroom', CQL'90 Proceedings of the conference on Computers and the quality of life, New York, Association for Computing Machinery.

General reference

3. Marttunen, M. (1997) 'Argumentation Course by Electronic Mail', Scandinavian Journal of Educational Research, vol. 41, iss. 1, pp.15-32

'promotes independent thinking and offers a motivating and democratic forum for interaction in which even shy and timid people can express their thoughts (Hiltz & Meinke, 1989..)' p. 28

'comments and critiques are more likely to be directed toward the ideas presented than their presenters (Hiltz & Meinke, 1989)' p.28

'face to face meetings could be introduces occasionally.... (cf. Hiltz & Meinke, 1989) p.29

4. Barreau, D., Eslinger, C., McGoff, K. & Tonnesen, C. () 'Group Collaboration In The Virtual Classroom: an Evaluation of Collaborative Learning in the Virtual Classroom of  CMSC 828S and the Technology that Supports It' Available from: http://www.hitl.washington.edu/projects/knowledge_base/virtual-worlds/JOVE/Articles/ClassEvaluation/Evaluation.main.txt [Accessed: 19th February 2007]

'Studies have shown that successful distance learning occurs when the tasks and activities are appropriate to the technology; are consistent with the instructor's philosophy and style of teaching; are convenient, accessible and relevant to the students; provide maximum interaction; and are well-organized and well-presented' [list of authors inc. Hiltz & Meinke, 1989]

'Student level of participation has proved to be a consistent predictor of reported satisfaction in many studies (Hiltz, 1986; Hiltz, 1993; Hiltz & Meinke, 1989)'

5. Book: Romiszowski, A. & Mason, R. (1992) Computer mediated communication: a selected bibliography. Available from: http://www.aect.org/edtech/15.pdf [Accessed:19th February 2011]


No refs found

Wegerif AND Mercer AND 1997

938 references found! No time to investigate them now but I intend to look at them later as the first three pages threw up some very relevant papers.

Activity 3.2 and 3: Examining, Sharing and Discussing Findings

I found that Google scholar had a much greater range of publications and, once I had set it to indicate the ones where I can access the full text, it was very easy to use. I found books and conference proceedings as well as journal articles and all the journal articles that I found using ISI Web of Knowledge were also included in these results. Google searches across many disciplines and, although they are all reported to come from academic sources, I do not think that it is as reliable as peer reviewed journals and I prefer to check the source. For example, it is wise to be aware of the prejudices of professional societies.

I have put full results in my blog and it was intriguing to find that I did not recognise some of the findings that were reported to come from the Hiltz & Meinke 1989 paper. Further reasons to go back to the original research wherever possible!

I also found that, in some cases, the reference was in the reference list but not in the text!

Once I had set Google Scholar up to indicate availability for my libraries, I found it much easier to use than ISI Web of Knowledge and appreciated the wide range of material available. I do think it is very important to be aware of the source of the material when using Google Scholar and weight it accordingly.


Permalink 3 comments (latest comment by Lynn Hunt, Monday, 21 Feb 2011, 17:37)
Share post

H809: Activity 2.5: Reflecting on the research methods

Visible to anyone in the world


  1. In the discussion of task A11 (pp. 279-81) the account of the students' utterances is plausible, but why is transcript data to be preferred to the video data for such a visual task?

It is much easier to analyse and report transcribed data. Video data is time consuming to analyse as it must be observed and re-observed in order to analyse in detail and it is easier to do this from a transcript. Ease of use means that more raw material can be examined.

Some forms of analysis can be done direct from the recording when the researcher needs to focus on what is going on and not get too focused on the detail of what people have said.

  1. A criticism sometimes made of quantitative research is that it uses preconceived categories rather than letting findings 'emerge' from the data. The 'Commentary' on task A11 (pp. 280-1) is qualitative rather than quantitative, but it could be argued that it also uses preconceived categories.

For example, Elaine's words before the intervention, 'No, because it will come along like that', and the fact that the next utterance is by John on the next question are interpreted as, 'She gives a reason to support her view and this is not challenged.'

Her words after the intervention, 'Now we're talking about this bit so it can't be number 2 it's that one. It's that one it's that one' are interpreted as, 'In proposing number 4 Elaine is building on these two earlier failed solutions' (p. 281).

Wegerif and Mercer have prior expectations about 'exploratory talk', defined as 'talk in which reasons are given for assertions and reasoned challenges made and accepted within a co-operative framework orientated towards agreement' (p. 277).

So notions such as 'reason', 'support', 'challenge' and 'failed solution' have specific, preconceived meanings. Do you think it would be possible to avoid the use of preconceived categories when analysing this data?

Glaser & Strauss (1967) published The Discovery of Grounded Theory which suggests that researchers should ignore the literature and theory and just work on the raw data to produce categories that are not contaminated by preconceptions. This originates from philosophers such as Bacon and Locke but, in that first book they do not account for the idea that 'there can be no sensations unimpregnated by expectations' (Lakatos, 1982, p.15). Later on, they separately discuss the preconceived knowledge and research that researchers have at their disposal before data collection and analysis.

Charles Sanders Peirce discusses the 'heuristic framework' of concepts that inform a researcher and suggests that abductive interference combines the new data with previous knowledge so that pre-conceptions often have to be abandoned or modified.

Following this the preconceptions must be regarded as heuristic concepts which form lenses by which the empirical world can be viewed.

http://www.socresonline.org.uk/2/2/1.html#s4 [Accessed 13th Feb 2011]

I think that it would be difficult to avoid preconceptions when conducting the research but the use of initial coding without concern about categorising the codes would help. Once any relationships had been explored using diagrams, then focused coding can be used to reduce the number of codes and identify repeating ideas and themes.

inductivism - theories have to be based on empirical observations which are generalised into statements that can be regarded as true or probably true. (from Hume)

positivism - the only authentic knowledge is that which is based on sense, experience and positive verification (from Compte)

  1. Again in relation to task A11, what evidence might support the following claim on p. 281?

'In the context of John's vocal objections to previous assertions made by his two partners his silence at this point implies a tacit agreement with their decision.'

Once again it is difficult to discuss this without resort to the original paper but it may be possible to back up this claim with evidence from the video on his facial expression and body language. For example, if he is still engaged with the group activity, this would back up this assumption whereas if he was sitting back or looking at posters on the wall then it would suggest he was bored of the conversation and the authors' assumption would probably be incorrect.

  1. On p. 281, the authors claim:

'It was generally found to be the case that the problems which had not been solved in the pre-intervention task and were then solved in the post-intervention task, leading to the marked increase in group scores, were solved as a result of group interaction strategies associated with exploratory talk and coached in the intervention programme.'

When you read this claim, did you ask yourself if the researchers had looked at whether this was also true of the control group? If time allows, feel free to look at the papers in which fuller accounts of the study appear.

I was concerned that the control group may have a different, less comfortable relationship with the teacher/researcher and still be inhibited in the discussion for the second test.

  1. In the post-intervention talk around problem A11, John says, 'No, it's out, that goes out look'.

This utterance doesn't use the words 'cos', 'because', 'if', 'so' or a question word, but it is plausible that John is giving a reason. How might one deal with such a problem?

A set of rules that identified reasoning behaviours in the video as well as in words

  1. Are you convinced that the study effectively demonstrates the authors' case that:

'the incorporation of computer-based methods into the study of talk offers a way of combining the strengths of quantitative and qualitative methods of discourse analysis while overcoming some of their main weaknesses'?

No, intuitively I agree with the authors but I cannot determine it from the evidence that they present.

  1. What does the computer add to the analysis?

The growing literature on computer assisted qualitative data analysis software (CAQDAS) expresses both hopes and fears. The hopes are that CAQDAS will: help automate and thus speed up and liven up the coding process; provide a more complex way of looking at the relationships in the data; provide a formal structure for writing and storing memos to develop the analysis; and, aid more conceptual and theoretical thinking about the data. In spite of these pros there are a good many criticisms and worries about the software in the literature: that it will distance people from their data; that it will lead to qualitative data being analysed quantitatively; that it will lead to increasing homogeneity in methods of data analysis; and that it might be a monster and hi-jack the analysis.

http://www.socresonline.org.uk/3/3/4.html [Accessed 13th Feb 2011]

  1. What is the status of computer-based text analysis 10 years on? Spend 20 minutes trying to answer this question by searching the web.

I had a look at ATLAS.ti which offers a variety of tools for accomplishing all the tasks associated with a  systematic approach to unstructured data, i.e. data that cannot be meaningfully analyzed by formal, statistical approaches. It is a tightly integrated suite of tools that support analysis of written texts, audio, video, and graphic data. ATLAS.ti brings to the job highly sophisticated tools to manage, extract, compare, explore, and reassemble meaningful segments of large amounts of data in flexible and creative, yet systematic ways.

Create quotations directly from audio and video files, work with or without transcriptions. Link audio to text and videos to photos. Treat videos clips as you would text files, draw connections between any kind of data and content.

  • create quotations in any type of audio / video / image file just like in textual documents
  • annotate multimedia quotations
  • hyperlink between multimedia quotations and text files (and vice versa, of course...)
  • assign multimedia files like textual files as standalone documents


  1. How does this paper compare with Reading 1?

My current thinking is that there seems to be two types of qualitative research: one which draws on theoretical concepts and draws up firm hypotheses which can be proved or disproved during the investigation; and one which uses theoretical concepts in order to produce vague conjectures about possible relationships and then examines these by investigating the raw material.

The Hiltz & Meinke paper uses two explicit research questions with firm hypothesise for each. One it investigates with a null hypothesis and quantitative analysis. The  reason for a null hypothesis is to suit the statistical tests we use to analyse the results

HI: There  will be no significant  differences in  scores  measuring mastery  of  material taught  in  the  virtual  and  the  traditional classrooms.

The other uses mainly one tailed correlational hypotheses in the form of predicting a significant positive correlation.

H2: VC students will perceive  the VC to be superior  to the TC on a number  of dimensions:
2.1  Convenient  access to  educational  experiences;
2.2 Improved  access  to  their  professor;
2.3 Increased  participation  in a course

I feel very uncomfortable with correlational hypotheses as it appears to me that they are likely to influence the researcher when collecting and analysing data. Think this comes from my science background!

The Wegerif & Mercer paper reports on research that seems to have used the second type of analysis with vague conjectures that are confirmed with conversation analysis. However, only a review of the research is presented in this paper and it is hard to judge whether firm hypotheses were used. Unfortunately the OU library only has the journal from 1997 and the full research was presented in an article from 1996! Tried other Uni libraries and cannot get it at all.


Permalink Add your comment
Share post

H809: Activity 2.4: Wegerif & Mercer 1997

Visible to anyone in the world

Wegerif, R. & Mercer, N. (1997) 'Using Computer-based Text Analysis to Integrate Qualitative and Quantitative Methods in Research on Collaborative Learning'. Language and Education, vol. 11, no. 4, pp. 271-286

Questions: What research questions are being addressed?

Can computer-based text analysis can overcome weaknesses of either quantitative or qualitative methods alone?

Illustrated by previously published research that addressed the question of whether a coaching intervention programme improved group problem solving abilities

Setting: What is the sector and setting? (e.g. school, higher education, training, informal learning)
Primary school

Concepts: What theories, concepts and key terms are being used?

Key terms: quantitative, qualitative, concrete, abstract, discourse analysis, concordance

Methods: What methods of data collection and analysis are used? (e.g. the number of participants; the type of technologies; the use of interviews, surveys, observation, etc.)

Main question addressed by using literature review and illustrated by use of previous research work.

Eight lessons coaching exploratory talk
Nine groups of three children aged 9-10yrs
pre and post-intervention evaluation of problem solving: group reasoning test and analysis of video talk
Control: same age children in neighbouring school (5 groups)
All increased but sig diff between control and test groups

Key word in context analysis - Qualitative analysis suggested key words such as cos/because. These used in quantitative analysis by !KwicTex.

Findings: What did this research find out?

Using computer-based transcript  analysis  to help combine qualitative and quantitative methods in the study of collaborative learning can produce an overall interpretation which is more convincing than either qualitative or quantitative  accounts  can  be  if  used  alone.

Limitations: What are the limitations of the methods used?

Using a coding system for qualitative data is regarded as quantitative - surely this is a mixed method as it uses researcher judgement on what and how to code? So it is actually a comparison of qualitative and mixed methods.

P281 - results are reported factually in most cases: 'Graham sees that she is right' - surely this is an assumption?
[John's] silence implies tacit agreement - does it or is he just fed up? May be able to see this from the video but reported here as a fact.

Were they the same tasks? It seems to suggest that they were and so this was a second attempt at the task with a group that were used to working together whilst the intervention was taking place. How were the control group formulated? Were they used to working together?

Who performed the research? One class teacher working across two schools will be known by one group but not the other; presume this may be the intervention group so having a strange person administer test may affect the discussion in the control group and thus results. Independent researcher would also become better known to intervention group with similar affect on the discussions.

How were the groups selected? Did they include all members of a class? Were they inclusive? Young children often have problems coping with group members with impairments such as deafness, blindness, Asperger Syndrome and slow speech. This may affect the results and is not reported, although the full report of this project is reported elsewhere and referred to in the paper. If

Ethics: Are there any ethical issues associated with the research?

Working with vulnerable group - researchers would need enhanced CRB check

Do parents need to be consulted?

Will the intervention give the teaching group an unfair advantage (if it works) over the rest of the year group?

Will the group miss out from other work by being separated from the year group during the intervention and tests?

Implications: What are the implications (if any) for practice, policy or further research?

The paper suggests benefits of using a combined approach for the research of talk and collaborative activity and illustrates one method of such an integrated approach. It is possible to expand these ideas to consider the use of such an integrated approach in other forms of discourse analysis.



Permalink Add your comment
Share post

H809: Activity 1.4: Hiltz & Meinke (1989)

Visible to anyone in the world

Hiltz, S.R. and Meinke, R. (1989) 'Teaching sociology in a virtual classroom', Teaching Sociology, vol. 17, no. 4, pp. 431-46.

Questions: What research questions are being addressed?
To see if CMC systems can improve access and effectiveness of post secondary delivery: is it a viable option? i.e. are outcomes as good?; what variables are assoc. with esp. good or poor outcomes?

Setting: What is the sector and setting?
US College - in the US the terms 'college' and 'university' are loosely interchangeable

Concepts: What theories, concepts and key terms are being used?

"Learning is the structuring of a situation in ways that help students change, through learning, in intentional (and sometimes unintentional) ways" (Johnson & Johnson, quoted on p. 432)
Suggests a behaviourist view of learning in that the students' behaviours are modified in the desired direction. (Operant conditioning, Skinner)

The VC still employs a transmissive format with e-lectures although some self-determination in order of activities is indicated, some active dialogue is employed and presentations are assessed.
Other parts of the paper suggest a more social and participatory form of education following Lave & Wenger's theories.

Methods: What methods of data collection and analysis are used? (e.g. the number of participants; the type of technologies; the use of interviews, surveys, observation, etc.)

  • Matched but 'non-equivalent' sections of the same course; same teacher; same text; same printed materials; same exams;
  • students self-select VC and TC
  • Some mixed mode courses - 25-75% VC (107 & 96 students)
  • Pre and post-course questionnaires
  • Grades and SAT scores
  • Records of time spent online
  • Observation
  • Interviews
  • Case reports
  • VC students still took exams in TC manner
  • Hypothesis testing

Findings: What did this research find out?

  • VC can increase access and effectiveness of college-level education
  • Similar levels of skill in both groups
  • No sig. diff. in final or mid-term scores
  • Some diff. in general scores for comp. sci. Students
  • Students thought VC was convenient
  • Students thought access to professor was improved in VC
  • There was improved participation in VC
  • No sig. diff. in increased level of interest
  • Communication variable; highest in mixed-mode courses

Limitations: What are the limitations of the methods used?

  • Self-selection - more computer literate may choose VC; those with time constraints, perhaps too busy for course, may choose VC;
  • Researcher delivering one course may affect outcomes
  • Two very different unis so introducing another variable - big difference in results
  • Everything in written form so disadvantaging those with SpLD
  • Variations in cognitive maturity
  • VC students formed F2F buddy groups thus changing format of research group
  • Students did not have computers at home or in dormitories so had to go out to access them - those with computers at home most likely to appreciate convenience
  • 50 hypotheses and over 200 variables - too many variables so difficult to determine any correlation e.g. low level student participating, high level student participating, low level student participating, high level student participating - what effect does this have on their opinions of communication?
  • Questionnaire is in complex English for lower level students - strongly agreeing with negative.
  • Compulsion to sign in twice a week reduces flexibility (p.440)

Ethics: Are there any ethical issues associated with the research?

  • Use of pen names could encourage disclosures/comments that were unwise in a class situation and, as course progressed, identities may be inadvertently revealed.
  • VC students may have discussed quiz answers with buddies in computer lab

Implications: What are the implications (if any) for practice, policy or further research?

'Results were superior for well-motivated and well-prepared students who had adequate access to necessary equipment and who took advantage of the opportunities..'

Motivation and access seem to be the main issues with the tutor having a crucial role in motivation by building a collaborative group.

'VC delivery mode seems better for courses that treat a few topics in depth rather than for......courses that attempt to cover a large number of topics in a short time'

I would agree with this as many people on H800 reported that they wanted time to do deeper research on some areas.

Meinke thought VC most suitable for those with advanced reading and writing skills but agreed that it was stimulating for all students as a component of mixed courses.

Activity 1.5: Reflecting on the paper (2 hours)

  1. What counts as evidence in this work?

Quantitative research: questionnaire results, grade scores, exam results, records of time spent online

Qualitative research: interviews, observation, case reports

  1. How do the two explicit research questions relate to the design of the research?

1)  Is the Virtual Classroom a viable option for educational delivery?  (On the whole, are outcomes at least as good as those for traditional face-to-face courses?)

2) What variables are associated with especially good and especially poor outcomes in this new teaching and learning environment?

The first question is addressed by using a null hypothesis and quantitative analysis of the course scores and results. It is a fairly straightforward yes/no answer.

The second is addressed by qualitative methods. I believe that this is the best way of investigating the introduction of new methodologies into such a complex social situation of a university setting. There is no straightforward yes/no answer but a mixture of student and staff perceptions and opinions. I am concerned about the prejudice shown in the hypothesis and in the selection of sub-hypotheses shown. Would this sort of prejudice influence interviews, observations and case reports? Investigating such a complex area needs to be carefully controlled but the authors suggest that they are testing 50 hypotheses and over 200 variables which would interact. An attempt has been made to make the findings more quantitative by using a questionnaire.

  1. In what ways is the wider literature used in the paper?

To reference the full evaluation of results (Hiltz 1986,1988) p.432

To provide a definition for education (Johnson & Johnson, 1975) p.432

To provide reference for:

  • previous work on CMC (p.432);
  • software descriptions (p.433);
  • collaborative learning (p.433);
  • cognitive maturity (p.434);

To reference quote (Harasism p.434)


  1. What views of education and learning underpin the research?

Technology as an add-on rather than an integral part of learning design.

"Learning is the structuring of a situation in ways that help students change, through learning, in intentional (and sometimes unintentional) ways" (Johnson & Johnson, quoted on p. 432)
Suggests a behaviourist view of learning in that the students' behaviours are modified in the desired direction. (Operant conditioning, Skinner)

The VC still employs a transmissive format with e-lectures although some self-determination in order of activities is indicated, some active dialogue is employed and presentations are assessed.
Other parts of the paper suggest a more social and participatory form of education following Lave & Wenger's theories.


Permalink Add your comment
Share post

H809: Activity 1.3: Introduction

Visible to anyone in the world

Hi to everyone.

I am really pleased to be starting this course. It is my last one for MAODE and I have previously studied H800 and H810 and I am still studying E810.

I live in Newcastle under Lyme in the UK at the moment although I am looking for jobs 'down South' and hope to move in the next year or so. I am currently working three days a week, for three different universities supporting students with various impairments. It is a great job and I get to meet a wide variety of really nice people. Technology plays a large part in both enabling and disabling access to course materials and it was this aspect of my job that encouraged me to start studying in this area. My modules have been chosen to compliment my work.

I thoroughly enjoyed H800 and the way I was introduced to so many new technologies that were so relevant to my life and work. I then continued to H810 which was very relevant to my work and initially I found it a little easy but the design and analysis of accessible course materials was fascinating. I studied E801 (literacy difficulties) at the same time and I am continuing it until October. The work load is much less than the H-courses. It contains some interesting material but I have found it a little isolating in contrast to the H-courses as I have missed the contact with the rest of the group.

I initially chose this module as I am considering going on to the EdD or PhD and I wanted to do a research module in preparation. Unfortunately I have just found out that you need 60 credits in research to do this so I will have to do another course - avoiding all the ones with exams of course!

'What kinds of evidence and inquiry methods were appropriate to the subject you studied for your undergraduate degree?'

My initial qualifications were in Biology and Biochemistry and I worked for several years for ICI (now Zeneca PLC). I changed tack and started working with students so I collected some post grad qualifications in Mentoring and Counselling too. I have found it really difficult to change methods from paradigm driven research to the debate-driven educational research. I have been so used to concise scientific writing that I found some of the papers that were full of rhetoric very annoying to read! I am more used to them now and I am even managing to write in the first person occasionally without wincing too much!

I use my blog regularly for notes and thoughts on all the courses I have been doing and I would love to hear from anyone who fancies putting any comments on the posts. I have added the address to the blogs link on the Wiki page. I also use Twitter on a daily basis to share any links that I find to ed tech materials and you can find me at @LMAHunt


Permalink Add your comment
Share post

This blog might contain posts that are only visible to logged-in users, or where only logged-in users can comment. If you have an account on the system, please log in for full access.

Total visits to this blog: 406311