Personal Blogs

H809: Activity 6.1: Audiences podcast (2 hours)

Listen to the podcast, and consider how the issues raised might be reflected (or not) in the Block 1 readings.

Notes:

ESRC concerned in user engagement and that effects how you turn your research idea into something they will fund

Policy-directed initiatives want specific and timely reporting; want guidelines; focused on one area; restrict language i.e. personalise it;

Academics can follow the money so influences research

Research paper for journal - no fixed audience so report on everything

Different types of journals - some have general audience; some specialist

Conference - draw more widely on work you have done; can be seen as stepping stone to journal article

Books - more interactive to write chap of book;

Heavy peer review for journal but lots of informed feedback for book chapter

Static writing of complex social situations is hard to portray; easier in conferences with video, photo and audio.

Can have censoring of policy directed research before allowed to cite elsewhere - may occur after contracting

Interdisciplinary nature of educational technology - can publish in wide area

Hiltz & Meinke, 1989

Starr Roxanne Hiltz:

Ph. D. in Sociology, Columbia University, June 1969

M. A. in Sociology, Columbia University, June 1964

A. B., Vassar College, June 1963, magna cum laude, Major in Sociology, Minor in Economics

Robert Meinke:?

Teaching Sociology (TS), published quarterly, provides articles, notes, and reviews intended to be helpful to the discipline's teachers. Articles range from experimental studies of teaching and learning to broad, synthetic essays on pedagogically important issues. The general intent is to share theoretically stimulating and practically useful information and advice with teachers.

Impact Factor: 0.582
Ranked: 75/114 in Sociology and 87/139 in Education & Educational Research

New Jersey Institute of Technology / Upsala College (now closed after financial problems in 1980s)

Annenberg/CPB Project funded Virtual Classroom at NJIT.

Annenberg Foundation is private foundation established in 1989, supports non-profit organisations

Wegerif & Mercer 1997

Rupert Wegerif - Dialogical approaches to teaching and learning with ICT

Neil Mercer - BSc (Hons) in Psychology, University of Manchester
PhD in Psycholinguistics, University of Leicester
Chartered Psychologist

Language & Education provides a forum for the discussion of recent topics and issues in the language disciplines which have an immediate bearing upon thought and practice in education. Articles draw from their subject matter important and well-communicated implications for one or more of the following: curriculum, pedagogy or evaluation in education.

The task of the Journal is to encourage language specialists and language in education researchers to organise and present their material in such a way as to highlight its educational implications, thereby influencing educational theorists and practitioners and therefore educational outcomes for individual children.

Open University

Roschelle 1992

Jeremy Roschelle:

• Ph.D., Education/Cognitive Science, University of California, Berkeley, 1991
• M.A., Education/Cognitive Science, University of California, Berkeley, 1989
• B.S., Computer Science, Massachusetts Institute of Technology, 1985

The Journal of the Learning Sciences provides a multidisciplinary forum for the presentation of research on learning and education. The journal seeks to foster new ways of thinking about learning that will allow our understanding of cognition and social cognition to have impact in education. It publishes research articles that advance our understanding of learning in real-world situations and of promoting learning in such venues, including articles that report on the roles of technology can play in promoting deep and lasting learning. The Journal of the Learning Sciences promotes engaging and thoughtful participation in learning activities, and articles reporting on new methodologies that enable rigorous investigation of learning in real-world situations.

2009 Impact Factor: 1.767
Ranking: 9/44 in Social Science, Psychology, Educational
Ranking: 15/139 in Social Science, Education & Educational Research

The Institute for Research on Learning (IRL) was a research group in Palo Alto, California founded by George Pake in 1986 through a grant from the Xerox Foundation, It was a non profit research organization that looked at learning in a wide variety of settings, including schools, workplaces, and informal settings, using collaborative, multidisciplinary terms. Research questions were based in real-world problems and settings defined in collaboration with the institutions who hired IRL. IRL had a significant impact on education and knowledge management (among many other fields) not only in the US but globally because of the development of the community of practice idea. Aims to construct an integrated approach to collaboration and conceptual change

Activity 4.2: Reading the Roschelle (1992) paper

Roschelle, J. (1992) 'Learning by collaborating: convergent conceptual change', Journal of the Learning Sciences, vol. 2, no. 3, pp. 235-76

We suggest that you divide up your reading of the paper in the following way:

1. Read pp. 235-41 (up to the heading 'The Case Study').

Both 'collaborative learning' and 'conceptual change' are areas of research. As you might guess, 'collaborative learning' is about learning together but it is used by different authors in a variety of ways. (You may remember Hiltz and Meinke (1989) provided their own definition.) 'Conceptual change' refers to changes in people's understanding of concepts. There is a great deal of research into conceptual change in science learning. Roschelle gives his own definition of conceptual change on p. 237.

You might also want to note that 'CA' generally refers to Conversation Analysis. However, 'CA' is often referred to in the paper in the context of 'conversational actions' so that overall we cannot be clear as to Roschelle's exact meaning.

Theory - relational, situated view of learning

As you read consider the following questions:

1. • What argument is being made?

That students working collaboratively need to converge to develop a shared meaning and this process is characterised by four features:

Production of a deep features situation
Interplay of physical metaphors
Interactive cycles of conversational turn-taking
Progressively higher standards of evidence for convergence

1. • How does it relate to the kind of research being reported?

Students were working in pairs on a computer simulation to refine their concept of acceleration. The pair of students analysed were considered by the author to have achieved convergent conceptual change.

1. • How does the research question interact with the research work being undertaken?

Research question: How can the students converge on a deep new conception with only figurative, ambiguous and imprecise language and physical interactions at their disposal?

The study analysed the interactions between the students and between the students and the computer simulation.

1. • What kinds of evidence are relevant to the research question?

Conversation
Body language
Tasks performed on the computer
Individual post-study interviews to determine whether the two students share the same conception.

2. Read pp. 241-63 (up to the heading 'Evaluation of the Claims').

Read this section through fully before considering the following questions:

2. • How are the results/data presented? Compare with how Wegerif and Mercer presented their results in Reading 2.

Extracts from conversations shown in each paper however Roschelle emphasises the context and interactions with the computer and the student's body language and gestures which fits well with the research argument concerning the interactions.

2. • What does this say about the kinds of material that count as 'evidence' for the claims being made?

Conversation is not enough to analyse group interactions. He has added in body language, gesture and also physical metaphors such as Carol's demonstration on her hands.

2. • What does the author consider relevant about the context or setting for the study?

Private, urban high school
past knowledge: algebra
intellectual ability: average students but struggling with science
Relationship: close friends who had worked together on previous occasions
Participation: voluntary
Timing: 2X one hour sessions after school
Analysis timing: after 12 problems where they had not yet developed an explanation that corresponded to a scientific understanding

3. Read the remainder of the paper.
   • What is the nature of the claims made in the evaluation?

A shared conceptual change occurred which was compatible with a scientific interpretation of velocity and acceleration.

Gradual convergence towards a shared  understanding.

3. • How is the previously reported data used to support these claims?

Data from results section used to illustrate each part of the claims

3. • In what ways does the author relate the reported results to the wider literature?

Restates the basis of the problem with reference to literature
Majority of literature uses basis of contructivism but states that few theories account for convergent constructions in face of tendencies to diverge.
Tendency to diverge is strong in science (not backed by reference)

Discussion on how computer simulation can be truly constructive and situated or misused by teacher. How relational theory can assist this.

Differences from Vygotsky and Piaget and how this work supplements it

3. • Are the theoretical recommendations justified by the reported research?

'A case study cannot prove or disprove a theory, but it can clarify the meaning and import of a set of ideas'

'Mainstream conceptual change research on science learning should focus attention on convergence'

I believe that this is justified by the evidence produced here from one case study. The case study shows some good indications on how the two students collaborate to produce shared knowledge and it does need further work to see if this case is reproduced for other successful cases and also the difference with unsuccessful cases.

http://learn.open.ac.uk/file.php/6758/Readings/ebook_h809_reading5_oliver_etal_e1i1_l3.pdf [Accessed 20th February 2011]

Read the Oliver et al. chapter, but in particular concentrate on the section headed 'Methodology' (pp. 30-7). Consider the following questions:

1. What do each of the various approaches listed highlight?

Action Research Approach - 'technical' - instrumentalist view i.e. use pre-formed models.

'practical' - models provide guidance but practitioner must develop own practice - wider application

'emancipatory' - critical theory - educators recognise constraints- some relation to socio-cultural perspective - establish more egalitarian practices - wider application

Behaviourist - operant conditioning can guide instructional materials and can compare cohorts over time

Activity theoretic perspective - useful to analyse situations to identify problems/conflicts. Used to analyse change.

Socio-cultural perspective - investigating power manifestations

2. How, if at all, are specific methods (interviews, surveys, focus groups, observation, etc.) and methodological approaches related?

The historical, cultural and political context influences the researcher's interests and thus their choice of topics to study and their methodological approach. Specific methods have been developed on the basis of methodological approaches and affect the types of conclusions that they are willing to draw.

For example, in an interview or questionnaire, the types of questions asked may reflect the culture/interests of the moderator or a difference in culture may cause the respondent to have a different understanding of the questions from the moderator.

The Hiltz & Meinke paper was written in 1989 which correlates with the end of phase two, the stand alone systems, described by Conole et al. (2007). They suggest that stage two shows 'increased activity in terms of multimedia functionality but that it is still content driven and focused on the interactive tutorial paradigm'. Hiltz & Meinke describe methods of teaching that are mainly behaviourist and mainly transmissive with e-lectures although there are some indications of self-determination and presentations are assessed. Other parts of the paper suggest a more social and participatory approach. This mixture of methods would link in with the end of stage two and the start of stage three which is characterised by the shift away from the individual and a move to more situated learning as described by Conole et al.

The Wegerif & Mercer paper was written in 1997 and describes research published in 1996. According to Conole et al. this time period fits towards the middle of phase three, networking technologies, where we are 'beginning to see more emphasis on the wider contextual issues (skills, strategy, importance of embedding and integration). Also a shift away from the emphasis on the individual to the concept of situated learning'. There is also evidence of 'linking of development to strategy' as the intervention described by Wegerif & Mercer was planned to coach young children in exploratory talk with the aim of improving group problem solving. According to Conole et al., this is more indicative of the growing awareness of collaborative activities which comes into evidence in phase four (after 2000) and so may suggest that the timeline needs adjusting or that this was school utilising innovative techniques.

In our tutor group timeline we have the introduction of social technologies such as the forum, podcasting and the wiki, in 1994 with blogging coming later in 1998. This seems to suggest that, although the technologies may not have been in widespread use in education, they were available in the market place and there would have been a growing awareness of collaborative work. I believe that this would have been likely to effect the attitudes of more innovative teachers at the time and encourage the form of forward thinking that suggested this intervention.

In the Wegerif & Mercer research, qualitative analysis suggested key words which were analysed by a concordance program which illustrates another aspect of phase three where higher education lecturers are using ICT as a 'tool of the trade'.

1. In the discussion of task A11 (pp. 279-81) the account of the students' utterances is plausible, but why is transcript data to be preferred to the video data for such a visual task?

It is much easier to analyse and report transcribed data. Video data is time consuming to analyse as it must be observed and re-observed in order to analyse in detail and it is easier to do this from a transcript. Ease of use means that more raw material can be examined.

Some forms of analysis can be done direct from the recording when the researcher needs to focus on what is going on and not get too focused on the detail of what people have said.

2. A criticism sometimes made of quantitative research is that it uses preconceived categories rather than letting findings 'emerge' from the data. The 'Commentary' on task A11 (pp. 280-1) is qualitative rather than quantitative, but it could be argued that it also uses preconceived categories.

For example, Elaine's words before the intervention, 'No, because it will come along like that', and the fact that the next utterance is by John on the next question are interpreted as, 'She gives a reason to support her view and this is not challenged.'

Her words after the intervention, 'Now we're talking about this bit so it can't be number 2 it's that one. It's that one it's that one' are interpreted as, 'In proposing number 4 Elaine is building on these two earlier failed solutions' (p. 281).

Wegerif and Mercer have prior expectations about 'exploratory talk', defined as 'talk in which reasons are given for assertions and reasoned challenges made and accepted within a co-operative framework orientated towards agreement' (p. 277).

So notions such as 'reason', 'support', 'challenge' and 'failed solution' have specific, preconceived meanings. Do you think it would be possible to avoid the use of preconceived categories when analysing this data?

Glaser & Strauss (1967) published The Discovery of Grounded Theory which suggests that researchers should ignore the literature and theory and just work on the raw data to produce categories that are not contaminated by preconceptions. This originates from philosophers such as Bacon and Locke but, in that first book they do not account for the idea that 'there can be no sensations unimpregnated by expectations' (Lakatos, 1982, p.15). Later on, they separately discuss the preconceived knowledge and research that researchers have at their disposal before data collection and analysis.

Charles Sanders Peirce discusses the 'heuristic framework' of concepts that inform a researcher and suggests that abductive interference combines the new data with previous knowledge so that pre-conceptions often have to be abandoned or modified.

Following this the preconceptions must be regarded as heuristic concepts which form lenses by which the empirical world can be viewed.

http://www.socresonline.org.uk/2/2/1.html#s4 [Accessed 13^th Feb 2011]

I think that it would be difficult to avoid preconceptions when conducting the research but the use of initial coding without concern about categorising the codes would help. Once any relationships had been explored using diagrams, then focused coding can be used to reduce the number of codes and identify repeating ideas and themes.

inductivism - theories have to be based on empirical observations which are generalised into statements that can be regarded as true or probably true. (from Hume)

positivism - the only authentic knowledge is that which is based on sense, experience and positive verification (from Compte)

3. Again in relation to task A11, what evidence might support the following claim on p. 281?

'In the context of John's vocal objections to previous assertions made by his two partners his silence at this point implies a tacit agreement with their decision.'

Once again it is difficult to discuss this without resort to the original paper but it may be possible to back up this claim with evidence from the video on his facial expression and body language. For example, if he is still engaged with the group activity, this would back up this assumption whereas if he was sitting back or looking at posters on the wall then it would suggest he was bored of the conversation and the authors' assumption would probably be incorrect.

4. On p. 281, the authors claim:

'It was generally found to be the case that the problems which had not been solved in the pre-intervention task and were then solved in the post-intervention task, leading to the marked increase in group scores, were solved as a result of group interaction strategies associated with exploratory talk and coached in the intervention programme.'

When you read this claim, did you ask yourself if the researchers had looked at whether this was also true of the control group? If time allows, feel free to look at the papers in which fuller accounts of the study appear.

I was concerned that the control group may have a different, less comfortable relationship with the teacher/researcher and still be inhibited in the discussion for the second test.

5. In the post-intervention talk around problem A11, John says, 'No, it's out, that goes out look'.

This utterance doesn't use the words 'cos', 'because', 'if', 'so' or a question word, but it is plausible that John is giving a reason. How might one deal with such a problem?

A set of rules that identified reasoning behaviours in the video as well as in words

6. Are you convinced that the study effectively demonstrates the authors' case that:

'the incorporation of computer-based methods into the study of talk offers a way of combining the strengths of quantitative and qualitative methods of discourse analysis while overcoming some of their main weaknesses'?

No, intuitively I agree with the authors but I cannot determine it from the evidence that they present.

7. What does the computer add to the analysis?

The growing literature on computer assisted qualitative data analysis software (CAQDAS) expresses both hopes and fears. The hopes are that CAQDAS will: help automate and thus speed up and liven up the coding process; provide a more complex way of looking at the relationships in the data; provide a formal structure for writing and storing memos to develop the analysis; and, aid more conceptual and theoretical thinking about the data. In spite of these pros there are a good many criticisms and worries about the software in the literature: that it will distance people from their data; that it will lead to qualitative data being analysed quantitatively; that it will lead to increasing homogeneity in methods of data analysis; and that it might be a monster and hi-jack the analysis.

http://www.socresonline.org.uk/3/3/4.html [Accessed 13th Feb 2011]

8. What is the status of computer-based text analysis 10 years on? Spend 20 minutes trying to answer this question by searching the web.

I had a look at ATLAS.ti which offers a variety of tools for accomplishing all the tasks associated with a  systematic approach to unstructured data, i.e. data that cannot be meaningfully analyzed by formal, statistical approaches. It is a tightly integrated suite of tools that support analysis of written texts, audio, video, and graphic data. ATLAS.ti brings to the job highly sophisticated tools to manage, extract, compare, explore, and reassemble meaningful segments of large amounts of data in flexible and creative, yet systematic ways.

Create quotations directly from audio and video files, work with or without transcriptions. Link audio to text and videos to photos. Treat videos clips as you would text files, draw connections between any kind of data and content.

• create quotations in any type of audio / video / image file just like in textual documents
• annotate multimedia quotations
• hyperlink between multimedia quotations and text files (and vice versa, of course...)
• assign multimedia files like textual files as standalone documents

http://www.atlasti.com/

9. How does this paper compare with Reading 1?

My current thinking is that there seems to be two types of qualitative research: one which draws on theoretical concepts and draws up firm hypotheses which can be proved or disproved during the investigation; and one which uses theoretical concepts in order to produce vague conjectures about possible relationships and then examines these by investigating the raw material.

The Hiltz & Meinke paper uses two explicit research questions with firm hypothesise for each. One it investigates with a null hypothesis and quantitative analysis. The  reason for a null hypothesis is to suit the statistical tests we use to analyse the results

HI: There  will be no significant  differences in  scores  measuring mastery  of  material taught  in  the  virtual  and  the  traditional classrooms.

The other uses mainly one tailed correlational hypotheses in the form of predicting a significant positive correlation.

H2: VC students will perceive  the VC to be superior  to the TC on a number  of dimensions:
2.1  Convenient  access to  educational  experiences;
2.2 Improved  access  to  their  professor;
2.3 Increased  participation  in a course

I feel very uncomfortable with correlational hypotheses as it appears to me that they are likely to influence the researcher when collecting and analysing data. Think this comes from my science background!

The Wegerif & Mercer paper reports on research that seems to have used the second type of analysis with vague conjectures that are confirmed with conversation analysis. However, only a review of the research is presented in this paper and it is hard to judge whether firm hypotheses were used. Unfortunately the OU library only has the journal from 1997 and the full research was presented in an article from 1996! Tried other Uni libraries and cannot get it at all.

Hiltz, S.R. and Meinke, R. (1989) 'Teaching sociology in a virtual classroom', Teaching Sociology, vol. 17, no. 4, pp. 431-46.

Questions: What research questions are being addressed?
To see if CMC systems can improve access and effectiveness of post secondary delivery: is it a viable option? i.e. are outcomes as good?; what variables are assoc. with esp. good or poor outcomes?

Setting: What is the sector and setting?
US College - in the US the terms 'college' and 'university' are loosely interchangeable

Concepts: What theories, concepts and key terms are being used?

"Learning is the structuring of a situation in ways that help students change, through learning, in intentional (and sometimes unintentional) ways" (Johnson & Johnson, quoted on p. 432)
Suggests a behaviourist view of learning in that the students' behaviours are modified in the desired direction. (Operant conditioning, Skinner)

The VC still employs a transmissive format with e-lectures although some self-determination in order of activities is indicated, some active dialogue is employed and presentations are assessed.
Other parts of the paper suggest a more social and participatory form of education following Lave & Wenger's theories.

Methods: What methods of data collection and analysis are used? (e.g. the number of participants; the type of technologies; the use of interviews, surveys, observation, etc.)

• Matched but 'non-equivalent' sections of the same course; same teacher; same text; same printed materials; same exams;
• students self-select VC and TC
• Some mixed mode courses - 25-75% VC (107 & 96 students)
• Pre and post-course questionnaires
• Grades and SAT scores
• Records of time spent online
• Observation
• Interviews
• Case reports
• VC students still took exams in TC manner
• Hypothesis testing

Findings: What did this research find out?

• VC can increase access and effectiveness of college-level education
• Similar levels of skill in both groups
• No sig. diff. in final or mid-term scores
• Some diff. in general scores for comp. sci. Students
• Students thought VC was convenient
• Students thought access to professor was improved in VC
• There was improved participation in VC
• No sig. diff. in increased level of interest
• Communication variable; highest in mixed-mode courses

Limitations: What are the limitations of the methods used?

• Self-selection - more computer literate may choose VC; those with time constraints, perhaps too busy for course, may choose VC;
• Researcher delivering one course may affect outcomes
• Two very different unis so introducing another variable - big difference in results
• Everything in written form so disadvantaging those with SpLD
• Variations in cognitive maturity
• VC students formed F2F buddy groups thus changing format of research group
• Students did not have computers at home or in dormitories so had to go out to access them - those with computers at home most likely to appreciate convenience
• 50 hypotheses and over 200 variables - too many variables so difficult to determine any correlation e.g. low level student participating, high level student participating, low level student participating, high level student participating - what effect does this have on their opinions of communication?
• Questionnaire is in complex English for lower level students - strongly agreeing with negative.
• Compulsion to sign in twice a week reduces flexibility (p.440)

Ethics: Are there any ethical issues associated with the research?

• Use of pen names could encourage disclosures/comments that were unwise in a class situation and, as course progressed, identities may be inadvertently revealed.
• VC students may have discussed quiz answers with buddies in computer lab

Implications: What are the implications (if any) for practice, policy or further research?

'Results were superior for well-motivated and well-prepared students who had adequate access to necessary equipment and who took advantage of the opportunities..'

Motivation and access seem to be the main issues with the tutor having a crucial role in motivation by building a collaborative group.

'VC delivery mode seems better for courses that treat a few topics in depth rather than for......courses that attempt to cover a large number of topics in a short time'

I would agree with this as many people on H800 reported that they wanted time to do deeper research on some areas.

Meinke thought VC most suitable for those with advanced reading and writing skills but agreed that it was stimulating for all students as a component of mixed courses.

Activity 1.5: Reflecting on the paper (2 hours)

1. What counts as evidence in this work?

Quantitative research: questionnaire results, grade scores, exam results, records of time spent online

Qualitative research: interviews, observation, case reports

2. How do the two explicit research questions relate to the design of the research?

1)  Is the Virtual Classroom a viable option for educational delivery?  (On the whole, are outcomes at least as good as those for traditional face-to-face courses?)

2) What variables are associated with especially good and especially poor outcomes in this new teaching and learning environment?

The first question is addressed by using a null hypothesis and quantitative analysis of the course scores and results. It is a fairly straightforward yes/no answer.

The second is addressed by qualitative methods. I believe that this is the best way of investigating the introduction of new methodologies into such a complex social situation of a university setting. There is no straightforward yes/no answer but a mixture of student and staff perceptions and opinions. I am concerned about the prejudice shown in the hypothesis and in the selection of sub-hypotheses shown. Would this sort of prejudice influence interviews, observations and case reports? Investigating such a complex area needs to be carefully controlled but the authors suggest that they are testing 50 hypotheses and over 200 variables which would interact. An attempt has been made to make the findings more quantitative by using a questionnaire.

3. In what ways is the wider literature used in the paper?

To reference the full evaluation of results (Hiltz 1986,1988) p.432

To provide a definition for education (Johnson & Johnson, 1975) p.432

To provide reference for:

• previous work on CMC (p.432);
• software descriptions (p.433);
• collaborative learning (p.433);
• cognitive maturity (p.434);

To reference quote (Harasism p.434)

4. What views of education and learning underpin the research?

Technology as an add-on rather than an integral part of learning design.

"Learning is the structuring of a situation in ways that help students change, through learning, in intentional (and sometimes unintentional) ways" (Johnson & Johnson, quoted on p. 432)
Suggests a behaviourist view of learning in that the students' behaviours are modified in the desired direction. (Operant conditioning, Skinner)

The VC still employs a transmissive format with e-lectures although some self-determination in order of activities is indicated, some active dialogue is employed and presentations are assessed.
Other parts of the paper suggest a more social and participatory form of education following Lave & Wenger's theories.