Activity 11.6: Reading the papers (5 hours)
Effects of Four Computer-Mediated Communications Channels on Trust Development. Bos et al. (2002)
Questions: What research questions are being addressed?
Is trust development in inhibited in video and audio when compared to face-to-face settings?
Setting: What is the sector and setting? (e.g. school, higher education, training, informal learning)
- University students - did not know each other outside games
- F2F, audio, video, text chat
- Single and Mixed gender groups
- Social dilemma game (Daytrader)
Concepts: What theories, concepts and key terms are being used?
- Trust - 'a willingness to be vulnerable based on positive expectations about the actions of others'
- Delayed trust
- Fragile trust
- Bordia (1997) - text-based interaction less effective for tasks that have a high social-emotive content.
- Audio-conferencing - encourages domination by high-status group members (France et al., 2001)
- Text-based - fosters equal participation (Kiesler et al., 1984)
Methods: What methods of data collection and analysis are used? (e.g. the number of participants; the type of technologies; the use of interviews, surveys, observation, etc.)
- 66 3-person groups - 105 male, 93 female - av. Age 23
- 9 all-male, 7 all-female, 36 mixed gender groups
- Distributed more or less equally
- Pre-questionnaire on general trustingness
- Post-questionnaire on game behaviours and attitudes to other players
- Co-operation measured by total payoff
Findings: What did this research find out?
- Communication condition had significant effect on investment
- Greatest distance between chat and the other three
- No significant difference between other three at end of test
- All three mediated conditions delayed trust
- Partial agreement used in mediated conditions (not signif)
- Mediated conditions are more vulnerable to defections and more fragile
- Audio and video almost identical
Limitations: What are the limitations of the methods used?
- Not allowed to exchange social information
- Assumption: trust determines cooperation
- Value judgements - 'the other players could be trusted' used in second assessment when students knew each other
- Uncertainty of to which trust situations the results apply
- All participants of same age
- All computer literate and used to technologies
Ethics: Are there any ethical issues associated with the research?
- Paid to participate, according to how well they played
Implications: What are the implications (if any) for practice, policy or further research?
Some indications that managers can make determinations on which equipment to use depending on degree of trust required in a particular situation.
Classify the studies using Tables 11.1 and 11.2.
The study is Semi-interventionist, A. Asking questions as it uses pre- and post-questionnaires. There is also an aspect of Non-interventionist, C. as records of the amount of money won in each round is used to determine co-operation.
It is difficult to classify the study according to table 11.2 as it does not mention how the questionnaires are distributed. It includes both old (f2f groups) and new (audio/video/ text) learning technologies and we have no information as to whether it was assessed by either old (pen/paper) or new (emailed/web) research methods. I would suspect old research methods so this would give sections 1 and 2.
We also want you to note any difficulties you have with this task:
- Are there words or concepts you don't understand?
Not done any research on trust so no knowledge of previous or other relevant work.
- Are there statistical terms or methods that are new to you?
I had to relook up Tukey's test - similar to t-test but more suitable for multiple comparisons
Finally, how convinced were you by the research?
Initially I thought that I was quite well convinced but I am wondering if this is because it conforms to my expectations. Some research on social dilemma games and how trust relates to cooperation revealed that I may have been correct to be sceptical:
'Some people are, by nature, more likely to trust others. In order to solve both the first-order dilemma (how to agree to organize collective action) and the second order dilemma (who's going to police the agreement), you need both kinds of people: the more trusting people are necessary in order to make an agreement, and the less trusting people are necessary in order to police the agreement'.
http://www.cooperationcommons.com/node/390
This suggests that the balance in the group may be as, or more, important to the outcome than the fact that all people are trusting each other enough to cooperate.
A comparison of student feedback obtained through paper-based and web-based surveys of faculty teaching. Ardalan (2007)
Questions: What research questions are being addressed?
Is there a statistically significance difference between student feedback obtained by web-based and paper-based surveys?
Sub questions: 1. The equality of the percentage of the total number of respondents in each semester out of the total enrolments was considered.
2. The equality of student ratings of faculty teaching for the eight quantitative questions.
3. Whether there was a statistically significant change between the two methods in student ratings for faculty who were rated above the college average in the paper-based method
4. Whether there was a statistically significant change between the two methods in student ratings for faculty who were rated below the college average when the paper-based method was used
5. The ratio of the number of students who provided a qualitative response to the number of students who completed the quantitative portion of the survey between the two methods
6. The number of students who gave a positive, mixed or negative response between the two methods
7. the length of comments made by students between the two methods
8. the ratio of the number of students who provided a constructive response to the number of students who provided a response between the two methods
9. the number of constructive comments in each student response between the two methods
10. the ratios of number of constructive comments and the number of constructive comments that were qualified by students between the two methods
Setting: What is the sector and setting? (e.g. school, higher education, training, informal learning)
- Higher education
Concepts: What theories, concepts and key terms are being used?
- Qualitative
- Quantitative
- Various theories on differences between pen and paper and web surveys
Methods: What methods of data collection and analysis are used? (e.g. the number of participants; the type of technologies; the use of interviews, surveys, observation, etc.)
- No incentives
- Email and class information
- Same survey for online and pen/paper
- Anonymity ensured
- 8 quantitative (Likert scale)
- Same semester; 2 consecutive years; paired as to course
- 46 pairs; u/g and graduate level
- No signif demographic changes
Findings: What did this research find out?
- A significantly larger number of students provided feedback when the paper-based method was used.
- No significant difference in ratings for faculty teaching
- Decrease in overall effectiveness rating for faculties with high ratings when using web survey
- Increase in overall effectiveness rating for faculties with low ratings when using web survey
- students who participate in the survey of faculty teaching on the Web are likely to provide the same amount of feedback as those who participate in the paper-based method.
- This result suggests no differences between the feedback students provide on their level of satisfaction with faculty teaching in the two methods
- a statistically significant increase in the feedback length for the web-based method
- although students may provide longer responses, the responses are not necessarily more constructive
- The results of Hypotheses 8 and 9 contradict the expectation stated in the literature that web-based feedback will be more thoughtful
- Students provided more qualified constructive comments in the web-based method than in the paper-based method.
Limitations: What are the limitations of the methods used?
Ethics: Are there any ethical issues associated with the research?
- Equal access? Some problems with accessibility for VI students but this can also be a problem with pen/paper
- Permission obtained from Dean and VP academic affairs in addition to normal procedure
- Data removed that would identify department/course/professor
Implications: What are the implications (if any) for practice, policy or further research?
- an environment without time pressure and in which the student rates a number of sections at the same time may be less conducive to the use of extremes in the rating
- Decrease in respondents may cause problems with small sample sizes and statistical analysis- incentives suggested
Classify the studies using Tables 11.1 and 11.2.
Semi-interventionist - A. Asking questions. Survey.
An old learning technology (F2F course) with a mixture of old and new research methods. Sections 1,3.
We also want you to note any difficulties you have with this task:
- Are there words or concepts you don't understand? No
- Are there statistical terms or methods that are new to you? No
Finally, how convinced were you by the research?
Interesting results. I was concerned about the definitions for constructive responses and qualified constructive responses and so I was not convinced by the conclusions that came from those hypotheses.