Activity 11.1: Podcast 1 (30 minutes)
Survey One: http://learn.open.ac.uk/mod/resourcepage/view.php?id=409227
Listen to this discussion between Alan Woodley and Adam Joinson on the use of electronic surveys and their pros and cons.
In his book, The Salmon of Doubt, Douglas Adams (2003) wrote '... everybody lies to people with clipboards' (p. 93) and, when asked about the benefits of speaking to his fans via email, said 'It's quicker, easier and involves less licking' (p. 101).
Do you think he would be in favour of electronic surveys?
Notes
AlanWoodley interviews Alan Joinson (psychologist, Bath)
Benefits - speed
E-surveys producing different answers from mail surveys:
- Is sample equivalent to mail survey?
- Does it match sample you intended?
- Can people access it through fire walls?
- Media effect - images, videos, response formats
- Context - where are there? Cyber cafe? Mobile device? Is it one person?
Lower on social desirability issues on web - more willing to admit socially undesirable issues
Psychometric measures - higher in anxiety online
Is it a sample issue, media issue, or interaction between two?
Which answers are correct?
e.g. are people more honest on line?
Sometimes it does not matter; all variables are inflated in various ways
e.g. self-esteem vs academic performance. Constantly high figure for self esteem does not matter as standardised
Problem comes from comparing online and offline sample - norms are different.
Pencil and paper - rarely true volunteers - students in classrooms where lecturer stands there and waits for paper back
Online surveys are truer volunteers
Classroom surveys - chat with peers as they fill it in.
Differences in nature of responses - are people honest or not
10 years ago - more candid in online surveys but more sophisticated audience now but there still seems to be some difference. Potentially may shift other way. Online is confidential rather than anonymous. Employer may track internet use etc. Paper questionnaire is more likely to be anonymous if posted back, less so in class situation.
User-focused idea of research rather than questionnaire - can find difficult to answer when you do not match sample answers.
Shift into public sphere - raw data linked from publication
Response rates are dropping - privacy concerns, claim less time, in past considered contributing to public good,
Can increase transparency to help this perception
Sociology research - participatory research form 60s and 70s
Online surveys - complete with audience in mind, talk to researcher in open-ended questions - people are used to this in online process as it is communication tool.
Humanise the process to increase participation rates but not too much or reintroduce interviewer bias e.g. realistic avatar speaking question - less self-disclosure.
Comments
A couple of comments on the podcast first:
- I love the fact that the OU put pictures of the people talking on the download page for the podcast - it makes it much easier for me to relate to the speakers, however, it would really help if they were labelled so I knew who was who. I was actually sad enough to right click each picture and check for associated text to find out!
- I missed a couple of points as I was listening to the podcast and quickly noted the time so I could refer to the transcript. I was disappointed to find that I could not reference these time points on the transcript.
I found this really interesting as it raised some things that had not occurred to me before. Full notes are on my blog for 17th April but my main thoughts are here.
Naively I had not realised that there was a difference between online and pen/paper survey results and so it was a shock to me that results could not be compared. I could understand that people were more candid in online surveys but was also interested in the fact that self-disclosure has changed over the last ten years so that would make comparisons between online surveys from ten years ago and current ones also suspect.
I have always preferred online surveys as, working with students with disabilities, I have been asked to fill in the forms that are given out in class. Often these are filled in by comparing answers with friends or the student would say 'I think this course is crap but I can't say that as the lecturer knows my handwriting'. I first began to get some doubts concerning online surveys when I found that universities were putting pressure on students to complete the National Students Survey positively by telling them that if they wanted their degree to be worth anything then they should say that the university was really good as it would raise its profile.
I think that I would have to be a lot more critical about using online surveys now I have listened to this podcast as thinking about true sampling and the context of where the respondent is filling in their answers is making me consider some extra questions on respondent demographics, context and what device they are using so that this can be accounted for. Of course the drawback to this is that the respondent may then feel that the survey is less anonymous.
As for the question about Douglas Adams (one of my favourite authors!) - I think that he would have liked e-surveys in the way that they are heading with more transparency as to why they are being conducted; user-centred, open-ended questions where there is a conversation between the researcher and the respondent; and the shift into the public sphere where the researcher links to the raw data when publishing so others can confirm and re-interpret the data. I know that humanising the process will draw accusations of interviewer-bias but do we want an abstract, quantitative survey which, despite claims, is influenced by context but where this is not openly acknowledged? Or do we want in depth, qualitative answers that can supply detailed information and, by reflexivity, we can work hard to account for interviewer influence? The first appears much more scientific and generalisable; and is certainly easier to carry out but I would consider it much less reliable.