In our elluminate session this evening, Bill expressed a lot of concern about being able to measure the educational benefit of certain tools (such as the blog). This bothered me in a way as I think it misses the bigger picture to an extent.
In the courses we run at work, the measurable effect we have on our use of technology is the fact that since introducing it the pass rates and retention rates of our distance learning students have massively increased. Also, we've gone from getting feedback highlighting a feeling of isolation to generally appreciating the fact that they can learn as part of a group in spite of being so spread out.
In all our courses, I can see how much an individual student is engaging with the activities, and there are tasks included to assess their progress through the course, so we can see how they're getting along. We can also see how well students are engaging with particular activities and tools in general.
It's not always possible to see evidence that each individual task has benefitted each individual student. However, my aim is to get them through the course as a whole rather than each individual activity in turn and I can track progress for that.
OK, if you wanted me to demonstrate the educational benefit of each individual tool and each individual activity, I would struggle, but I could do it for the course as a whole.
What I look for in the courses I design is that students are generally engaging with the activities and demonstrating progress through the course ready for the exams.
I have my ideas for the intended aims for each individual task and tool, but at the end of the day, if a student uses it in a different way, which they feel is of benefit to them and still clearly demonstrates progress through the course, I can suggest other ways they might like to look at things, but I'm not going to complain they're not doing things my way.
For a tool such as blogs, we use Moodle like the OU does for our VLE, so providing these doesn't cost us any extra, the students that use them seem to enjoy using them and find them useful, but they're not actively assessed on them. I couldn't tell you how educationally beneficial they were in comparison to other tools by themselves, but they're part of a course that demonstrably increases students knowledge and skills to analyse issues within the wine trade.
While the students I have that are in the trade or want to join the trade are at least in part taking the course to be able to demonstrate they have some knowledge and skills relating to the industry, the students that are just interested consumers with no aspirations to work in the industry are just looking for something that's fun that will help them learn a bit about wine. The course has to cater to both needs, it's got to be both educational and enjoyable. While you would never want to design a course that was boring, the fun bit becomes all the more important when students become more like paying customers.
In short, if a tool is clearly being used for a purpose that the students feel is useful and enjoy and they are clearly progressing through the course itself, how important is it that you can clearly measure the educational benefit of the individual tool/task for each individual?