On 7 November 2017, I facilitated a number of focus groups for STEM staff tutors to elicit views about tutorial observations. During this session, I asked the following open questions: (1) What is the most important reason to carry out an observation? (2) What procedure or procedures do you follow? (3) How do you record an observation? (4) What do you look for? (5) How do you share feedback? (6) Is there anything special about online observations? And (7) Should there be standardised guidelines for STEM?
This blog has taken quite a bit of time to prepare, since I’ve been involved in all kinds of other things, most notably, trying to get everything sorted for the October starts. It seems that November is a month when I can start to do other things.
In some respects, this post is a sister post to the one that I made about some tutor focus groups, that had the title: Tutorials and tutorial observations: what works and what helps tutors?
During this session, each group was asked to make notes about the discussions that took place, and to summarise some of the key themes in a plenary session. What follows is a summary of some of the themes that were discussed in each of the groups, followed by a brief summary of the plenary discussion.
Since a number of discussion points were common between different groups, I have chosen to highlight sections from each group that appear to be the most significant.
Group 1 began with the first question: why is it important to carry out observations? Observations can be linked to and connected with staff development; it can facilitate a two way process that is also linked to quality assurance. It also represents an opportunity to get to know associate lecturers. There are other reasons too, which is: staff tutors can get a feel for what is happening ‘on the ground’ and understand how distance learning materials are being used and interpreted by students and whether the tutorial strategy (which is sometimes designed by a staff tutor) is working as planned. Also, observations can help facilitate discussions during a tutor’s appraisal.
Group 2 gave a very notable answer to question 4, what to look for: the answer will depend on the level of study, i.e. whether they had recently started at the university, or were coming to the end of their studies. For introductory level (level 1) students, staff tutors would look for: encouragement, enthusiasm, positivity, bounce, involvement, thank students for attending, and whether they were motivating. For final equivalent (level 3) students, staff tutors would look for expertise and subject knowledge, competence, confidence, ability to respond to questions encouragingly and supportively, using the question well, understand where the students ‘are’ and the range of abilities, and how to address any gaps of understanding between where student is and where a student needs to be.
Regarding the question of procedure or procedures, group 3 mentioned that observations could relate to an associate lecturers probation period (which lasts for two years), but an observation need not necessarily take place in the first year. A related reflection is that the idea of a ‘one off visit’ to provide support or to ensure that teaching quality may be okay could now be outdated due to online tuition; it is now possible to look at a ‘bigger’ picture (and more points of interactivity).
This group gave some reflections about online tutorials, stating that a staff tutor or line manager watching the recording can also be considered as a form of online observation. It was also reflected that online observations does offer unique challenges, in that it is very difficult to observe the effectiveness of online group work. In terms of feedback to tutors, it shouldn’t be in the form of a formal report, but there could be verbal feedback which is then supplemented by written feedback.
Group 5 emphasised procedures. A memorable suggestion was: don’t visit the first tutorial for a new associate lecturer. Also, ask tutors which tutorial they would like their line manager to visit so they can showcase a session that they might have been particularly proud of, and also include a visit to the tutor’s tutor group forum to gain a complete picture of their online teaching. Gather observation feedback using a form and if the tutorial is good, send the form to the tutor and give the tutor a copy of the feedback form in advance, so they know what they staff tutor is going to be looking for. If there is a need for development, have a discussion with the tutor.
Group 6 referenced the use of peer observation, which could be used in situations where staff tutors might not have sufficient time to carry out their own observations. There were differences in terms of how tuition observations were recorded: 2 line managers used a proforma with space for qualitative feedback, 3 line managers write notes and then write a summary, and 2 line managers use of a loose proforma to provide semi-structured notes.
Following on from the discussion about recording observations, group 7 noted that the former Faculty of Science used a form. A form should also help to emphasise what went well (within a tutorial), what not so well, and what might be potentially improved. Like the previous group, peer observations was also referenced, in the sense that tutors could present to other tutors.
This final group raised many of the points were highlighted earlier, but placed particular emphasis on online tutorials. Some key points that line managers would look for included: whether or not tutors were prepared, whether they were clear vocally and had a relaxed attitude, whether they encouraged interaction from students and designed interactions. After an observation, members of this group would have an informal chat with a tutor which would be followed by an informal letter. The tone of this correspondence is important: suggestions rather than instructions for improvements would be offered.
Towards the end of the session, there was a facilitated discussion to draw out key discussion points from each of the groups. What follows is a brief summary of the main points, any commonalities between the groups and implications for practice.
An early comment was that observations are important not only in terms of quality, but they also help to develop the line manager’s relationship with the tutor. A useful perspective was that a line manager’s view of tuition of teaching should not begin or end with an observation. Instead, an observation should contribute to a holistic view of tuition practice. One participant made reference to the concept of a ‘learning walk’.
There were also messages that were common between the groups. In terms of practice, the importance of an informal conversation with the tutor after an observation was emphasised, followed by a letter or an email. There was also the view that there should not be a ‘ticklist’ or standard form that staff tutors should apply to complete observations. Instead, there should be guidelines rather than mandated procedures, to offer flexibility.
Regarding online tutorials, tutor managers should look for interactivity. Since observations can also be through recordings, it was also noted that the choice of the recordings could be directed by the tutor.
A further point acknowledged the challenge of online teaching. Online teaching using synchronous tools and live environments requires significant skill, knowledge and experience. Reflecting the TPACK model, tutors need to acquire and apply technical, pedagogical and content knowledge, and dynamically respond to the needs of students. Acknowledging these challenges, one tutor manager reported that it important to tell the tutors that it is okay not to present or deliver a session that is ‘technically perfect’, and what really matters is whether student learning is taking place. Put another way, the tutor line managers cannot only have a role in developing practice and teaching quality, they can also have a role in developing tutor and teacher confidence too.
One of the tasks that I have to do in my day job is to carry out tutorial visits. I’ve seen a variety of different sessions, ranging from very informal sessions, sessions that are tightly structured around a PowerPoint presentation, and sessions that showed the use of software tools that are taught within an Open University modules. There have also been these occasions when I’ve been left astonished and the skills and abilities of tutors to convey technical concepts in interesting and creative ways.
In my career as a tutor, I’ve also been observed during a tutorial. I think I’ve been observed three times; once during my first year, and another occasion when only a single tutor arrived at a tutorial. My line manager did some of the things that were mentioned by my peers: there was an informal chat, there were suggestions (and not instructions) about how my teaching might be improved, and I was sent a letter that summarised the observation (and also what happened during the tutorials that I facilitated). Importantly, this always felt like a positive process. I really felt that my line manager had taken the time to listen to and respect what I had done.
I’m sure it comes across in this post that I think that tuition observations are important. My own view is that they should be about developing and supporting tutors (and teachers) first, and about institutional and organisation management second. They should also be fun too.