Personal Blogs

image source: Jisc Learning analytics going live

Overcoming the barriers to learning analytics implementation

The following is my reaction to an H817 activity where we are asked to create a short report on implementing a policy of learning analytics to improve teaching and learning. The request was for help developing a plan for rolling out learning analytics, either across the institution or across one section of the institution. My response is an elucidation of what institutional strategy would be involved in implementing a change of this sort, which departments would be involved in and/or affected by and what changes might be required to help ensure success. The stated purpose was a basis for the claim “learning and teaching at this institution are supported by learning analytics”, which the university could place on its website for promotional purposes. 

The following is my response to your request, based on my know-how about implementing technological innovations of this kind; the accompanying institutional change policies that have been found to be necessary in these cases and current TEL research.

Overall approach or strategy
The first point to note is how institutional change, whether within one department or across the whole institution is still on that requires looking at the whole workings of that institution. This is because, as Scanlon et al. (2013, 28) argue, “TEL should be considered as… made up of a series of interconnected elements that cannot be changed in isolation…centred on a vision of educational change.” The management proposed that they will be able to claim on their website that ‘learning and teaching at this institution are supported by learning analytics’ on that basis is not a viable vision as it is not one of educational change. The same authors propose that “TEL innovation should be…design-based research” (p.28).  A more appropriate vision statement would reflect a research-driven approach that considered the full context in which the innovation takes place. Given that, the vision statement would recognise the growing evidence that learning analytics can be a basis for improved quality and sustainability in education, and the need to involve all stakeholders for its potential benefit to be realised.

Which barriers will need to be overcome?
Some of the biggest barriers to TEL innovation and more specifically to learning analytics were identified (for e.g.) by Macfadyen and Dawson (2012); Mosadeghrad et al (2012); Ferguson et al (2014).  Ferguson et al identifies one of the challenges as complexity:  the need to work with learners, educators, administrators and support staff in ways that suit “the practices of those groups, their understandings of how teaching and learning take place, the technologies they use and the specific environments within which they operate”. Ferguson et al. note how this requires “explicit and careful consideration during the process of implementation, in order to avoid failure and maximise the chances of success”. Mosadeghrad et al (2012) are in accord with this viewpoint. They cite poor management, deficient leadership, and lack of strategic planning (p. 193).

The cultural barriers identified in Macfadyen and Dawson (2012) can be categorised as the education’s resistance to innovation and change in and the tendency of educational organizations to add resources rather than strategically allocate them; its tendency to assume learner homogeneity, rather than to explore diversity.  Macfadyen and Dawson (2012) identify "strategic barriers, structural barriers, human resources barriers, contextual barriers, and procedural barriers" (p.193). These include resistance to change that impinges on faculty autonomy, especially if it is perceived to derive from the “cost-consciousness-and-efficiency” as well as unwillingness to accept the extra workload learning how to use complex new tools.

Not exclusive of any of the above cultural, operational and strategic, I also wish to list the psychological domain.  Freud’s theories has been applied to the workplace.  In particular, his theory of psychogenic disturbance of vision could be applied to sentiments like:  'It could negatively affect me'; 'Does it fit with how I do things now?'; 'Does it help me?'; 'It will make me look bad'; 'I get nothing out of this' which from personal experience I can attribute to reactions to forthcoming or proposed institutional changes.

References:
Ferguson, R., Clow, D., Macfadyen, L., Essa, A., Dawson, S. and Alexander, S., 2014, March. Setting learning analytics in context: Overcoming the barriers to large-scale adoption. In Proceedings of the Fourth International Conference on Learning Analytics And Knowledge (pp. 251-253). [Online]. Available at: https://dl.acm.org/doi/abs/10.1145/2567574.2567592 Accessed July 23 2020
Freud, Sigmund. (1910). The psychoanalytic view of psychogenic disturbance of vision. SE, 11: 209-218.
Macfadyen, L.P. and Dawson, S. (2012) ‘Numbers are not enough. Why e-learning analytics failed to inform an institutional strategic plan’, Educational Technology & Society, vol. 15, no. 3, pp. 149–63 [Online]. Available at https://www.open.ac.uk/ libraryservices/ resource/ article:106516&f=28635 (Accessed 19 July 2020).
Mosadeghrad, Ali & Ansarian, Maryam. (2014). Why do organisational change programmes fail?. International Journal of Strategic Change Management. 5. 189. 10.1504/IJSCM.2014.064460. [Online]. Available at https://www.researchgate.net/publication/275590169_Why_do_organisational_change_programmes_fail/citation/download (Accessed 19 July 2020).

image source: jisc.ac.uk

My last two blogs try to summarise my views and knowledge of ways Big data are used in learning and teaching (where it can be referred to as educational learning analytics) and ethical practices related to it. 

The following tries to highlights the relatedness but also the differences between learning analytics (LA) that is learner/teacher focussed and when it is more informs the wider body of educators such as managers, administrators, the institution, government, other funding bodies. Also, the reasons why LA emerged in the 2000s.

Reasons for the emergence

Campbell et al. (2007) and Norris et al. (2009) examine the need for analytics from the viewpoint of the USA, where at the time of writing, they suggest that levels of degree qualified workers were falling overall as well as the quality of learning.  In the UK in 2020 (Guardian, 2020) 'nearly three-quarters of the country's universities slipped down the rankings in the UK's worst-ever performance in the table compiled by data and research group QS.'  Studies had long since shown a steady decline in higher education standards in the UK since the 90s (Cameron, and Smart,1998). Analytics offers a way of 'responding to internal and external pressures for accountability in higher education (and) improved learning outcomes and student success' (Campbell et al., 2007).  I should be noted, however, that since the turn of the century, the UK has made big strides in Maths, Science, and Reading and currently stood in the top 20 in these areas in the OECD (2018) PISA standings, which it should be noted, only refers to school learning outcomes.

The US has also seen a sharp improvement in school results over the same period and it could in part be put down to the increase and improvement in learning analytics initiatives and theory.

Used to benefit educators

Institutions are of course very interested in student performance because it reflects on the institution and its popularity and on their funding and staff jobs. They are trying to reach certain external and internal key indicators. For those reasons, they are also interested in things that are not about the individual, rather, they are interested in increasing student numbers, administrative, and academic productivity and cost-cutting for profitability and data related to this, a focus is known as academic analytics. They are interested in general account analytics, which allows them to see what students, teachers are doing within the account.  Activity by date allows the admin to view student participation in Assignments, Modules, Discussions, and teachers' completion of Grades, Files, Collaborations, Announcements, Groups, Conferences, etc  In general one can view how the users are interacting with the courses in the term. This means the content can be adapted to improve efficiency, productivity, and adherence to policy and practice based on deductions and predictions made from the use of content.

Used to benefit students

I categorize all the above (under benefitting the educator) as also benefitting the learner. The above would reach the learner via tutorials the teacher has with them and reports if any.  However, the analytics that the student is likely to actually access mainly include only their grades and records of their assignments.  Students can use these to track and assess their progress, this way they can see where they need to improve as they go along. Teachers should use educational data mining (EDM) - 'analysis of logs of student-computer interaction' (Ferguson, 2012) to improve learning and teaching. Romero and Ventura (2007 cited in Ferguson, 2012), identified the goal of EDM as ‘turning learners into effective better learners’ by evaluating the learning process, preferably alongside and in collaboration with learners. These data are often only available if the teacher makes them viewable.

In my situation, the LMS used is compatible with various apps such as Turnitin (a plagiarism checker), students also have access to its analytics, used at the drafting phase of the writing course, and at the end of the course only if students query their report writing score and plagiarism has some relevance. 

Challenges

Some of the educational challenges in the environment that I work in include adapting to online as opposed to f2f teaching. A way to sum up the challenge is it is completely different because the contact and communication have technology running through it.  One of the challenges involved in implementing learning analytics is mistrust of how data will be used. Students I work with sometimes avoid leaving digital traces for fear of it ending up being a means of covertly 'assessing' them. This is why their focus is on final tests, where they are fully aware of what performance data will be collected and how it will be used. 'Therefore,  it is necessary to make the goals of the LA  initiative transparent,  clarifying exactly what is going to happen with the information and explicitly' (Leitner et al. 2019, p.5).  Researchers also point to privacy and ethical issues. 

Recommendations 

1. Teachers should get training in the use of data analytics for use in the classroom

2. There should be an open dialogue about  what learners' rights to their own learning analytics should be; what learning analytics should be available to them; how to give them access (including training in accessing their own data) 

3. Analytics is too much management centred - data-mining and academic analytics and its often not shared with teachers. Learning analytics need to be much more classroom and teacher/student relationship centred. 

I think the adoption of these recommendations could improve engagement, ownership, motivation to learn better, and also improve learning directly.

References

Cameron, K. and Smart, J., (1998) Maintaining effectiveness amid downsizing and decline in institutions of higher education. Research in Higher Education, 39(1), pp.65-86.Cameron, K. and Smart, J., 1998. Maintaining effectiveness amid downsizing and decline in institutions of higher education. Research in Higher Education, 39(1), pp.65-86.