in this exercise I shall analyse two
obstacles for effective implementation of learning analytics (LA) that had been
identified in the previous week; existing organizational culture (OC) and
quality of analytical data (QAD), using the ROMA framework as suggested.
The ROMA framework:
1. Define a clear set of overarching policy objectives
a. OC: it is clear that changing the existing culture in
any context may be extremely difficult and may take a long time. in this
regard, the policy I’d like to adapt is to create a need and urgency within the
organization in order to facilitate a much easier and expedited adoption of LA.
b. QAD: this is somewhat associated with the policy
objectives above, but not exactly. increasing the quality of data can only be realized
through adoption of higher quality research projects. this is expected to be a
long and painstaking process. on the other hand, it is necessary because;
i. it is the only real way to map the context accurately,
and
ii. higher quality data may make the paradigm shift in OC
easier.
2. Map the context
a. OC:
i. clear definitions of what we have (for our purposes,
as LA)
ii. clear definitions of what we may have, or what we want
to have
iii. why should we have them, and why is it urgent?
b. QAD:
i. clear definition of our open research questions
ii. analysis of the quality of analytical data currently
available
iii. analysis of the disparity between what is available and
what is required.
3. Identify the key stakeholders
a. OC:
i. Managers, directors, policy makers
ii. Teachers, teachers’ associations
iii. students, students’ associations and advocates
iv. IT and software producers
b. QAD:
i. Managers, directors, policy makers
ii. Teachers, teachers’ associations
iii. Research organizations (Universities, NGOs), researchers
iv. funding organizations
v. academic societies, publishers, meeting organizers
etc.
4. Identify learning analytics purposes
a. OC:
i. develop metrics to be used for quality improvement
steps (in education)
ii. develop a better understanding of our educational
ecosystem
iii. gathering evidence on our advantages and
disadvantages, rights and wrongs
iv. gaining competitive advantage (everybody else has it!)
b. QAD:
i. gathering, filtering and sorting data for higher
quality research
5. Develop a strategy
a. OC: Steps:
i. clear delineation of the need and its urgency
ii. identifying priorities in LA, what do we need data on?
iii. developing the methodology
iv. addressing and convincing the practitioners at the
field (i.e., teachers) on the need and urgency
v. developing human resources for LA
b. QAD: Steps:
i. clear delineation of the need and its urgency
ii. identifying priorities in research, what do we need answers
for?
iii. deciding on the methodology
iv. addressing and convincing the practitioners at the
field (i.e., teachers) on the need and urgency
v. establishing resources for sustainable funding
vi. developing human resources for research
vii. dissemination of research results
6. Analyze capacity; develop human resources
a. OC:
i. analysis of whether our organization
1. wants to change (even at times of evident urgency)
2. can be changed (internal dynamics)
ii. analysis of human resources:
1. are the policy makers convinced?
2. do we have the buy off of the practitioners in the
field?
3. do we have an effective IT system and staff?
b. QAD:
i. analysis of whether:
1. we have open research questions
2. we want to be involved in research (it is expensive
and time and effort consuming)
3. we have an existing research culture
ii. analysis of resources:
1. are the policy makers convinced?
2. do we have personnel experienced in planning and
conducting research?
3. do we have the buy off of the practitioners in the
field?
4. do we have an effective IT system and staff?
7.
Develop a
monitoring and learning system (evaluation)
a.
for both (OC and QAD):
i. what
are the outcome measures
ii. the
quality of data and analysis, do they warrant policies built upon them?
week 24, activity 19: the ROMA framework
Reflecting on the ROMA framework
in this exercise I shall analyse two obstacles for effective implementation of learning analytics (LA) that had been identified in the previous week; existing organizational culture (OC) and quality of analytical data (QAD), using the ROMA framework as suggested.
The ROMA framework:
1. Define a clear set of overarching policy objectives
a. OC: it is clear that changing the existing culture in any context may be extremely difficult and may take a long time. in this regard, the policy I’d like to adapt is to create a need and urgency within the organization in order to facilitate a much easier and expedited adoption of LA.
b. QAD: this is somewhat associated with the policy objectives above, but not exactly. increasing the quality of data can only be realized through adoption of higher quality research projects. this is expected to be a long and painstaking process. on the other hand, it is necessary because;
i. it is the only real way to map the context accurately, and
ii. higher quality data may make the paradigm shift in OC easier.
2. Map the context
a. OC:
i. clear definitions of what we have (for our purposes, as LA)
ii. clear definitions of what we may have, or what we want to have
iii. why should we have them, and why is it urgent?
b. QAD:
i. clear definition of our open research questions
ii. analysis of the quality of analytical data currently available
iii. analysis of the disparity between what is available and what is required.
3. Identify the key stakeholders
a. OC:
i. Managers, directors, policy makers
ii. Teachers, teachers’ associations
iii. students, students’ associations and advocates
iv. IT and software producers
b. QAD:
i. Managers, directors, policy makers
ii. Teachers, teachers’ associations
iii. Research organizations (Universities, NGOs), researchers
iv. funding organizations
v. academic societies, publishers, meeting organizers etc.
4. Identify learning analytics purposes
a. OC:
i. develop metrics to be used for quality improvement steps (in education)
ii. develop a better understanding of our educational ecosystem
iii. gathering evidence on our advantages and disadvantages, rights and wrongs
iv. gaining competitive advantage (everybody else has it!)
b. QAD:
i. gathering, filtering and sorting data for higher quality research
5. Develop a strategy
a. OC: Steps:
i. clear delineation of the need and its urgency
ii. identifying priorities in LA, what do we need data on?
iii. developing the methodology
iv. addressing and convincing the practitioners at the field (i.e., teachers) on the need and urgency
v. developing human resources for LA
b. QAD: Steps:
i. clear delineation of the need and its urgency
ii. identifying priorities in research, what do we need answers for?
iii. deciding on the methodology
iv. addressing and convincing the practitioners at the field (i.e., teachers) on the need and urgency
v. establishing resources for sustainable funding
vi. developing human resources for research
vii. dissemination of research results
6. Analyze capacity; develop human resources
a. OC:
i. analysis of whether our organization
1. wants to change (even at times of evident urgency)
2. can be changed (internal dynamics)
ii. analysis of human resources:
1. are the policy makers convinced?
2. do we have the buy off of the practitioners in the field?
3. do we have an effective IT system and staff?
b. QAD:
i. analysis of whether:
1. we have open research questions
2. we want to be involved in research (it is expensive and time and effort consuming)
3. we have an existing research culture
ii. analysis of resources:
1. are the policy makers convinced?
2. do we have personnel experienced in planning and conducting research?
3. do we have the buy off of the practitioners in the field?
4. do we have an effective IT system and staff?
7. Develop a monitoring and learning system (evaluation)
a. for both (OC and QAD):
i. what are the outcome measures
ii. the quality of data and analysis, do they warrant policies built upon them?
iii. are we ready to (re)change?