Personal Blogs

Murphy, S. 'Literacy assessment and the politics of identities' [Course website]

Notes on implications and equity issues in individual assessment.

General problems

• Low scoring students results based on smaller sample of responses than higher scoring students
• No credit for partial knowledge
• Summaries masks differences between test scores i.e. student scores uniformly low or one test is very low and rest above average
• Knowledge of format of tests affects performance
• Limited generalisability from test to context e.g. comprehension test can be answered without reading the passage
• Limited relevance to non-school environment
• One narrow path to success - perceived as unfair by those with strengths in other areas.

Systematic bias

Testing takes place in context. It has behavioural rules on inter-personal communication and participation

• Interaction between examiner/examinee
• Anxiety
• Over-testing and speed of testing for African-American children
• Low achievers more anxious
• Negative about process that sorts them for race and class
• Freq. of specific word use differs across languages so students may not recognise words even if translated to their own language
• In translation the resultant tasks may not be equivalent

Teachers administering tests

• Successful result of the system
• Forced to participate in system even if do not agree
• Psychologists do not know child; teacher/parent does so ideas of teacher/parent often confirmed even though results are ambiguous
• Teachers focus on those pupils likely to show gains in test results
• Teachers reportedly alter results, teach to the test, use test items in class. Resistance but pressure to get pupils to achieve.

Psychologists administering tests

• Seem to be autonomous in selection and administration of tests
• Milofsky found psychologists in suburbs worked in environment and identified barriers; those in urban environment were too busy and identified individual differences
• Objectivity of test legitimises the work;
• Working in marginal position so power important to personal identity
• Legal requirements require audit trail

E801: Action 2.6: Inclusion and Globalisation

Johnson, D. & Kress, G. (2009) 'Globalisation, literacy and society: redesigning pedagogy and assessment' in in Soler, J., Fletcher-Campbell, F. & Reid, G. Understanding Difficulties in Literacy Development: Issues and Concepts, London, Sage.

The inclusion of digital literacies as part of the portfolio of literacies required by modern learners has been studied in many contexts - that of the different forms of digital literacy, the skills required for accessing information in a digital format, the skills required for extraction and evaluation of information etc.

A lot of institutions initially believed that a techno-based curriculum would cure all problems and level the playing field for people with disabilities. A lot of money was spent on various forms of technology and much of it was wasted.

Currently, in online and distance learning, the readjustment is towards  learner-centred activities with universal design for online learning materials BUT with an emphasis on user-controlled flexibility. Collaborative learning is on the increase with the use of peer networks via Twitter, forums and blogging.

Policies and institutions may take a while to catch up but individual teachers are leading the way with some fabulous activities for their students.

Assessment is also improving dramatically with online portfolios and peer feedback.

The following is from a previous blog post:

Universal Design for Learning (UDL) Guidelines - Version 1.0 (CAST, 2007)

UDL has three primary principles that provide the structure for these Guidelines:

Principle I: Provide Multiple Means of Representation (the "what" of learning). Students differ in the ways they perceive and comprehend the information presented to them.

I am currently working with a student who has a severe visual impairment. She lost most of her sight at the age of 16 years by which time she had already discovered that her preferred learning style was visual. She still has enough sight to revise by drawing out large diagrams but it is not easy for her.

Principle II: Provide Multiple Means of Expression (the "how" of learning). Students differ in the ways they are able to navigate a learning environment and express what they know.

I have experienced the following adjustments in the universities where I work: allowing speech impaired people to plan and design PowerPoint presentations using the inbuilt speech features; allowing a student with ME to verbally present the information rather than spend all evening writing a report on a field course;

Principle III: Provide Multiple Means of Engagement (the "why" of learning). Students differ markedly in the ways they can be engaged or motivated to learn.

One third year module at Keele University is Inspirational Landscapes in Geography. Assessment is 20% test and 80% project. Previous student projects have included:

• • Impact of the Malvern Hills on Elgar's music
  • Video diary of a walk in Wordsworth's footsteps
  • Photomontage of the experience of Dovedale
  • Influences of Indian landscape on fashion design
  • Johnny Depp: face, costume and landscape
  • Landscapes of Lord of The Rings
  • Thomas Hardy and the "Wessex" landscape
  • Landscape design for computer games

The module sounds fascinating and I know several students who really enjoyed it. http://www.esci.keele.ac.uk/people/pgk/geg-30014/handbook.html

Week 13: Activity 25.2

Make notes on the three important points which are noted in the Introduction:

• whether particular assessments or examinations are core to the course

I have been working with a student with no usable sight studying for a computing degree. One of the modules covers coding for graphics. The university are happy to make reasonable adjustments but these generally involve working with a support worker which reduces his independence. He is very frustrated and suggested that the department let him study an alternative module but they will not do this as it is a core module and they have no time to write a module from scratch for him.

• what adjustments are permissible within particular assessments or examinations without compromise to academic, or other prescribed, standards, such as competences required by professional bodies

I work with a blind student studying law and they have several exams coming up in May where they answer three questions in three hours. Last year the department organised her 100% extra time and rest breaks and she took the exams but was completely exhausted at the end of an 8 hour stretch and she had to postpone one exam and sit it in August as a first attempt. This week the department approached her and suggested that she answered one question in the exam and the other two as an assignment. In this way she will not be too exhausted to do her best work.

Another university has found problems on physiotherapy and medical courses as the professional body will not allow notetakers for dyslexic students. This one is a current dispute between the university and the NHS!

• whether the successful achievement of the highest grades and awards, based on performance in examinations and other assessments, is equally attainable by disabled students.

Another university has a student who uses notetakers on field trips for geology. After discussion, it has been decided, that the notetaker should take notes in the assessed notebook but to label each section with the name of the person speaking i.e. the student who is dictating notes or one of the lecturers. In this way the student is only marked on their work rather than that of the notetaker's ability to record the lecturer's words. This would seem to be a reasonable approach but actually it works to the student's detriment as the other students mingle their thoughts with that of the lecturer's comments and get marked on their good ideas which may actually have come from the lecturer.

Would you have emphasised the same three points?

I think that these points are important but I also think that the perception of the other students needs to be considered. In the case of the law student, her friends and colleagues think she is being assessed just as stringently as they are being assessed. In the case of the notetaker on field trips, one student I worked with went to great lengths to explain to the other course members the way her assessment worked and the fact that her DSA paid her notetaker on several occassions. She informed me that there had been very rude comments from other students about the fact that she was marked on a professional notetaker's work and that the others had to pay for it too!

Are there any positive or negative aspects of online assessment for disabled students?

Online assessment can work well but too many institutions consider online assessment to mean multiple choice tests with a time limit so that students do not have an opportunity to research the questions. This can disadvantage many disabled students. For example, in the last few weeks one student, who has Asperger syndrome and is severely dyslexic, has had mid-term tests in his first year at university. The two modules he has found most difficult had straight tests and he achieved first class honours in both. The two modules he has no problems with, used multiple choice tests and he has failed both. Researchers put the problem down to problems with working memory and poor eye coordination.

Creating Accessible Examinations and Assessments for Disabled Students

Evaluating Practice

• Staff are consciously aware of, and in agreement about, what aspects of student attainment or performance they are trying to assess.
  This seems to vary between the three universities where I am employed and also between departments. I know that the OU have course team meetings to try to ensure that this happens but I have personally been penalised for following guidelines issued by one tutor which my tutor did not agree with.
• Students are aware of the aspects of attainment or performance which are the subject of assessment.
  Not always
• The nature of marking criteria are kept under regular review: such matters as the importance of spelling, grammar, the ability to calculate, and the ability to remember dates and constants are collectively evaluated by the staff including part-time staff and teaching assistants.
  There is a lot of disagreement on this one. Some lecturers place great emphasis on grammar and spelling in examination conditions and penalise heavily. Lack of knowledge, and in some cases, disbelief concerning dyslexia leads to a response that is less than helpful. This summer I was assisting a student on a field course that involves spot tests in the field. I know all the students well and several were upset as they could not immediately recall the information. These were students with dyslexia and I spoke to the lecturer who was very concerned that he had disadvantaged those students. He had no idea that short term memory problems were an issue.
• Policies concerning electronic aids to spelling, grammar and calculation in examinations are kept under regular review.
  A real annoyance of mine! One university insists that they will not allow electronic spelling aids in examinations despite the student's access report stating that they need one. They supply dictionaries for students with dyslexia, which the majority cannot use!
• Where a student is unable by reason of an impairment to show evidence of relevant attainment or performance in the standard way, alternative arrangements are put in place if it is possible to do so.
  Yes, at all three universities
• The flexibility referred to above is available in terms of deadlines and timetabling of assessments.

Yes, but this varies between universities. Two universities require an extenuating circumstances form to be completed; the third states that extenuating circumstances are for unexpected occurrences and extensions required due to disability are just to be granted with consultation between student, tutor and lecturer.

• Alternative assessment arrangements as referred to above are well controlled to ensure consistency and fairness, vis-à- vis both the students taking them and other students.
  No, at one university everyone who is assessed with dyslexia gets 25% extra time; everyone who needs rest breaks is allowed 15 minutes despite disability or length of exam.
• Assessment feedback to students is accessible to all our students, both in terms of content and format.
  No, some departments put handwritten reports on a printed version of an assessment. This results in a student with reading difficulties or visual impairment having to ask a support worker or friend to read this private material to them.
• Those responsible for our examinations and assessment appeals are well versed in the ways in which procedures may need to be adjusted in acknowledgement of the needs of some disabled students.
  Definitely not!