Edited by Christopher Douce, Wednesday, 28 June 2023, 10:28
One of my colleagues volunteers for the Open University audio recording project. The audio recording project takes course material produced by course teams and makes audio (spoken) equivalents for people with visual impairments. Another project that is currently underway is the digital audio project which aims to potentially take advantage of advances in technology, mobile devices and international standards.
Some weeks ago, my colleague tweeted along the lines of 'it must be difficult for people with visual disabilities to learn how computer programs are written and structured' (I am heavily paraphrasing, of course!) As soon as I read this tweet I began to think about two questions. The first question was: how do I go about learning how a fragment of source code works? and secondly, what might be the best way to convert a function or a 'slice' of programming code into an audio representation that helps people to understand what it does and how it is structured?
Learning from source code
How do I learn how a fragment of source code works? More often than not I view code through an integrated development environment, using it to navigate through the function (or functions) that I have to learn. If I am faced with some code that is really puzzling I might reach for some search tools to uncover the connections between different parts of the system.
If the part of the code that I am looking at is quite small and extremely puzzling, I might go as far as grab a pen and paper and begin to sketch out some notes, taking down some of the expressions that may appear to be troubling and maybe split these apart into their constituent components. I might even try to run the various code fragments by hand. If I get really confused I might use the 'immediate' window of my development environment ask my computer to give me some hints about the code I am currently examining.
When trying to understand some new source code my general approach is to try to have a 'conversation' with it, asking it questions and looking at it from a number of different perspectives. In the psychology of programming literature some researchers have written about developers using 'top down' and 'bottom up' strategies. You might have a high level hypothesis about what something does on one hand, but on the other, sections of code might help you to understand the 'bigger picture' or the intentions behind a software system.
In essence, I think understanding software is a really hard task. It is harder and more challenging than many people seem to imagine. Not only do you have to understand the language that is used to describe a world, but you also have to understand the language of the world that is described. The world of the machine and the world of the problem are intrinsically and intimately connected through what can sometimes seem an abstract collection of words and symbols. Your task, as a developer, is to make sense of two hidden worlds.
I digress slightly... If learning about computer programming code is a hard task, then it is possible that it is likely to be harder for people with visual impairments. I cannot imagine how difficult it must be to be presented with a small computer program or a function that has been read out to you. Much of the 'secondary notation', such as tabbing and white space can be easily lost if there are no mechanisms to enable them to be presented through another modality. There is also the danger that your working memory may become quickly overwhealmed with names of identifiers and unfamiliar sounding functions.
Assistive technology for everyone
The tasks of learning the fundamentals of programming and learning about a program are different, yet related. I have heard it said that people with disabilities are given real help if technologies are created that are useful for a wide audience. A great example of this is, for example, optical character recognition. Whilst OCR technology can save a great deal of cost typing, it has also created tools that enable people with low vision to scan and read their post.
Bearing the notion of 'a widely applicable technology' in mind, could it be possible to create a system that creates an interactive audio description that could potentially help with the teaching of some of the concepts of computer programming for all learners?
Whenever I read code I immediately begin to translate the notion of code into my own 'internal' notation (using different types of memory, both internal and external - such as scraps of paper!) to iteratively internalise and make sense of what I am being presented with. Perhaps equivalents of programming code could be created in a form that could be navigated. Code it not something that you read in a linear fashon - code is something you work with.
If an interesting and useful (and interactive) audio equivalent of programming code could be created there then might be the potential that these alternative forms might be useful to all students, not only to learners who necessarily require auditory equivalents.
Development directions
There are a number of tools that could help us to create what might amount to 'interactive audio descriptions of programming code'. The first is the idea of plan or schema theory (wikipedia) – the notion that your understanding of something is drawn from previous experience. Some theorists from the Psychology of Programming have extended and drawn upon these ideas, positing ideas such as key lines of code such as beacons.
Another is Green's Cognitive Dimensions framework (wikipedia). Another area to consider looking at is the interesting sub-field of Computer Science Education research. There must be other tools, frameworks and ideas that can be drawn upon.
Have you got a sec?
Another approach that I sometimes take when trying to understand something is that I ask other more experienced people for help. I might ask the question, 'what does this section represent?' or, 'what does this section do?' The answers from collegues can be instrumental in helping me to understand the purpose behind fragments of programming code.
Considering browsing
I can almost imagine what could be an audio code browser that has some functionality that allows you to change between different levels of abstraction. At one level, you may be able to navigate through sets of different functions and hear descriptions of what they are intended to do and hope to receive by way of parameters (which could be provided through comments). On another level there may be summaries of groups of instructions, like loops, with descriptions that might sound like, 'a foreach loop that contains four other statements and a call to two functions'. Finally, you may be able to tab into a group of statements to learn about what variables are manipulated, and how.
Of course this is all very technical stuff, and it could be stuff that has already been explored before. If you know of similar (or related) work, please feel free to drop me a line!
Acknowledgement: random image of code by elliotcable, licenced under creative commons, discovered using Flickr.
Source code accessibility through audio streams
One of my colleagues volunteers for the Open University audio recording project. The audio recording project takes course material produced by course teams and makes audio (spoken) equivalents for people with visual impairments. Another project that is currently underway is the digital audio project which aims to potentially take advantage of advances in technology, mobile devices and international standards.
Some weeks ago, my colleague tweeted along the lines of 'it must be difficult for people with visual disabilities to learn how computer programs are written and structured' (I am heavily paraphrasing, of course!) As soon as I read this tweet I began to think about two questions. The first question was: how do I go about learning how a fragment of source code works? and secondly, what might be the best way to convert a function or a 'slice' of programming code into an audio representation that helps people to understand what it does and how it is structured?
Learning from source code
How do I learn how a fragment of source code works? More often than not I view code through an integrated development environment, using it to navigate through the function (or functions) that I have to learn. If I am faced with some code that is really puzzling I might reach for some search tools to uncover the connections between different parts of the system.
If the part of the code that I am looking at is quite small and extremely puzzling, I might go as far as grab a pen and paper and begin to sketch out some notes, taking down some of the expressions that may appear to be troubling and maybe split these apart into their constituent components. I might even try to run the various code fragments by hand. If I get really confused I might use the 'immediate' window of my development environment ask my computer to give me some hints about the code I am currently examining.
When trying to understand some new source code my general approach is to try to have a 'conversation' with it, asking it questions and looking at it from a number of different perspectives. In the psychology of programming literature some researchers have written about developers using 'top down' and 'bottom up' strategies. You might have a high level hypothesis about what something does on one hand, but on the other, sections of code might help you to understand the 'bigger picture' or the intentions behind a software system.
In essence, I think understanding software is a really hard task. It is harder and more challenging than many people seem to imagine. Not only do you have to understand the language that is used to describe a world, but you also have to understand the language of the world that is described. The world of the machine and the world of the problem are intrinsically and intimately connected through what can sometimes seem an abstract collection of words and symbols. Your task, as a developer, is to make sense of two hidden worlds.
I digress slightly... If learning about computer programming code is a hard task, then it is possible that it is likely to be harder for people with visual impairments. I cannot imagine how difficult it must be to be presented with a small computer program or a function that has been read out to you. Much of the 'secondary notation', such as tabbing and white space can be easily lost if there are no mechanisms to enable them to be presented through another modality. There is also the danger that your working memory may become quickly overwhealmed with names of identifiers and unfamiliar sounding functions.
Assistive technology for everyone
The tasks of learning the fundamentals of programming and learning about a program are different, yet related. I have heard it said that people with disabilities are given real help if technologies are created that are useful for a wide audience. A great example of this is, for example, optical character recognition. Whilst OCR technology can save a great deal of cost typing, it has also created tools that enable people with low vision to scan and read their post.
Bearing the notion of 'a widely applicable technology' in mind, could it be possible to create a system that creates an interactive audio description that could potentially help with the teaching of some of the concepts of computer programming for all learners?
Whenever I read code I immediately begin to translate the notion of code into my own 'internal' notation (using different types of memory, both internal and external - such as scraps of paper!) to iteratively internalise and make sense of what I am being presented with. Perhaps equivalents of programming code could be created in a form that could be navigated. Code it not something that you read in a linear fashon - code is something you work with.
If an interesting and useful (and interactive) audio equivalent of programming code could be created there then might be the potential that these alternative forms might be useful to all students, not only to learners who necessarily require auditory equivalents.
Development directions
There are a number of tools that could help us to create what might amount to 'interactive audio descriptions of programming code'. The first is the idea of plan or schema theory (wikipedia) – the notion that your understanding of something is drawn from previous experience. Some theorists from the Psychology of Programming have extended and drawn upon these ideas, positing ideas such as key lines of code such as beacons.
Another is Green's Cognitive Dimensions framework (wikipedia). Another area to consider looking at is the interesting sub-field of Computer Science Education research. There must be other tools, frameworks and ideas that can be drawn upon.
Have you got a sec?
Another approach that I sometimes take when trying to understand something is that I ask other more experienced people for help. I might ask the question, 'what does this section represent?' or, 'what does this section do?' The answers from collegues can be instrumental in helping me to understand the purpose behind fragments of programming code.
Considering browsing
I can almost imagine what could be an audio code browser that has some functionality that allows you to change between different levels of abstraction. At one level, you may be able to navigate through sets of different functions and hear descriptions of what they are intended to do and hope to receive by way of parameters (which could be provided through comments). On another level there may be summaries of groups of instructions, like loops, with descriptions that might sound like, 'a foreach loop that contains four other statements and a call to two functions'. Finally, you may be able to tab into a group of statements to learn about what variables are manipulated, and how.
Of course this is all very technical stuff, and it could be stuff that has already been explored before. If you know of similar (or related) work, please feel free to drop me a line!
Acknowledgement: random image of code by elliotcable, licenced under creative commons, discovered using Flickr.