john Gill technology header image

The Design of Man-Machine Interfaces for Use by Visually Disabled People

Dr John Gill
July 1995

One aim of Royal National Institute of the Blind is to stimulate and co-ordinate technological research and development to benefit visually disabled persons. This involves determining priorities since there are finite resources - both in skilled manpower and finance. However an area of increasing importance to people with disabilities, in particular visual disability, is the design of the man-machine interface.

For someone with no useful vision, the interface usually has to employ auditory or tactual output. Although considerable research has been done in this area, there is a danger of there being an increasing mismatch between the research which is being done and the needs of the users. For instance, ten years ago the main users of synthetic speech devices were blind computer programmers as an aid to employment, but in the next few years older blind persons will need to use speech output devices to control domestic equipment such as microwave ovens. It is not obvious that those undertaking research on speech technology appreciate that 35% of the visually disabled population have a hearing deficit. Therefore there is an important role for user organisations to ensure that research workers are aware of the present and future unmet needs of visually disabled people.

Speech Technology

Recent developments in speech technology have led to significant improvements in quality and accuracy and a reduction in prices. This is important for people with disabilities since speech technology can be used for interpersonal communication, access to information and control of the environment.

Although the intelligibility of the available speech synthesis systems is quite high for a number of languages, there is evidence that significantly better quality should be achievable. Research is focused on naturalness, prosody and flexibility (eg different voices). Other important research areas are modelling of emotions and speech synthesis from the meaning of the text.

Recent developments in speech recognition have been in two areas - the academic where the focus is on improved techniques such as stochastic modelling, search and neural networks, and the pragmatic where the focus is on accuracy, robustness and computational efficiency permitting real-time performance with affordable hardware.

Speech recognition is difficult because:

  • The basic units of speech are hard to recognise.
  • Continuous speech adds more difficulties.
  • Speaker and environmental differences are very important.
  • There is insufficient knowledge about the human language understanding process.

Speech dialogue systems, in which speech synthesis and recognition are used in a man-machine dialogue, have been demonstrated for applications such as bank cash dispensers and hands-free voice diallers on public telephones. Current systems are limited to vocabularies of about 100 words, but laboratory prototypes exist with vocabularies of over 1000 words.

Speech synthesis is a stable technology, but speech recognition is still substantially inadequate when compared with human capabilities. Research on speech processing has had a significant impact on the design of cochlea implants as well as digital hearing aids. However there is a need for better understanding of user needs and matching these needs to appropriate research and development. There is also a need to use current knowledge in practical situations to evaluate potential benefits.

Access to Text

Some blind people have seen technology as being the magic answer to their problems in accessing information; an example is the distribution of textual information for blind persons in digital form. However, this involves there being some standards for the data format which must be such that it facilitates easy searching, browsing and reading by a blind person with a variety of output modalities such as Braille or synthetic speech. Structures such as hypertext often require intervention to convert the raw text to an optimal structured file - this adds to the cost. A separate matter is the method of distribution - this could include floppy disc as well as telecommunication links which will come increasingly attractive when ISDN becomes generally available to domestic users.

However developments for the sighted population are creating a new set of problems. For instance the increasing use in schools of multimedia CD-ROMs (eg Microsoft Encarta) makes it increasingly difficult for a blind child integrated in a class of sighted children. There have been various attempts to encourage the developers of such systems to add a second set of access software on the CD-ROM which could permit access to the textual information using DOS screen readers.

Another area giving considerable cause for concern is the increasing use of the World Wide Web on Internet; this is a hypermedia system. The textual part can be accessed using synthetic speech or Braille, but there are problems if the hypertext links are done using a graphical object on the screen. It is possible for the designer of the web page to associate hidden text with a graphical object, but this rarely done in practise.

The Graphical User Interface

For many software houses, user-friendly computer interfaces mean GUIs (graphical user interfaces). In the medium term, new operating systems, such as the next version of X-Windows, will have "hooks" which will assist in the development of adaptations for blind persons.

Most of the available screen readers for Windows use an off-screen model and provide the user with synthetic speech and/or Braille output. Some systems additionally use non-speech sounds and graphics tablets to give extra information about the layout of the visual presentation; this can be important in some applications. One problem with off-screen models is that they need to be updated for each new version of the GUI.

There are still significant problems in the speed of use of these systems compared to a sighted person, and the amount of extra training needed by the blind user. These areas have been largely ignored by the commercial developers of GUI screen readers.

Smart Housing

GUIs will be used in other applications than access to conventional computer systems; for instance "smart housing". The area of smart housing for disabled persons is of great interest to the governments of a number of countries. The reason being that they have done a simple calculation of the number of disabled and elderly persons by the turn of the century, and the number of people in work at that time who will be paying taxes. This calculation has led them to conclude that a greater percentage of elderly and disabled persons will have to be able to live independently, and they foresee that the appropriate use of smart housing will make this economically possible. It is important that the design of the man-machine interfaces for smart housing takes into account the needs of disabled persons.

Within a traditional home, automation can assist a person to control a device in another part of the house as well as establish the status of the device (eg is the electric blanket on?). In some situations, the controller can suggest appropriate actions, which can be of great help to those with short-term memory problems. The design of the controller can be tailored to the needs of the user; for instance, a controller for use by a blind person could incorporate speech output.

Smart houses often use a central data highway, called a "bus", which can be implemented using cable, infra-red or radio links; the choice depends on factors such as speed of data transmission, building design and cost. It is much easier to incorporate such a bus into a new purpose-built environment than to adapt an old building. A major problem is still the lack of a single standard covering home bus systems.

The user interface is the single component in such systems, upon which everything else will be judged. If the interface is confusing, the system will be thought of in that way. To make such systems appear simple is extremely complex, but nonetheless essential. The interface must be appropriate to the special needs of the user. It is important that the user interface is consistent across all the applications.

The concept of smart houses is both exciting and extremely challenging. Much like the introduction of television, you may love it or hate it; but either way it is here to stay. The further development of smart houses as a tool to help people with disabilities is currently inhibited by a lack of clear standards and the availability of the technology. However there are indications that these problems are being overcome, and that people with disabilities may be the first to benefit from smart housing.

Audio Description

One piece of domestic equipment, which cannot be adapted for blind persons just by changing the interface of the controller, is the television set. However audio description of television programmes has proved popular in north America. The audio description of the action of the programme is inserted in the gaps in the dialogue. The technical problem is that, in Europe, there is not a spare audio channel for transmitting the audio description. Therefore other methods are being investigated, but with the most promising option being to use two lines of the vertical blanking interval.

Talking Books

The storage of audio information has long been of importance to blind persons. Problems with conventional compact cassette systems have included controlling the speed of playback and indexing. Compressed (and expanded) speech chips have been around for some years, but relatively few tape recorders include this facility. Although various methods have been devised for indexing, none could be described as being ideal. Therefore the advent of digital storage (eg compact disc) offers exciting possibilities since it is capable of giving excellent audio quality in addition to sophisticated facilities such as compressed speech and indexing. It would also be relatively simple to incorporate circuitry to partly compensate for any hearing loss of the user. However there is a significant problem in that CD type technology could be superseded by solid state recording at some date in the future. Therefore the investment in CD type technology has to be written off before the time that any new technology means that the manufacturer will no longer support the old CD technology.

Developments in telecommunications may also have an impact in this area. For instance it would be possible to consider audio books stored on a central computer and transferred to the user's machine over the telephone line. If a book used 2 Gigabits, then it would take 20 minutes to transfer at 2 Mbits/sec (via copper wire connection) and 20 seconds at 155 Mbits/sec (via fibre optic connection). However it is now technically possible to transfer data at 2 Gigabits/sec on an optical fibre, which would reduce the time to 1 second. If this was ASCII text, then the whole Encyclopaedia Britannia could be transmitted in half a second.

Self-service Terminals

Self-service terminals are being used by the general public for an increasing range of applications. Bank cash dispensers and ticket selling machines for public transport now offer a bewildering number of choices to the user. To handle this increased number of choices, the terminal often incorporates a sophisticated interface which can cause problems for users who are elderly or have a disability. To help these users it may be possible to modify the terminal interface to meet the user’s needs (eg large characters on the screen).

To select a preferred interface, the user could simply press a button or select from a menu on the screen. Another method would be to store the information on the customer’s card. With a magnetic stripe card there is very limited spare capacity for storing this information, but this method has been used successfully for storing the user’s preference for displayed language (eg English or French). A smart card has fewer restrictions on storage capacity so appears to be ideal for this purpose, as long as an international standard is agreed for the coding of this information.

The customer is often required to input information to a terminal, but many elderly customers need more time to complete the transaction before being ‘timed out’; on a public telephone, this could involve storing the number being dialled and then sending it at a press of a function key.

Some elderly and disabled customers may want the terminal to offer them a restricted number of choices, such as automatically dialling a pre-stored number on a telephone card, or the ability to store a few telephone numbers and select them by two or three key presses.

Terminals often display information which some users find difficult to read or understand. These users might be helped by large characters on the screen, or speech prompts, or the ability to select a preferred frequency response to compensate for their hearing loss.

Since there is no card insertion, contactless cards would help people in wheelchairs who cannot reach the slot for the card reader, those with hand tremor or arthritis, and blind persons.

Systems for interrogating cards at distances of a few metres were developed for road charging, but this technology could have numerous benefits for blind and physically disabled persons. For instance it could trigger an audible location signal which would help blind persons find the terminal.

Orientation and Navigation

Despite considerable effort devoted to the technological development of electronic mobility aids, their market penetration has been very low. In recent years, there has been more interest in orientation systems. Typically these involve infra-red transmitters which are modulated with a speech message. The blind user carries a small receiver which gives the speech message via a loudspeaker or an earpiece. The transmitters are mounted at street corners, on light-controlled pedestrian crossings (to give the status of the lights), and on buses (to give the destination of the bus). A different approach is to use tag technology as used in shop security systems. In this case, the tag indicates to a machine the presence of a blind person and the facilities they require; the tag may be pre-coded or controlled by the user. In effect, this is a form of contactless smart card.

There has been interest in utilising geographic information systems which are being developed for the sighted population. Many of these systems not only include geographic information but also information such as bus timetables. The user’s position is determined using a global positioning system, but there are practical problems such as the need to have line of sight to at least three satellites.

Despite these practical problems, this area is attracting considerable interest in both Europe and north America. In Europe, the emphasis is on assessing user needs and designing the optimum man-machine interface for the blind pedestrian.

Virtual Reality

In the longer term, technological developments in other fields could have benefit for those working with visually disabled persons. For instance, virtual reality technology could provide realistic simulations of visual defects including fields, acuity, colour discrimination and effects of illumination; in addition, multiple handicaps could be simulated. Such a system might be useful for evaluating proposed public buildings.

Virtual reality (VR) technology is of inherent interest to people with disabilities since:

  • It allows people with sensory disabilities to perceive what they might not otherwise be able to since it can gather information in a sensory modality in which they are impaired and deliver it to one where they are not.
  • It can render a world in a customised manner - this can help people to start learning activities in a simplified form before transferring their skills to the more complex real world.
  • Well-design VR devices are inherently adaptable to a wide range of individual needs.
  • Users of networked virtual environments will have control over the way in which they project themselves to others. This means that those with special needs can interact with other users on an equal footing.

VR techniques can be used to compensate for motor and sensory deficits, allowing a disabled person to explore and manipulate new environments. It therefore has the potential to be used as a training aid for skills such as spatial co-ordination and orientation.

As intriguing as VR is, the enabling technology is still crude. Major technological hurdles exist in the area of tracking a person’s motion and position in a non-intrusive way, in displaying high definition stereo colour images of the scene covering the user’s peripheral vision and, in the area of image generation, speed for a smooth and realistic animation of the scene. Tactile output and the construction of physical images, to support the visual images in virtual environments, require further development to produce realistic sensations.

Virtual reality and telecommunications are both fields that will evolve and grow very rapidly by the end of the century. Applications that are currently stand-alone will be supported on high bandwidth public multimedia networks that are arising out of the integration of telecommunication and computing. In coming together, VR and telecommunication technology have much to offer people with disabilities.

The Information Superhighway

The 1990s represent a period of great technological, regulatory and commercial change in the telecommunications world. Depending on how it evolves, the emerging "information superhighway" has the potential to make possible a wide range of applications and services which can contribute to the quality of life. For people with disabilities new opportunities will be presented for interpersonal communication (eg videoTelephony), for access to remote services (eg Telemedicine), for carrying out transactions from the home (eg Teleshopping) and for new forms of participation (eg distance learning and Teleworking).

These new opportunities bring a new dimension to the question of access to telecommunications, namely, will they be available for everyone who could benefit from them? For some groups, high quality videoTelephony services may be desired to support sign language and other forms of visual communication, for others, a good quality and inexpensive voice Telephony service may the main priority. For some groups, remote access to entertainment and transaction services may be the primary demand, for others, the networking and other information services provided by Internet-type facilities are likely to be of great interest and value.

This diversity means that all levels of access, including PSTN, ISDN, mobile and broadband need to be taken into account in the development of policies to ensure that needs will be met as the information superhighway evolves. One factor of importance will be the approach to infrastructure roll-out which is adopted in Europe, and the extent to which this will be left solely to the marketplace or be incorporated within public policy. A related factor will be the geographical distribution of access to advanced services and the possibility that access will evolve in an uneven fashion between and within countries. Finally, given the limited financial means of many people with disabilities, connection and usage costs will exert a crucial influence on whether the new opportunities will be taken up.

These policy issues are important but policy makers too often are unaware of the need to develop appropriate man-machine interfaces for people with disabilities. An important message is that good design for disabled persons is frequently good design for everyone.


John Gill Technology Limited Footer
John Gill Technology Limited Footer