john Gill technology header image

Requirements for the Interconnection of Assistive Technology Devices and Information and Communication Technology Systems

John Gill

July 2001


1 Introduction

2 Scenarios

3 The Interface

4 The Questionnaire

5 Recommendations

Appendix 1 Websites

Appendix 2 Relevant Assistive Devices in ISO 9999

Appendix 3 The Wireless Interconnection Technologies

Appendix 4 The Questionnaire

Appendix 5 Responses to the Questionnaire

Appendix 6 Alternate User Interface Standard

Appendix 6

Alternate User Interface Standard

This appendix comes from the Trace website.

V2 is the name of a technical committee at the National Committee for Information Technology Standards (NCITS) that is charged with developing national standards for Information Technology Access Interfaces. The first project of the V2 technical committee will be to develop standards for an Alternative Interface Access Protocol (AIAP). This protocol would complement and build on industry activity in home networking, wireless networking, and metadata registries for discovery and interoperation of devices. The Alternative Interface Access Protocol and related standards will enable IT products to be more accommodating of the needs and preferences of the consumer by allowing for alternative user interfaces. While addressing the special needs of people with disabilities, the option to change interfaces will have a broader market application.

Developing a Markup Language for Alternate Interfaces

A core part of V2's mandate is to develop a standard specification for remote consoles. This will involve an XML-based markup language that conveys user interface information from a target service to the remote console device. A device-specific browser on the remote console device interprets this information and renders it in a way that fits the device's capabilities and user's preferences. Rendering examples include visual display, audio output, keyboard input, speech input, Braille output, in any reasonable combination.

Trace contributes to this activity in several ways:

  • By bringing in its expertise in the area of IT accessibility, and drawing from the experiences made in the former URCC project.
  • By driving the edit process of a remote console standard specification working document.
  • By leveraging its prototypical implementations to explore the solution space for a markup language for alternate interfaces and to validate possible solutions.

Experimental Prototyping

The experimental prototyping activity at Trace aims to implement an array of remote control scenarios, varying both, the remote console device and the target service. Prototypes take advantage of powerful existing technologies, like Bluetooth, Universal Plug and Plug (UPnP), and Jini.

Goals of the prototyping thread include:

  • Exploring the usefulness of different existing technologies for implementing a remote console standard specification.
  • Identifying the requirements of a remote console standard specification regarding its underlying network environment.
  • Providing a testbed for the development of a markup language specification.
  • Providing proof-of-concept implementations for the remote console standard specification.
  • Given, a standard specification for remote consoles will be released, providing potential reference implementations with Bluetooth, Universal Plug and Play, and Jini.

Designing a Comprehensive AIAP Architecture

Activities serve the development of a comprehensive standard for alternate user interfaces, in which a remote console specification would be one part.



The General Input Device Emulating Interface (GIDEI) Proposal defines a connection and data communication protocol between alternate input devices (e.g., augmentative and alternative communication [AAC] devices) and interfaces which emulate standard computer input devices.

There are several alternate input or AAC devices capable of emulating the standard computer input devices. However, by design the computer may not understand how to talk with an AAC device without some kind of communication translation taking place. This communication or "input" translation is usually accomplished by the emulating interface. The emulating interface can be a hardware device (eg Trace Transparent Access Module) or special software running on a computer (eg "SerialKeys" feature of AccessDOS) that converts the characters and commands transmitted over the serial cable into keystroke and mouse actions understood by the receiving computer. In either case, a hardware interface is necessary to supply or support the connection from the alternate input device to the emulating interface. Therefore, the hardware interface can be physically located on an external hardware device attached to the computer or be part of the computer itself.

GIDEI Purpose

The main purpose of the GIDEI Proposal is to provide developers of AAC devices a viable approach that enables them to use their devices as replacements for the normal keyboard and mouse on general purpose computers. This allows people who are "unable" to effectively operate the normal keyboard and mouse, to access computer systems and software from their alternate input device.


A person who is unable to access a computer is at a great disadvantage in today's society. Many schools and jobs require the use of computers daily. The person who is physically unable to use a computer keyboard or mouse can not participate in certain activities on an equal level with others. Fortunately, many special electronic aids used by people with disabilities can be used as alternate input devices for standard computers. Any device which is capable of connecting to a computer as described in the GIDEI Proposal and also capable of transmitting ASCII information arranged to conform with the GIDEI data communication protocol, should be capable of acting as an alternate input device.

The following Figures 1 and 2, should help you to better understand the terminologies of GIDEI, alternate input device, and emulating interface. Each figure contains a picture of a computer, showing the connection for the standard keyboard and mouse input devices. Each figure also contains a picture of an alternative input device (e.g., AAC device) that is connected in some fashion to the computer. It is this AAC device-to-computer connection that is the critical component you need to understand, when determining if you need to follow the GIDEI Proposal for connection and data communication purposes between your AAC device and the computer. Hopefully, the detailed discussion which follows will assist you to better understand the proper application of the GIDEI Proposal, with or without Figures 1 and 2 as a reference.

Image of AAC device-to-computer connection

Figure 1

The example in Figure 1 shows an AAC device connected directly to the computer using a serial cable. The serial cable from the AAC device connects directly to the computer serial or "com" port, which is usually located at the rear of the computer. The user in Figure 1 wishes to make selections on their AAC device, transmit the information across the serial cable to the computer, and expects the computer to understand that their selections represent keyboard or mouse actions. In Figure 1, the "connection" protocol as defined in the GIDEI Proposal includes the serial cable, its internal wiring or "pin-out" arrangement, and the amount, speed, and characteristics of the information the user sends from their AAC device to the computer. The "data communication" protocol as defined in the GIDEI Proposal concerns the contents of the information or "data packets" themselves, that the user either creates or which may be prestored on their AAC device.

You might expect that once the user sets up their AAC device to conform to both the connection and data communication protocol as defined in the GIDEI Proposal, that their AAC device would be able to communicate with and control the computer. Unfortunately, without one additional missing piece, the computer will not be able to understand what the AAC device is transmitting.

Computers are very powerful tools. However from a user input standpoint, most computers only understand the user when given input or instructions from the keyboard or the mouse. By design, keyboards and mice each use their own language to communicate with the computer. Therefore, once the AAC device is connected to the computer as shown in Figure 1 (e.g., either directly through the use of a serial cable or indirectly via some kind of wireless link), something must act as a "go-between" to translate the AAC device language into the keyboard or mouse language understood by the computer. Since the information transmitted from the AAC device is well defined by the GIDEI Proposal data communication protocol, a good portion of the translation work is already completed. In Figure 1, the missing "go-between" piece is the software that functions as the "emulating interface" on the computer receiving the AAC device information. Emulating interface software would do the translation of the information sent from the AAC device into keyboard and mouse language understood by the computer. The emulation interface software is the missing "go-between" piece that allows the AAC device which is connected directly to the computer com port to emulate the standard input devices. The SerialKeys feature provided as part of the AccessDOS software package is an example of emulating interface software.

Image of AAC device-to-computer connection

Figure 2

The AAC to computer connections shown in Figure 2 are very similar to the example shown in Figure 1, except the serial cable coming from the AAC device is now connected to a "black box" outside the computer rather than directly to the computer. (Again, please note that the serial cable could also be a wireless serial link). The black box has cables coming "from" the keyboard and mouse, and cables "exiting" the black box which connect to the computer at the same place the keyboard and mouse cables would have been plugged into the computer, had the black box not been used. The black box also has a connection for a serial cable used to connect to the AAC device. Again the "connection" protocol as defined in the GIDEI Proposal includes the serial cable, its internal wiring or "pin-out" arrangement, and the amount, speed, and characteristics of the information the user sends from their AAC device to the black box. The "data communication" protocol as defined in the GIDEI Proposal concerns the contents of the information or "data packets" themselves, that the user either creates or which may be prestored on their AAC device.

In this example, information is sent from the AAC device to the black box. Therefore, the black box needs to perform the necessary translation of the AAC device information prior to sending it to the computer through the keyboard and mouse cables. The black box becomes the missing "go-between" piece in this example. Since the black box is connected between the standard input devices and the computer, it understands the language used by the keyboard and mouse when talking with the computer, and is therefore capable of allowing the AAC device to again "emulate" the standard input devices. In this case, we would refer to the black box as an "emulating interface device", since it is actually a piece of hardware which a user could take from computer-to-computer, as their needs might dictate. The Trace Transparent Access Module (T-TAM) is one example of an emulating interface device.

GIDEI Commands At A Glance

Keyboard Commands

Typing Characters: simply send the character, which is sometimes referred to as an "ASCII character," to the emulating interface. Some examples of ASCII characters are the lowercase letters "a-z", uppercase letters "A-Z", most common punctuation characters like comma ",", period ".", etc., and the numbers 0 through 9.

Typing Specific Keys: each key is designated by a name. To type the key, use the Key Name in an escape sequence. An escape sequence is simply the ESCAPE character (ASCII 27, represented as "<esc>" below), followed by a sequence of other ASCII characters, and ending with the PERIOD character (ASCII 46, "."). (Note: Many AAC devices can generate an ESCAPE character using a combination of the control key plus a left bracket (e.g., control+[), which may appear on the AAC device display as "^[" .)

<esc> Key Name .
For example: <esc> pageup.

Other Key Actions: to perform other actions use escape sequences of the general form:

<esc> , command [ , Key Name 1 ] [ , Key Name 2 ] [ , Key Name 3 ] [ , Key Name 4 ] [ , Key Name 5 ] .

Valid keyboard commands are:


used to type one to five keys "simultaneously".

<esc> , combine , ctrl , alt , del .


holds the specified key(s) down until the next key is "typed".

<esc> , hold , shift .


used for future expansion of the standard and to access a list of commands related to either changing or querying the keyboard of the computer the user is trying to emulate. GIDEI2 supports the subcommand "lang" to allow augmentative communication users better emulation of keyboards related to their particular geographic locale.


locks keys down until the rel command is used to release them (e.g., a "key down" is transmitted with no "key up").

<esc> , lock , ctrl .


releases all or only specified "locked" keys (e.g., sends "key up") and clears any pending hold keys.

<esc> , rel .

Mouse Commands

Mouse button actions are performed using escape sequences of the general form:

<esc> , command , button identifier .

Button identifier now refers to a mouse button action, such as pressing mouse button #1, rather than a location, such as pressing the "left" mouse button. This distinction is being made because many operating systems now allow the user to reprogram the buttons on the mouse to perform any desired action. Therefore, users are encouraged to follow the new button identifiers, rather than previous GIDEI conventions which defined a "left" or "right" mouse button.

Valid mouse button commands are:


sends a "button down" immediately followed by a "button up" of the specified mouse button identifier.

<esc> , click .


sends two click actions of the specified mouse button identifier.

<esc> , dblclick .


locks the specified mouse button identifier(s) down (e.g., a "button down" is transmitted with no "button up").

<esc> , moulock , but1 .


releases "locked" mouse button identifier(s) (e.g., sends "button up").

<esc> , mourel .

Mouse movement actions are performed using the escape sequence formats shown below:


used to remember a specific location of the mouse cursor for subsequent return actions.

<esc> , anchor , X .

(where X is a lowercase letter of "a" through "z", allowing for 26 separate "anchor" locations.)


moves the mouse cursor a specified distance in the specified direction(s).

<esc> , move , X direction , Y direction .


moves the mouse cursor in a continuous motion, in the direction specified and also at the speed specified. On systems which support this type of mouse motion using an alternate method (e.g., MouseKeys), this command is not necessary.


A) moves the mouse cursor to a specified absolute screen location when both an X and Y position is indicated.

<esc> , goto , X position , Y position .

B) moves the mouse cursor to a specified screen location when only a single X value is entered, based upon that location having been preassigned using the "anchor" command (see above).

<esc> , goto , X .

(where X is a lowercase letter of "a" through "z", allowing for 26 separate "anchor" locations.)


allows the user to stop the mouse cursor after the mouse cursor was put in continuous motion using the mouse "mougo" command (see above). On systems which support this type of mouse motion using an alternate method (e.g., MouseKeys), this command is not necessary.


initializes the mouse settings and coordinates to (0,0) and moves the pointer to the upper left hand corner of the screen.

<esc> , moureset .

Miscellaneous Command(s)

If a user attaches to a computer which has the "emulating interface" already running, they need to first establish a connection or get their AAC device to send characters at the same "rate" at which the emulating interface is accepting characters. The easiest and quickest way to ensure this, is to cause the emulating interface to reset to a "known" state. To accomplish this, if the user can cause 3 "framing" communication errors to occur, any emulating interface which follows the GIDEI Proposal is supposed to perform a reset. If the user transmits 3 consecutive NULL characters (ASCII 0) at 300 baud, any GIDEI compliant emulating interface, if not already at 300 baud, should reset itself to 300 baud. This ensures a "known-data-rate" communication connection between the user's AAC device and the emulating interface. The full GIDEI Proposal has an entire section describing the "Initial Connection Protocol".


sets the baud rate of the GIDEI.

<esc> , baudrate , XXXXX .

When the AAC user is communicating with the emulating interface at a known "rate", the baudrate command can be used to change-to another "rate", should the user wish to do so.


John Gill Technology Limited Footer
John Gill Technology Limited Footer