The haptic exploration of 2-D images is a challenging problem in computer haptics. Research on the topic has primarily been focused on the exploration of maps and curves. This thesis describes the design and implementation of a system for the haptic exploration of photographs. The system builds on various research directions related to assistive technology, computer haptics, and image segmentation. An object-level segmentation hierarchy is generated from the source photograph to be rendered haptically as a contour image at multiple levels-of-detail. A tool for the authoring of object-level hierarchies was developed, as well as an innovative type of user interaction by region selection for accurate and efficient image segmentation. According to an objective benchmark measuring how the new method compares with other interactive image segmentation algorithms shows that our region selection interaction is a viable alternative to marker-based interaction. The hierarchy authoring tool combined with precise algorithms for image segmentation can build contour images of the quality necessary for the images to be understood by touch with our system. The system was evaluated with a user study of 24 sighted participants divided in different groups. The first part of the study had participants explore images using haptics and answer questions about them. The second part of the study asked the participants to identify images visually after haptic exploration. Results show that using a segmentation hierarchy supporting multiple levels-of-detail of the same image is beneficial to haptic exploration. As the system gains maturity, it is our goal to make it available to blind users.

Access technologies facilitate novel and alternative methods for individuals with disabilities to interact with their environment. Finding suitable access solutions for children with severe spastic quadriplegic cerebral palsy can be difficult because of their poor motor control and targeting abilities due to spasticity at the limbs, neck, and head. In this research a multiple camera tongue switch was developed for a 7 year-old case study participant with severe spastic quadriplegia. Remotely via video, this system reacts to tongue protrusions as cues for single-switch access. Having multiple cameras mitigates targeting problems with the head that conventional single camera systems would present. Results of a usability experiment with the participant show that good sensitivity (82%) and specificity (80%) can be achieved with a non-contact tongue protrusion access modality for a user with spastic quadriplegia. Moreover, the experiment verified that the extra cameras improve utility of video-based access technologies for the target population.

People with severe motor-impairments due to Spinal Cord Injury (SCI) or Spinal Cord Dysfunction (SCD), often experience difficulty with accurate and efficient control of pointing devices (Keates et al., 02). Usually this leads to their limited integration to society as well as limited unassisted control over the environment. The questions “How can someone with severe motor-impairments perform mouse pointer control as accurately and efficiently as an able-bodied person?” and “How can these interactions be advanced through use of Computational Intelligence (CI)?” are the driving forces behind the research described in this paper. Through this research, a novel fuzzy mouse cursor control system (FMCCS) is developed. The goal of this system is to simplify and improve efficiency of cursor control and its interactions on the computer screen by applying fuzzy logic in its decision-making to make disabled Internet users use the networked computer conveniently and easily. The FMCCS core consists of several fuzzy control functions, which define different user interactions with the system. The development of novel cursor control system is based on utilization of motor functions that are still available to most complete paraplegics, having capability of limited vision and breathing control. One of the biggest obstacles of developing human computer interfaces for disabled people focusing primarily on eyesight and breath control is user’s limited strength, stamina, and reaction time. Within the FMCCS developed in this research, these limitations are minimized through the use of a novel pneumatic input device and intelligent control algorithms for soft data analysis, fuzzy logic and user feedback assistance during operation. The new system is developed using a reliable and cheap sensory system and available computing techniques. Initial experiments with healthy and SCI subjects have clearly demonstrated benefits and promising performance of the new system: the FMCCS is accessible for people with severe SCI; it is adaptable to user specific capabilities and wishes; it is easy to learn and operate; point-to-point movement is responsive, precise and fast. The integrated sophisticated interaction features, good movement control without strain and clinical risks, as well the fact that quadriplegics, whose breathing is assisted by a respirator machine, still possess enough control to use the new system with ease, provide a promising framework for future FMCCS applications. The most motivating leverage for further FMCCS development is however, the positive feedback from persons who tested the first system prototype.

Access technologies facilitate novel and alternative methods for individuals with disabilities to interact with their environment. Finding suitable access solutions for children with severe spastic quadriplegic cerebral palsy can be difficult because of their poor motor control and targeting abilities due to spasticity at the limbs, neck, and head. In this research a multiple camera tongue switch was developed for a 7 year-old case study participant with severe spastic quadriplegia. Remotely via video, this system reacts to tongue protrusions as cues for single-switch access. Having multiple cameras mitigates targeting problems with the head that conventional single camera systems would present. Results of a usability experiment with the participant show that good sensitivity (82%) and specificity (80%) can be achieved with a non-contact tongue protrusion access modality for a user with spastic quadriplegia. Moreover, the experiment verified that the extra cameras improve utility of video-based access technologies for the target population.

The haptic exploration of 2-D images is a challenging problem in computer haptics. Research on the topic has primarily been focused on the exploration of maps and curves. This thesis describes the design and implementation of a system for the haptic exploration of photographs. The system builds on various research directions related to assistive technology, computer haptics, and image segmentation. An object-level segmentation hierarchy is generated from the source photograph to be rendered haptically as a contour image at multiple levels-of-detail. A tool for the authoring of object-level hierarchies was developed, as well as an innovative type of user interaction by region selection for accurate and efficient image segmentation. According to an objective benchmark measuring how the new method compares with other interactive image segmentation algorithms shows that our region selection interaction is a viable alternative to marker-based interaction. The hierarchy authoring tool combined with precise algorithms for image segmentation can build contour images of the quality necessary for the images to be understood by touch with our system. The system was evaluated with a user study of 24 sighted participants divided in different groups. The first part of the study had participants explore images using haptics and answer questions about them. The second part of the study asked the participants to identify images visually after haptic exploration. Results show that using a segmentation hierarchy supporting multiple levels-of-detail of the same image is beneficial to haptic exploration. As the system gains maturity, it is our goal to make it available to blind users.

For persons with disabilities, their impairments lead to many inconveniences in their daily life. Luckily, assistive technology devices can help compensate functional limitations and enable them to participate in the activities of daily life. In 2010, there were 126,693 disabilities in the Kaohsiung area. Subsidies provided by Kaohsiung city government and Kaohsiung county government reach respectively NT$39,145,000 and NT$ 25,489,000 in 2010. Nevertheless, such great amount of money invested does not work as it has been expected. There appear to remain a number of barriers. One such example would be that persons with disabilities lack the knowledge about assistive technology, so they do not know where to ask for help in the first place. On the other hand, lack of consideration for the needs of a person with disability result in them getting inappropriate assistive technology devices. These problems deserve the attention of government and related associations. This research aims to solve the problems between the supply and demand of assistive technology devices or services. It reviews and studies papers concerning the same issue in an attempt to create a new service platform. This platform is created according to information and suggestion gathered from experts of this field and voice from users of assistive devices. Their voices and assessments provide valuable direction to shape this new platform. This research assesses the possibility of this new assistive technology service platform from several aspects, including its political, legislative, supplying, and demanding dimensions. The purpose of this study is to reorganize diverse resources to provide a more convenient information-gathering method and to offer a complete service for disabilities people.

The CopyCat project is an interdisciplinary effort to create a set of computer-aided language learning tools for deaf children. The CopyCat games allow children to interact with characters using American Sign Language (ASL). Through Wizard of Oz pilot studies we have developed a set of games, shown their efficacy in improving young deaf children's language and memory skills, and collected a large corpus of signing examples. Our previous implementation of the automatic CopyCat games uses automatic sign language recognition and verification in the infrastructure of a memory repetition and phrase verification task. The goal of my research is to expand the automatic sign language system to transition the CopyCat games to include the flexibility of a dialogue system. I have created a labeling ontology from analysis of the CopyCat signing corpus, and I have used the ontology to describe the contents of the CopyCat data set. This ontology was used to change and improve the automatic sign language recognition system and to add flexibility to language use in the automatic game.

A brain-machine interface (BMI) uses signals from the brain to control electronic devices. One application of this technology is the control of assistive devices to facilitate movement after paralysis. Ideally, the BMI would identify an intended movement and control an assistive device to produce the desired movement. To implement such a system, it is necessary to identify different movements involving a single limb and users must be able to issue commands at any instant instead of only during specific time windows determined by the BMI itself. A novel processing technique to identify voluntary movements using only four electrodes is presented. Histograms containing the spectral components of intracranial neural signals displaying power changes correlated with movement were unique for each of three movements performed with one limb. Off-line classification of the histograms allowed the identification of the performed movement with an accuracy of 89%. This movement identification system was interfaced with a neuroprosthesis for grasping, fitted to a tetraplegic individual. The user pressed a button triggering the random selection and classification of a brain signal previously recorded intracranially from a different person while performing specific arm movements. Correct identification of the movement triggered grasping functions. Movement identification accuracy was 94% allowing successful operation of the neuroprosthesis. Finally, two BMIs for the real-time asynchronous control of two-dimensional movements were created using a single electrode. One EEG-based system was tested by a healthy participant. A second system was implemented and tested using recordings from an individual undergoing clinical intracranial electrode implantation. The users modulated their 7 Hz-13 Hz oscillatory rhythm through motor imagery. A power decrease below a threshold activated a ``brain-switch''. This switch was coupled with a novel asynchronous control strategy to control a miniature remotely-controlled vehicle as well as a computer cursor. Successful operation of the EEG system required 6 hrs of training. ECoG control was achieved after only 15 minutes. The operation of the BMI was simple enough to allow users to focus on the task at hand rather than on the actual operation of the BMI.

A brain-machine interface (BMI) uses signals from the brain to control electronic devices. One application of this technology is the control of assistive devices to facilitate movement after paralysis. Ideally, the BMI would identify an intended movement and control an assistive device to produce the desired movement. To implement such a system, it is necessary to identify different movements involving a single limb and users must be able to issue commands at any instant instead of only during specific time windows determined by the BMI itself. A novel processing technique to identify voluntary movements using only four electrodes is presented. Histograms containing the spectral components of intracranial neural signals displaying power changes correlated with movement were unique for each of three movements performed with one limb. Off-line classification of the histograms allowed the identification of the performed movement with an accuracy of 89%. This movement identification system was interfaced with a neuroprosthesis for grasping, fitted to a tetraplegic individual. The user pressed a button triggering the random selection and classification of a brain signal previously recorded intracranially from a different person while performing specific arm movements. Correct identification of the movement triggered grasping functions. Movement identification accuracy was 94% allowing successful operation of the neuroprosthesis. Finally, two BMIs for the real-time asynchronous control of two-dimensional movements were created using a single electrode. One EEG-based system was tested by a healthy participant. A second system was implemented and tested using recordings from an individual undergoing clinical intracranial electrode implantation. The users modulated their 7 Hz-13 Hz oscillatory rhythm through motor imagery. A power decrease below a threshold activated a ``brain-switch''. This switch was coupled with a novel asynchronous control strategy to control a miniature remotely-controlled vehicle as well as a computer cursor. Successful operation of the EEG system required 6 hrs of training. ECoG control was achieved after only 15 minutes. The operation of the BMI was simple enough to allow users to focus on the task at hand rather than on the actual operation of the BMI.

The haptic exploration of 2-D images is a challenging problem in computer haptics. Research on the topic has primarily been focused on the exploration of maps and curves. This thesis describes the design and implementation of a system for the haptic exploration of photographs. The system builds on various research directions related to assistive technology, computer haptics, and image segmentation. An object-level segmentation hierarchy is generated from the source photograph to be rendered haptically as a contour image at multiple levels-of-detail. A tool for the authoring of object-level hierarchies was developed, as well as an innovative type of user interaction by region selection for accurate and efficient image segmentation. According to an objective benchmark measuring how the new method compares with other interactive image segmentation algorithms shows that our region selection interaction is a viable alternative to marker-based interaction. The hierarchy authoring tool combined with precise algorithms for image segmentation can build contour images of the quality necessary for the images to be understood by touch with our system. The system was evaluated with a user study of 24 sighted participants divided in different groups. The first part of the study had participants explore images using haptics and answer questions about them. The second part of the study asked the participants to identify images visually after haptic exploration. Results show that using a segmentation hierarchy supporting multiple levels-of-detail of the same image is beneficial to haptic exploration. As the system gains maturity, it is our goal to make it available to blind users.