Published Article / This paper aims to describe the state of South African Sign Language (SASL) at the University of the Free State (UFS). It provides background to the development of SASL, the development of Deaf education, the teaching of sign language and sign language interpreting at the UFS, research and community development. SASL is one of the languages offered at the UFS. The University has been involved with ground-breaking research to implement a training course on all levels of tertiary education. Students can complete a degree in sign language and also do postgraduate studies in sign language and sign language interpreting. This article aims to indicate how teaching of SASLas an official language empowers marginalised culture.

>Magister Scientiae - MSc / This thesis describes a multi-disciplinary collaboration towards iterative development of a mobile communication tool to support a Deaf person in understanding usage directions for medication dispensed at a pharmacy. We are improving usability and correctness of the user interface. The tool translates medicine instruction given in English text to South African Sign Language videos, which are relayed to a Deaf user on a mobile phone. Communication between pharmacists and Deaf patients were studied to extract relevant exchanges between the two users. We incorporated the common elements of these dialogues to represent content in a veri able manner to ensure that the mobile tool relays the correct information to the Deaf user. Instructions are made available for a Deaf patient in sign language videos on a mobile device. A pharmacy setup was created to conduct trials of the tool with groups of end users, in order to collect usability data with recorded participant observation, questionnaires and focus group discussions. Subsequently, pre-recorded sign language videos, stored on a phone's memory card, were tested for correctness. Lastly we discuss the results and implications of the study and provide a conclusion to our research.

>Magister Scientiae - MSc / In order to classify South African Sign Language as a signed gesture, five fundamental parameters need to be considered. These five parameters to be considered are: hand shape, hand orientation, hand motion, hand location and facial expressions. The research in this thesis will utilise Deep Learning techniques, specifically Convolutional Neural Networks, to recognise hand shapes in various hand orientations. The research will focus on two of the five fundamental parameters, i.e., recognising six South African Sign Language hand shapes for each of five different hand orientations. These hand shape and orientation combinations will be recognised by means of a video stream captured on a mobile device. The efficacy of Convolutional Neural Network for gesture recognition will be judged with respect to its classification accuracy and classification speed in both a desktop and embedded context. The research methodology employed to carry out the research was Design Science Research. Design Science Research refers to a set of analytical techniques and perspectives for performing research in the field of Information Systems and Computer Science. Design Science Research necessitates the design of an artefact and the analysis thereof in order to better understand its behaviour in the context of Information Systems or Computer Science. / National Research Foundation (NRF)

Philosophiae Doctor - PhD / Hand motion provides a natural way of interaction that allows humans to interact not only with the environment, but also with each other. The effectiveness and accuracy of hand-tracking is fundamental to the recognition of sign language. Any inconsistencies in hand-tracking result in a breakdown in sign language communication. Hands are articulated objects, which complicates the tracking thereof. In sign language communication the tracking of hands is often challenged by the occlusion of the other hand, other body parts and the environment in which they are being tracked. The thesis investigates whether a single framework can be developed to track the hands independently of an individual from a single 2D camera in constrained and unconstrained environments without the need for any special device. The framework consists of a three-phase strategy, namely, detection, tracking and learning phases. The detection phase validates whether the object being tracked is a hand, using extended local binary patterns and random forests. The tracking phase tracks the hands independently by extending a novel data-association technique. The learning phase exploits contextual features, using the scale-invariant features transform (SIFT) algorithm and the fast library for approximate nearest neighbours (FLANN) algorithm to assist tracking and the recovering of hands from any form of tracking failure. The framework was evaluated on South African sign language phrases that use a single hand, both hands without occlusion, and both hands with occlusion. These phrases were performed by 20 individuals in constrained and unconstrained environments. The experiments revealed that integrating all three phases to form a single framework is suitable for tracking hands in both constrained and unconstrained environments, where a high average accuracy of 82,08% and 79,83% was achieved respectively.

Thesis (MA)--Stellenbosch University, 2015. / ENGLISH ABSTRACT: Reading skills of Deaf schoolchildren in the United States of America are reported to be significantly below those of their hearing peers. In South Africa, Deaf learners’ prospects of attaining high levels of literacy are even bleaker. This fact gave rise to the current study, which examines the impact of a newly introduced Curriculum Assessment Policy Statements (CAPS) curriculum with South African Sign Language (SASL) both as a school subject and as language of learning and teaching on (i) the course of language development in SASL and (ii) early reading development in second language (L2) Afrikaans amongst young Deaf learners. The expressive language (SASL) of four participants with various home language backgrounds (some with signing, Deaf parents; others with hearing, non-signing Afrikaans- or English-speaking parents) was regularly video-recorded over a period of three years, from the beginning of their pre-Grade R (i.e., Grade 0) year until the end of their Grade 1 year. These learners all started reading at the beginning of Grade 1, but were exposed to SASL for varying periods prior to that. The language development (in terms of handshape and discourse skills) and literacy development (reading comprehension) of the participants were qualitatively analysed based on the recordings. Despite the diverse circumstances of the four participants in terms of date of diagnosis, length of exposure to SASL and age of first exposure, parental mastery of SASL, and length of attendance of the school in which the SASL curriculum was piloted, they all showed notable progress in terms of SASL acquisition over the course of the study period, and they all managed to acquire reading skills up to the level expected of a Grade 1 learner. As was found for hearing users of a spoken language, language competence appears to be a prerequisite for the development of literacy skills in Deaf children (see Adams 1990; Gathercole and Baddeley 1993). The assumption is that the four children studied here were successful readers in their L2 (despite Deaf children usually finding the acquisition of reading skills very challenging) because they had sufficiently developed first language (SASL) skills as a foundation for literacy acquisition. The participants’ phonological, morphological, syntactic, semantic and pragmatic knowledge of SASL appeared to have “come together” by the time they reached Grade 1, although these were not necessarily taught explicitly. This, again, enabled the acquisition of grade-appropriate reading skills in the L2. The findings of this study indicate the potential benefit of early intervention for Deaf children and that the SASL CAPS curriculum ought to be introduced at the time of enrolment in a school for the Deaf (i.e. from age three years onwards), and not only in Grade R. / AFRIKAANSE OPSOMMING: Leesvaardighede van dowe skoolkinders in die Verenigde State van Amerika is na berigte beduidend laer as dié van hul horende eweknieë. In Suid-Afrika is dowe leerders se kanse om hoë geletterdheidsvlakke te bereik selfs skraler. Hierdie feit het aanleiding gegee tot die huidige studie, wat die impak ondersoek van ‘n nuut-bekendgestelde Kurrikulum-assesseringsbeleidstellings- (KABS) kurrikulum met Suid-Afrikaanse Gebaretaal (SAGT) as beide skoolvak en taal van leer en onderrig op (i) die verloop van taalontwikkeling in SAGT en (ii) vroeë leesontwikkeling in tweedetaal- (T2) Afrikaans onder jong Dowe leerders. Daar is gereeld oor die verloop van drie jaar video-opnames gemaak van die ekspressiewe taal (SAGT) van vier deelnemers met diverse huistaalagtergronde (sommige met Dowe gebaretaalgebruikers as ouers; andere met horende Afrikaans- of Engelssprekende ouers wat geen gebaretaal ken nie), van die begin van hul pre-Graad R- (i.e., Graad 0-) jaar tot die einde van hul Graad 1-jaar. Hierdie leerders het almal begin lees aan die begin van Graad 1, maar is voor dit vir verskillende periodes aan SAGT blootgestel. Hul taalontwikkeling (in terme van handvorm en diskoersvaardighede) en geletterdheidsontwikkeling (leesbegrip) is aan die hand van die opnames kwalitatief geanaliseer. Ondanks die diverse omstandighede van die vier deelnemers in terme van diagnosedatum, lengte van blootstelling aan SAGT en ouderdom van eerste blootstelling, ouers se bemeestering van SAGT, en lengte van bywoning van die skool waarin die SAGT-kurrikulum geloods is, het hulle almal noemenswaardige vordering getoon in terme van SAGT-verwerwing oor die verloop van die studietydperk, en het hulle almal daarin geslaag om leesvaardighede te verwerf tot op die vlak wat van ‘n Graad 1-leerder verwag word. Soos gevind is in die geval van horende gebruikers van ‘n gesproke taal, blyk taalkompetensie ‘n voorvereiste te wees vir die ontwikkeling van geletterdheidsvaardighede in dowe kinders (kyk Adams 1990; Gathercole en Baddeley 1993). Die aanname is dat die vier bestudeerde kinders suksesvolle leerders in hul T2 was (ondanks die feit dat die verwerwing van leesvaardighede gewoonlik vir dowe kinders ‘n groot uitdaging is) omdat hulle eerstetaalvaardighede (in SAGT) voldoende ontwikkel was om as fondament vir geletterdheidsverwerwing te dien. Die deelnemers se fonologiese, morfologiese, sintaktiese, semantiese en pragmatiese kennis van SAGT blyk “byeen te gekom het” teen die tyd wat hulle Graad 1 bereik het, alhoewel daar nie noodwendig eksplisiete onderrig hieroor gegee is nie. en dit het die verwerwing van graadtoepaslike leesvaardighede moontlik gemaak. Die bevindinge dui die belang van vroeë intervensie vir die Dowe kind aan. Die bevindinge ondersteun ook die seining dat die SAGT KABS-kurrikulum vanaf toetrede tot ‘n skool vir Dowes gevolg behoort te word (d.i. vanaf ouderdom drie jaar), en nie eers vanaf Graad R nie.

Doctor Educationis / Research has shown that five parameters are required to recognize any sign language gesture: hand shape, location, orientation and motion, as well as facial expressions. The South African Sign Language (SASL) research group at the University of the Western Cape has created systems to recognize Sign Language gestures using single parameters. Using a single parameter can cause ambiguities in the recognition of signs that are similarly signed resulting in a restriction of the possible vocabulary size. This research pioneers work at the group towards combining multiple parameters to achieve a larger recognition vocabulary set. The proposed methodology combines hand location and hand shape recognition into one combined recognition system. The system is shown to be able to recognize a very large vocabulary of 50 signs at a high average accuracy of 74.1%. This vocabulary size is much larger than existing SASL recognition systems, and achieves a higher accuracy than these systems in spite of the large vocabulary. It is also shown that the system is highly robust to variations in test subjects such as skin colour, gender and body dimension. Furthermore, the group pioneers research towards continuously recognizing signs from a video stream, whereas existing systems recognized a single sign at a time. To this end, a highly accurate continuous gesture segmentation strategy is proposed and shown to be able to accurately recognize sentences consisting of five isolated SASL gestures.

>Magister Scientiae - MSc / The South African Sign Language research group at the University of the Western Cape is in the process of creating a fully-edged machine translation system to automatically translate between South African Sign Language and English. A major component of the system is the ability to accurately recognise facial expressions, which are used to convey emphasis, tone and mood within South African Sign Language sentences. Traditionally, facial expression recognition research has taken one of two paths: either recognising whole facial expressions of which there are six i.e. anger, disgust, fear, happiness, sadness, surprise, as well as the neutral expression; or recognising the fundamental components of facial expressions as defined by the Facial Action Coding System in the form of Action Units. Action Units are directly related to the motion of specific muscles in the face, combinations of which are used to form any facial expression. This research investigates enhanced recognition of whole facial expressions by means of a hybrid approach that combines traditional whole facial expression recognition with Action Unit recognition to achieve an enhanced classification approach.

Doctor Pharmaceuticae - DPharm / South Africa's White Paper for the transformation of the health care system in South Africa (DOH, 2007) acknowledges major disparities and inequalities as a result of an imprint by apartheid policies. In its transition to democracy, health promotion strategies have been initiated to address these disparities. However, such strategies have been narrowed and "favoured target audiences that are literate, urban-based and who have easy access to print and audio-visual media" (DOH, 1997). This implies that many vulnerable and marginalised groupings in South Africa, including the Deaf community are excluded from health promotion endeavours. Deaf people in South Africa communicate using South African Sign Language (SASL) and majority of the Deaf community exhibit poor literacy levels. Deafness is a significant communication barrier which limits a Deaf person's prospect to attain the best possible health care (Barnett, et al 2011). Various means of communication including spoken language, written instructions and the use of pictograms are used by healthcare workers to communicate health-related information. For many members of the Deaf community who communicate primarily in sign language, these methods are a sub-standard and prevent the attainment of optimum therapeutic outcomes. With regard to pharmaco-therapeutic services, Deaf people cannot hear the spoken language used by pharmacists during patient counselling, and their compromised functional literacy hinders the ability to read instructions on medicine labels. With both the spoken and written means of communication compromised, the Deaf patient's ability to comprehend instruction by pharmacists on how to use their medicines is inadequate and as a result, a Deaf patient may leave the pharmacy with medicine, but a poor understanding of how to use the medicine safely and effectively. Previous researchers have worked on building a technology base, including industrial design and computer science expertise to conceptualize the groundwork of a mobile phone application called SignSupport to facilitate communication between medical doctors and Deaf individuals. The particulars of the pharmacy scenario however, require a pharmacy-specific device to be of use in the dispensing of medicines to a Deaf patient in a pharmacy. The over-arching goal of this thesis is to design and evaluate a mobile phone application to facilitate the communication of medicine instructions between a Deaf patient and a pharmacist. Qualitative, participatory action research and community-based co-design strategies were directed toward Deaf participants, senior pharmacy students and pharmacists to create a prototype of the afore-mentioned mobile phone application. Preliminary results indicated that the application was suitable to pharmacists and Deaf community. Furthermore, both sets of users approved the overall design and were receptive to and keen on the practical uses of the application. Inadequacies pointed out by the Deaf community and pharmacists were addressed as an iterative modification to the prototype and culminated in version 2 which was deployed in an actual hospital pharmacy in 2015. Hospital usability studies generated largely positive results from both Deaf users and pharmacists, indicating that SignSupport is able to facilitate communication between pharmacists and Deaf patients. Next steps include advancing the application to a market–ready version that is downloadable and available as an application on the play stores of commercially available smart phones. / National Research Foundation

Masters of Science / Much work has been done in building systems that can recognise gestures, e.g. as a component of sign language recognition systems. These systems typically use whole gestures as the smallest unit for recognition. Although high recognition rates have been reported, these systems do not scale well and are computationally intensive. The reason why these systems generally scale poorly is that they recognize gestures by building individual models for each separate gesture; as the number of gestures grows, so does the required number of models. Beyond a certain threshold number of gestures to be recognized, this approach becomes infeasible. This work proposes that similarly good recognition rates can be achieved by building models for subcomponents of whole gestures, so-called cheremes. Instead of building models for entire gestures, we build models for cheremes and recognize gestures as sequences of such cheremes. The assumption is that many gestures share cheremes and that the number of cheremes necessary to describe gestures is much smaller than the number of gestures. This small number of cheremes then makes it possible to recognize a large number of gestures with a small number of chereme models. This approach is akin to phoneme-based speech recognition systems where utterances are recognized as phonemes which in turn are combined into words. We attempt to recognise and classify cheremes found in South African Sign Language (SASL). We introduce a method for the automatic discovery of cheremes in dynamic signs. We design, train and use hidden Markov models (HMMs) for chereme recognition. Our results show that this approach is feasible in that it not only scales well, but it also generalizes well. We are able to recognize cheremes in signs that were not used for training HMMs; this generalization ability is a basic necessity for chemere-based gesture recognition. Our approach can thus lay the foundation for building a SASL dynamic gesture recognition system.

Research conducted on the prevalence of behavioural adjustment in Deaf children and adolescents, in erstwhile countries, points towards an appreciably elevated percentage of emotional and behavioural problems amongst this population group when compared to hearing normative groups. Studies specify that the prevalence of behaviour and emotional problems in Deaf children and adolescents varies from 4.8% to 50.3%. From existing research conducted, it is ambiguous as to why the reported prevalence rates of maladjustment are higher amongst Deaf children and adolescents. This pioneering study is the first of its kind to research dissimilar modalities of language used as the language of learning and teaching (LoLT) in schools for Deaf learners and how this could possibly correlate to learner behaviour in the classroom. Taking into consideration the reported pervasiveness of maladjustment in Deaf children and adolescents; this study uses the Teacher Report Form (TRF) to investigate the types of behaviour problems displayed by Deaf learners in the classroom. It further investigates whether Deaf learners display certain types of behaviour problems when dissimilar modalities of language are used as the language of learning and teaching. The overall findings of this study suggest that teachers who use manually coded spoken language report an elevated prevalence of behaviour problems on the TRF compared to teachers who use South African Sign Language (SASL). Results further suggest that the group of teachers who use SASL report somatic complaints and attention problems as the most frequently encountered behaviour problems in their classrooms. In comparison the group of teachers who use manually coded spoken English (MCE) report social problems and attention problems as the most frequently encountered behaviour problems in their classrooms. Limitations of this study and suggestions for future research are discussed.