This thesis comprises four separate studies of the same material: a ten-minute Swedish Sign Language monologue. Study I describes the form, meaning, and use of the sign INDEX-c, a pointing toward the chest traditionally described as a first person pronoun. It is argued that INDEX-c is used not only with specific reference to the signer or a quoted signer, but also with non-specific reference. Contrary to what has been reported, INDEX-c is used not only for constructed dialogue, but also in constructed action. The analysis reveals two separate forms, as well, labeled as reduced INDEX-c and distinct INDEX-c, respectively. Study II describes the activities of the non-dominant hand when it is not part of a two-handed sign. A continuum is suggested, moving from different rest positions that do not contribute to the discourse content, via mirroring of the dominant hand, for example, to instances where the non-dominant hand produces signs of its own while the dominant hand remains inactive, i.e. dominance reversal. Several of the activities of the non-dominant hand, including the four types of buoys that are described, help structure the discourse by indicating the current topic. Study III uses Mental Space Theory and Conceptual Blending Theory to describe the use of signing space for reference. A correlation is shown between discourse content and the area in the signing space toward which signs are meaningfully directed, and also between these directions and which types of Real Space blends the signer mainly uses: token blends or surrogate blends. Finally Study IV looks in more detail at three segments of the discourse and their Real Space blend structure. An initial analysis of eight interpretations into spoken Swedish is also conducted, focusing on whether preselected content units (discourse entities and relations) are identified. A large number of Real Space blends and blended entities are argued to result in less successful renditions measured in terms of preselected content units. / För att köpa boken skicka en beställning till exp@ling.su.se/ To order the book send an e-mail to exp@ling.su.se

Sign language is used widely around the world as a first language for those that are unable to use spoken language and by groups of people that have a disability which precludes them from using spoken language (such as a hearing impairment). The importance of effective learning of sign language and its applications in modern computer science has grown widely in the modern aged society and research around sign language recognition has sprouted in many different directions, some examples using hidden markov models (HMMs) to train models to recognize different sign language patterns (Swedish sign language, American sign language, Korean sign language, German sign language and so on).  This thesis project researches the assessment and skill efficiency of using a simple video game to learn Swedish sign language for children in the ages within the range of 10 to 11 with no learning disorders, or any health disorders. During the experimental testing, 38 children are divided into two equally sized groups of 19 where each group plays a sign language video game. The context of the video game is the same for both groups, where both listened to a 3D avatar speak to them using both spoken language and sign language. The first group played the game and answered questions given to them by using sign language, whereas the other group answered questions given to them by clicking on an alternative on the video game screen. A week after the children have played the video game, the sign language skills that they have acquired from playing the video game are assessed by simple questions where they are asked to provide some of the signs that they saw during the duration of the video game. The main hypothesis of the project is that the group of children that answered by signing outperforms the other group, in both remembering the signs and executing them correctly. A statistical null hypothesis test is performed on this hypothesis, in which the main hypothesis is confirmed. Lastly, discussions for future research within sign language assessment using video games is described in the final chapter of the thesis. / Teckenspråk används i stor grad runt om i världen som ett modersmål för dom som inte kan använda vardagligt talsspråk och utav grupper av personer som har en funktionsnedsättning (t.ex. en hörselskada). Betydelsen av effektivt lärande av teckenspråk och dess tillämpningar i modern datavetenskap har ökat i stor utsträckning i det moderna samhället, och forskning kring teckenspråklig igenkänning har spirat i många olika riktningar, ett exempel är med hjälp av statistika modeller såsom dolda markovmodeller (eng. Hidden markov models) för att träna modeller för att känna igen olika teckenspråksmönster (bland dessa ingår Svenskt teckenspråk, Amerikanskt teckenspråk, Koreanskt teckenspråk, Tyskt teckenspråk med flera). Denna rapport undersöker bedömningen och skickligheten av att använda ett enkelt teckenspråksspel som har utvecklats för att lära ut enkla Svenska teckenspråksmönster för barn i åldrarna 10 till 11 års ålder som inte har några inlärningssjukdomar eller några problem med allmän hälsa. Under projektets experiment delas 38 barn upp i två lika stora grupper om 19 i vardera grupp, där varje grupp kommer att få spela ett teckenspråksspel. Sammanhanget för spelet är detsamma för båda grupperna, där de får höra och se en tredimensionell figur (eng. 3D Avatar) tala till dom med både talsspråk och teckenspråk. Den första gruppen spelar spelet och svarar på frågor som ges till dem med hjälp av teckenspråk, medan den andra gruppen svarar på frågor som ges till dem genom att klicka på ett av fem alternativ som finns på spelets skärm. En vecka efter att barnen har utfört experimentet med teckenspråksspelet bedöms deras teckenspråkliga färdigheter som de har fått från spelet genom att de ombeds återuppge några av de tecknena som de såg under spelets varaktighet. Rapportens hypotes är att de barn som tillhör gruppen som fick ge teckenspråk som svar till frågorna som ställdes överträffar den andra gruppen, genom att både komma ihåg tecknena och återuppge dom på korrekt sätt. En statistisk hypotesprövning utförs på denna hypotes, där denna i sin tur bekräftas. Slutligen beskrivs det i rapportens sista kapitel om framtida forskning inom teckenspråksbedömning med tv spel och deras effektivitet.

Thesis (MSc)--University of Stellenbosch, 2007. / ENGLISH ABSTRACT: Machine translation, especially machine translation for sign languages, remains an active research area. Sign language machine translation presents unique challenges to the whole machine translation process. In this thesis a prototype machine translation system is presented. This system is designed to translate English text into a gloss based representation of South African Sign Language (SASL). In order to perform the machine translation, a transfer based approach was taken. English text is parsed into an intermediate representation. Translation rules are then applied to this intermediate representation to transform it into an equivalent intermediate representation for the SASL glosses. For both these intermediate representations, a tree adjoining grammar (TAG) formalism is used. As part of the prototype machine translation system, a TAG parser was implemented. The translation rules used by the system were derived from a SASL phrase book. This phrase book was also used to create a small gloss based SASL TAG grammar. Lastly, some additional tools, for the editing of TAG trees, were also added to the prototype system. / AFRIKAANSE OPSOMMING: Masjienvertaling, veral masjienvertaling vir gebaretale, bly ’n aktiewe navorsingsgebied. Masjienvertaling vir gebaretale bied unieke uitdagings tot die hele masjienvertalingproses. In hierdie tesis bied ons ’n prototipe masjienvertalingstelsel aan. Hierdie stelsel is ontwerp om Engelse teks te vertaal na ’n glos gebaseerde voorstelling van Suid-Afrikaanse Gebaretaal (SAG). Ons vertalingstelsel maak gebruik van ’n oorplasingsbenadering tot masjienvertaling. Engelse teks word ontleed na ’n intermediˆere vorm. Vertalingre¨els word toegepas op hierdie intermediˆere vorm om dit te transformeer na ’n ekwivalente intermediˆere vorm vir die SAG glosse. Vir beide hierdie intermediˆere vorms word boomkoppelingsgrammatikas (BKGs) gebruik. As deel van die prototipe masjienvertalingstelsel, is ’n BKG sintaksontleder ge¨ımplementeer. Die vertalingre¨els wat gebruik word deur die stelsel, is afgelei vanaf ’n SAG fraseboek. Hierdie fraseboek was ook gebruik om ’n klein BKG vir SAG glosse te ontwikkel. Laastens was addisionele nutsfasiliteite, vir die redigering van BKG bome, ontwikkel.

Submitted in fulfillment of the requirements of the degree of Master of Technology: Language Practice, Department of Media, Language and Communication, Durban University of Technology, Durban, South Africa, 2016. / IsiZulu is a previously marginalized language and is spoken by 78% of people in KwaZulu-Natal (South Africa.info 2012). Signed language on the other hand, is not an official language in South Africa, although it is recognised despite the profession having undergone major transformation since democracy. This study hypothesizes that isiZulu and South African Sign Language interpreters both face challenges when given interpreting assignments. This study aims to create an awareness of the needs of the isiZulu speakers and deaf people when seeking judicial assistance and also to contribute towards the provision of quality interpreting services in some of the courts in KwaZulu-Natal. With this study it is hoped to assist the Department of Justice and Constitutional Development to be able to see where they can still improve on their system. This study was conducted only in four courts, therefore, the findings cannot be generalized to be the same in all the courts in South Africa. IsiZulu kusewulimi obelucindezelwe, kanti lusetshenziswa abantu abangamaphesenti angama-78 KwaZulu-Natali (SouthAfrica.info 2012). Ulimi lwezandla ngakolunye uhlangothi, akulona ulimi olusemthethweni eNingizimu Afrika nangale koshintsho oluningi olwenziwe kusukela kwaqala intando yabantu. Lolu cwaningo lucabangela ukuthi otolika besiZulu kanye naboLimi Lwezandla babhekana nezingqinamba uma benikwe umsebenzi wokutolika. Lolu cwaningo Iuhlose ukwazisa ngezidingo zabantu abakhuluma isiZulu kanye nabangezwa uma bedinga usizo lwezomthetho kanye nokuxhasa ekunikezeleni ukutolika okusezingeni elifanele kwezinye zezinkantolo KwaZulu-Natali. Ngalolu cwaningo kuthenjwa ukuthi luzosiza uMnyango Wobulungiswa kanye nokuThuthukiswa koMthethosisekelo ukuba ubone ukuthi yikuphi la okungalungiswa khona inqubo yokwenza yawo. Lolu cwaningo lwenziwe ezinkantolo ezine kuphela, ngakho-ke okutholakele ngeke kuze kuthathwe ngokuthi kuyafana ezinkantolo zonke zaseNingizimu Afrika. / M

Thesis (PhD)--Stellenbosch University, 2014. / ENGLISH ABSTRACT: A sign language tutoring system capable of generating detailed context-sensitive feedback to the user is presented in this dissertation. This stands in contrast with existing sign language tutor systems, which lack the capability of providing such feedback. A domain specific language is used to describe the constraints placed on the user’s movements during the course of a sign, allowing complex constraints to be built through the combination of simpler constraints. This same linguistic description is then used to evaluate the user’s movements, and to generate corrective natural language feedback. The feedback is dynamically tailored to the user’s attempt, and automatically targets that correction which would require the least effort on the part of the user. Furthermore, a procedure is introduced which allows feedback to take the form of a simple to-do list, despite the potential complexity of the logical constraints describing the sign. The system is demonstrated using real video sequences of South African Sign Language signs, exploring the different kinds of advice the system can produce, as well as the accuracy of the comments produced. To provide input for the tutor system, the user wears a pair of coloured gloves, and a video of their attempt is recorded. A vision-based hand pose estimation system is proposed which uses the Earth Mover’s Distance to obtain hand pose estimates from images of the user’s hands. A two-tier search strategy is employed, first obtaining nearest neighbours using a simple, but related, metric. It is demonstrated that the two-tier system’s accuracy approaches that of a global search using only the Earth Mover’s Distance, yet requires only a fraction of the time. The system is shown to outperform a closely related system on a set of 500 real images of gloved hands. / AFRIKAANSE OPSOMMING: ’n Gebaretaaltutorstelsel met die vermo¨e om konteks-sensitiewe terugvoer te lewer aan die gebruiker word uiteengesit in hierdie proefskrif. Hierdie staan in kontras met bestaande tutorstelsels, wat nie hierdie kan bied vir die gebruiker nie. ’n Domein-spesifieke taal word gebruik om beperkinge te definieer op die gebruiker se bewegings deur die loop van ’n gebaar. Komplekse beperkinge kan opgebou word uit eenvoudiger beperkinge. Dieselfde linguistieke beskrywing van die gebaar word gebruik om die gebruiker se bewegings te evalueer, en om korrektiewe terugvoer te genereer in teksvorm. Die terugvoer word dinamies aangepas met betrekking tot die gebruiker se probeerslag, en bepaal outomaties die maklikste manier wat die gebruiker sy/haar fout kan korrigeer. ’n Prosedure word uiteengesit om die terugvoer in ’n eenvoudige lysvorm aan te bied, ongeag die kompleksiteit van die linguistieke beskrywing van die gebaar. Die stelsel word gedemonstreer aan die hand van opnames van gebare uit Suid-Afrikaanse Gebaretaal. Die verskeie tipes terugvoer wat die stelsel kan lewer, asook die akkuraatheid van hierdie terugvoer, word ondersoek. Om vir die tutorstelsel intree te bied, dra die gebruiker ’n stel gekleurde handskoene. ’n Visie-gebaseerde handvormafskattingstelsel wat gebruik maak van die Aardverskuiwersafstand (Earth Mover’s Distance) word voorgestel. ’n Twee-vlak soekstrategie word gebruik. ’n Rowwe afstandsmate word gebruik om ’n stel voorlopige handpostuurkandidate te verkry, waarna die stel verfyn word deur gebruik van die Aardverskuiwersafstand. Dit word gewys dat hierdie benaderde strategie se akkuraatheid grens aan die van eksakte soektogte, maar neem slegs ’n fraksie van die tyd. Toetsing op ’n stel van 500 re¨ele beelde, wys dat hierdie stelsel beter presteer as ’n naverwante stelsel uit die literatuur.

Behavioral skills training (BST) packages have been successful in increasing change agents’ correct implementation of various procedures. The current study evaluated the effects of a brief BST package to train daycare teachers to implement incidental teaching procedures with toddlers. The brief BST consisted of a set of written instructions, a two-minute video model, rehearsal, and feedback during session. Results demonstrated that teachers increased their correct implementation of incidental teaching procedures following training. In addition, two of the three toddlers increased the frequency of signs to request attention.

This study documents and analyses the first established record of taxi hand signs and their respective destinations in South Africa. It demonstrates how taxi hand signification developed into a useful language over time, out of a desperate need for transport amongst black, multi-cultural and multi-lingual people living in South Africa. Its central objective is to recognise taxi hand signs as metaphors for processes of history in pre- and post-apartheid South Africa. This is a study that crosses disciplinary boundaries and marries fine art, anthropology and philosophy in exploring new meanings and understandings of taxi hand signs. In this way, it demonstrates the extent to which art informs other disciplines in extraordinary ways, adding to the value of inter-disciplinary research. The research indicates that taxi signs are part of an evolving, well-functioning, gestural language for sighted commuters. It goes further to probe the question of how blind commuters might have access to the signs, thereby enhancing their independence and movement. The study responds to this question through the design of a new, tactile shape-language of taxi hand signs for blind people. Qualitative research techniques were employed throughout the three phases of the research, namely: preliminary research, research design, and social and fine art responses. The methodologies utilised in the phases were sampling, semistructured interviews and participant observation. These were each employed at specific times to meet specific needs of different phases. I, along with some coresearchers, applied these in taxi ranks, taxi associations and on the streets of Gauteng. The methods used attest to the fact that when new knowledge was sought with key informants in the taxi industry, the different methodologies could be used to verify and corroborate the informants’ information, which in turn become the keystones of knowledge distribution in the thesis. With limited documentation on the emergence of taxi hand signs in the industry, the informants furnished unexplored background information, which I have interpreted in my artworks, films, books, stamps, maps and the blind shape-language. 16 The anthropological research also probed the function of signification through literary criticism. This involved an investigation of the components of the process of signification into its constituent parts in order to conceptualise and contextualise taxi hand signing and its particular relations and narrative content within the greater field of gestural signification. The response of art and artists to anthropological, historical and current approaches was also explored, again to provide context to my art that evolved out of the research. These involved conceptual and graphic art interpretations probing movement, time, space and signification, which led to an art exhibition at the Wits Art Museum (henceforth referred to as WAM) from 12 June to 14 July 2013. Taxi hand signs are continually evolving as new destinations and narratives arise. Together with the art responses document, this thesis records and promotes the established body of the current taxi hand signs, destinations and narratives, for both sighted and blind people, by providing written, visual and sensory evidence of a cultural phenomenon that was previously uncharted.

No description available.

O reconhecimento de língua de sinais é uma importante área de pesquisa que tem como objetivo atenuar os obstáculos impostos no dia a dia das pessoas surdas e/ou com deficiência auditiva e aumentar a integração destas pessoas na sociedade majoritariamente ouvinte em que vivemos. Baseado nisso, esta dissertação de mestrado propõe o desenvolvimento de um sistema de informação para o reconhecimento automático de Língua Brasileira de Sinais (LIBRAS), que tem como objetivo simplificar a comunicação entre surdos conversando em LIBRAS e ouvintes que não conheçam esta língua de sinais. O reconhecimento é realizado por meio do processamento de sequências de imagens digitais (vídeos) de pessoas se comunicando em LIBRAS, sem o uso de luvas coloridas e/ou luvas de dados e sensores ou a exigência de gravações de alta qualidade em laboratórios com ambientes controlados, focando em sinais que utilizam apenas as mãos. Dada a grande dificuldade de criação de um sistema com este propósito, foi utilizada uma abordagem para o seu desenvolvimento por meio da divisão em etapas. Considera-se que todas as etapas do sistema proposto são contribuições para trabalhos futuros da área de reconhecimento de sinais, além de poderem contribuir para outros tipos de trabalhos que envolvam processamento de imagens, segmentação de pele humana, rastreamento de objetos, entre outros. Para atingir o objetivo proposto foram desenvolvidas uma ferramenta para segmentar sequências de imagens relacionadas à LIBRAS e uma ferramenta para identificar sinais dinâmicos nas sequências de imagens relacionadas à LIBRAS e traduzi-los para o português. Além disso, também foi construído um banco de imagens de 30 palavras básicas escolhidas por uma especialista em LIBRAS, sem a utilização de luvas coloridas, laboratórios com ambientes controlados e/ou imposição de exigências na vestimenta dos indivíduos que executaram os sinais. O segmentador implementado e utilizado neste trabalho atingiu uma taxa média de acurácia de 99,02% e um índice overlap de 0,61, a partir de um conjunto de 180 frames pré-processados extraídos de 18 vídeos gravados para a construção do banco de imagens. O algoritmo foi capaz de segmentar pouco mais de 70% das amostras. Quanto à acurácia para o reconhecimento das palavras, o sistema proposto atingiu 100% de acerto para reconhecer as 422 amostras de palavras do banco de imagens construído, as quais foram segmentadas a partir da combinação da técnica de distância de edição e um esquema de votação com um classificador binário para realizar o reconhecimento, atingindo assim, o objetivo proposto neste trabalho com êxito. / The recognition of sign language is an important research area that aims to mitigate the obstacles in the daily lives of people who are deaf and/or hard of hearing and increase their integration in the majority hearing society in which we live. Based on this, this dissertation proposes the development of an information system for automatic recognition of Brazilian Sign Language (BSL), which aims to simplify the communication between deaf talking in BSL and listeners who do not know this sign language. The recognition is accomplished through the processing of digital image sequences (videos) of people communicating in BSL without the use of colored gloves and/or data gloves and sensors or the requirement of high quality recordings in laboratories with controlled environments focusing on signals using only the hands. Given the great difficulty of setting up a system for this purpose, an approach divided in several stages was used. It considers that all stages of the proposed system are contributions for future works of sign recognition area, and can contribute to other types of works involving image processing, human skin segmentation, object tracking, among others. To achieve this purpose we developed a tool to segment sequences of images related to BSL and a tool for identifying dynamic signals in the sequences of images related to the BSL and translate them into portuguese. Moreover, it was also built an image bank of 30 basic words chosen by a BSL expert without the use of colored gloves, laboratory-controlled environments and/or making of the dress of individuals who performed the signs. The segmentation algorithm implemented and used in this study had a average accuracy rate of 99.02% and an overlap of 0.61, from a set of 180 preprocessed frames extracted from 18 videos recorded for the construction of database. The segmentation algorithm was able to target more than 70% of the samples. Regarding the accuracy for recognizing words, the proposed system reached 100% accuracy to recognize the 422 samples from the database constructed (the ones that were segmented), using a combination of the edit distance technique and a voting scheme with a binary classifier to carry out the recognition, thus reaching the purpose proposed in this work successfully.

O reconhecimento de expressões faciais é uma área de interesse da ciência da computação e tem sido um atrativo para pesquisadores de diferentes áreas, pois tem potencial para promover o desenvolvimento de diferentes tipos de aplicações. Reconhecer automaticamente essas expressões tem se tornado um objetivo, principalmente na área de análise do comportamento humano. Especialmente para estudo das línguas de sinais, a análise das expressões faciais é importante para a interpretação do discurso, pois é o elemento que permite expressar informação prosódica, suporta o desenvolvimento da estrutura gramatical e semântica da língua, e ajuda na formação de sinais com outros elementos básicos da língua. Nesse contexto, as expressões faciais são chamadas de expressões faciais gramaticais e colaboram na composição no sentido semântico das sentenças. Entre as linhas de estudo que exploram essa temática, está aquela que pretende implementar a análise automática da língua de sinais. Para aplicações com objetivo de interpretar línguas de sinais de forma automatizada, é preciso que tais expressões sejam identificadas no curso de uma sinalização, e essa tarefa dá-se é definida como segmentação de expressões faciais gramaticais. Para essa área, faz-se útil o desenvolvimento de uma arquitetura capaz de realizar a identificação de tais expressões em uma sentença, segmentando-a de acordo com cada tipo diferente de expressão usada em sua construção. Dada a necessidade do desenvolvimento dessa arquitetura, esta pesquisa apresenta: uma análise dos estudos na área para levantar o estado da arte; a implementação de algoritmos de reconhecimento de padrões usando Multilayer Perceptron e misturas de especialistas para a resolução do problema de reconhecimento da expressão facial; a comparação desses algoritmos reconhecedores das expressões faciais gramaticais usadas na concepção de sentenças na Língua Brasileira de Sinais (Libras). A implementação e teste dos algoritmos mostraram que a segmentação automática de expressões faciais gramaticais é viável em contextos dependentes do usuários. Para contextos independentes de usuários, o problema de segmentação de expressões faciais representa um desafio que requer, principalmente, a organização de um ambiente de aprendizado estruturado sobre um conjunto de dados com volume e diversidade maior do que os atualmente disponíveis / The recognition of facial expressions is an area of interest in computer science and has been an attraction for researchers in different fields since it has potential for development of different types of applications. Automatically recognizing these expressions has become a goal primarily in the area of human behavior analysis. Especially for the study of sign languages, the analysis of facial expressions represents an important factor for the interpretation of discourse, since it is the element that allows expressing prosodic information, supports the development of the grammatical and semantic structure of the language, and eliminates ambiguities between similar signs. In this context, facial expressions are called grammatical facial expressions. These expressions collaborate in the semantic composition of the sentences. Among the lines of study that explore this theme is the one that intends to implement the automatic analysis of sign language. For applications aiming to interpret signal languages in an automated way, it is necessary that such expressions be identified in the course of a signaling, and that task is called \"segmentation of grammatical facial expressions\'\'. For this area, it is useful to develop an architecture capable of performing the identification of such expressions in a sentence, segmenting it according to each different type of expression used in its construction. Given the need to develop this architecture, this research presents: a review of studies already carried out in the area; the implementation of pattern recognition algorithms using Multilayer Perceptron and mixtures of experts to solve the facial expression recognition problem; the comparison of these algorithms as recognizers of grammatical facial expressions used in the conception of sentences in the Brazilian Language of Signs (Libras). The implementation and tests carried out with such algorithms showed that the automatic segmentation of grammatical facial expressions is practicable in user-dependent contexts. Regarding user-independent contexts, this is a challenge which demands the organization of a learning environment structured on datasets bigger and more diversified than those current available