Many companies implemented remote work procedures during the pandemic and for many organizations video meetings have since remained a staple. Remote working has enabled employees to be more flexible with their schedules and technical solutions such as live captioning has been identified as potentially enabling deaf/hard-of-hearing employees during meetings. However with new procedures comes concern about how we potentially can be affected by the changes. Some earlier research has shown that speech intelligibility can be improved by the inclusion of text support, but they also raised the possibility that it could have unwanted adverse effects on cognitive abilities (Zhong, Noud et al., 2022). This study was conducted with this focus, studying the effects of text support on specifically listening effort and comprehension during normal as well as adverse conditions (featuring added noise). To investigate the effects of text support a 2 (Noise, No Noise) x 2 (Text Support, No Text Support) design was used. The participants were shown 16 short videos simulating video meetings and after each video were asked to rate their perceived listening effort as well as a comprehension question about the contents of the discussion. Each of the four conditions were equally represented but the order of the specific video files and conditions that applied were randomised for each participant to mitigate undue effects. The results of the study indicate that the presence of captions decrease effort and raise comprehension in both normal and adverse conditions. Noise was found to strongly effect the listening effort required by participants but no significant effect was found upon comprehension. Some concerns regarding the ecological validity were identified during the course of the study such as only studying energetic noise and unrealistic presentation of captions. However the results are nonetheless believed to be generalizable in most regards and showcase that captions can have a positive influence during video meetings.

Recognizing activities in real-world videos is a difficult problem exacerbated by background clutter, changes in camera angle & zoom, occlusion and rapid camera movements. Large corpora of labeled videos can be used to train automated activity recognition systems, but this requires expensive human labor and time. This thesis explores how closed captions that naturally accompany many videos can act as weak supervision that allows automatically collecting 'labeled' data for activity recognition. We show that such an approach can improve activity retrieval in soccer videos. Our system requires no manual labeling of video clips and needs minimal human supervision. We also present a novel caption classifier that uses additional linguistic information to determine whether a specific comment refers to an ongoing activity. We demonstrate that combining linguistic analysis and automatically trained activity recognizers can significantly improve the precision of video retrieval. / text

Playing and enjoying games might seem like a simple task to most people even though many games rely heavily on sensory, cognitive and motoric ability. To create games that are accessible and inclusive to a wide range of audiences, it is important that design choices are considered carefully. This thesis aims to provide an investigation into closed captioning and enhanced speech bubbles. The goal is to clarify and to compare what effects these features have on the perception of situational and affective information and how they affect the immersion of non-disabled players. Two experiments were performed with a total of 38 participants at the University of Skövde. Each experiment investigated a separate accessibility feature and evaluated their influence on the gameplay experience. By performing two experiments, it was possible to identify the strengths and weaknesses of both features in comparison to each other. The results show that closed captions can have a negative effect on the immersion of the player and that enhanced speech bubbles increase immersion and provide additional affective information to the player.

Information literacy is a relatively new concept in Turkey and is most accessible to English Language Learners. This article-format dissertation identifies the information needs and habits of English Language and Literature majors at a major Turkish university, discusses the development of an online intervention to teach information literacy to these students, and tests the efficacy of using Turkish subtitles to teach information literacy skills to these English majors. Article 1 surveyed students majoring in English Language and Literature about their attitudes towards library usage and sources. Student attitudes revealed a preference for internet sources over library sources and a belief that internet sources are more likely than library sources to provide students with the information they need. In response to the need for increased information literacy instruction, an online tutorial was developed to teach information literacy skills that focus on the Framework for Information Literacy for Higher Education. Article 2 discusses the design and development of this tutorial. Article 3 tests the most effective language configuration for students to learn these information literacy skills, finding that students that completed the tutorial with an English-language soundtrack and Turkish subtitles finished tasks at a statistically significant faster pace than other groups and with statistically significant more success. Overall, Turkish English majors showed a need for increased information literacy skills. A subtitled tutorial is one way to provide this information literacy training.

京都大学 / 0048 / 新制・課程博士 / 博士(人間･環境学) / 甲第20722号 / 人博第831号 / 新制||人||199(附属図書館) / 29||人博||831(吉田南総合図書館) / 京都大学大学院人間・環境学研究科共生人間学専攻 / (主査)教授 田地野 彰, 教授 桂山 康司, 准教授 金丸 敏幸, 教授 植松 茂男 / 学位規則第4条第1項該当 / Doctor of Human and Environmental Studies / Kyoto University / DGAM

Submitted by Josimara Dias Brumatti (bcgdigital@ndc.uff.br) on 2017-04-03T17:21:49Z No. of bitstreams: 1 Tese.pdf: 8046152 bytes, checksum: 4c2296741943286511a163a4bb5ec21b (MD5) / Approved for entry into archive by Josimara Dias Brumatti (bcgdigital@ndc.uff.br) on 2017-04-03T17:35:10Z (GMT) No. of bitstreams: 1 Tese.pdf: 8046152 bytes, checksum: 4c2296741943286511a163a4bb5ec21b (MD5) / Made available in DSpace on 2017-04-03T17:35:10Z (GMT). No. of bitstreams: 1 Tese.pdf: 8046152 bytes, checksum: 4c2296741943286511a163a4bb5ec21b (MD5) / Alinhada à perspectiva dos estudos do discurso, a pesquisa concentra-se na análise de fotografias e legendas em capas de jornais impressos. À luz da Semiolinguística, abordagem de Análise do Discurso que relaciona fatores linguageiros, discursivos e situacionais, o trabalho pretende analisar de que modo o duplo contrato de comunicação da mídia, consubstanciado pelas visadas de informação e de captação, constitui-se na relação entre duas semioses: a verbal e a visual. O corpus compõe-se de pares de fotografias e legendas extraídas de capas de O Globo e da Folha de S. Paulo, contemplando um recorte temático e diacrônico que corresponde a quase um século de existência dos jornais. A fim de atender a esse propósito, parte-se do exame da diagramação das capas, bem como da configuração discursiva mais recorrente nesse espaço, considerando as imagens e suas legendas. Assim, devido à natureza híbrida dos itens investigados, recorre-se, ainda, a estudos sobre o fotojornalismo e à natureza gráfica das imagens. As concepções de imaginário sociodiscursivo e intericonicidade serão também convocadas, uma vez que a visada de captação, a partir do apelo às emoções do leitor pelo discurso, pressupõe a mobilização de representações sociais. Com o enfoque voltado, especificamente, para três áreas temáticas (futebol, movimentos populares e episódios de guerra e terrorismo), tornou-se possível inferir paradigmas específicos de semiotização dos eventos tanto na fotografia e na legenda quanto na integração entre as duas materialidades, aspecto que representa o objeto central de investigação da pesquisa. / Based on discursive studies, this research focuses on the analysis of photographies and their captions in newspapers’ front pages. Through the semiolinguistic perspective, which integrates linguistic, discursive and situational aspects, the study aims at examining the way in which the double media contract – to inform and to catch reader’s attention – is formed by the connection between the verbal and visual text. The data consist of photographies and captions taken from the front pages of O Globo and Folha de S. Paulo, a selection which results from a thematic and diachronic criterium. In order to fulfill this objective, the front page organization is considered, as well as its most recurrent discursive orientation as far as photos and captions are concerned. Due to the hybrid nature of the analyzed data, thus, researches on photojournalism and graphic images are referred to. The conception on social and discursive representations and intericonicity are also explored, since the purpose of catching readers’ attention by the use of discursive-oriented emotions lies on the reference to social imaginary. Resorting to the three thematic areas selected (soccer, popular demonstrations and war and terrorism), it was possible to infer specific paradigms of the semiotization of the events at stake both in photographies and captions, as well as in the integration of the two semiotic materials (visual and verbal), which stands as the main study object of this research.

The purpose of this study was to investigate to what extent bi-modal input improves the word segmentation ability of L2 learners of Japanese. Accurately identifying words in continuous speech is a fundamental process for comprehending the overall message, but studies show that second language (L2) learners often find this task difficult, even when all individual words are familiar to them (e.g. Field, 2003; Goh, 2000). This is where the combination of written and audio input (bi-modal input), like when providing captions in the target language, could be helpful because it can provide orthographical image of the sound they hear, which in turn makes the input more intelligible (Charles & Trenkic, 2015). This study was implemented through a single-case design (SCD), where 12 third-year Japanese learners at a public university in the Midwestern United States underwent a semester-long pre-post design experiment. Participants watched a series of Japanese documentary with sound and captions (bi-modal input) throughout the semester. Before and after viewing each video, participants took Elicited Imitation Tasks (EIT) as the pre-post-tests, as well as at the beginning and at the end of the semester. The result showed that most participants improved their EIT scores throughout the semester, even to utterances from videos and speakers to which they had not been exposed. This study provided evidence that bi-modal input has the potential to help learners’ internal phonological representations of lexical items to become more stable and sophisticated, which would in turn contribute to L2 Japanese learners’ speech processing efficiency.

For my graduation project I was interested in title cards, intertitles and captions from past films and clips. I am fascinated by their typography and art, but also the visual aspects of age and use. Aided by the writings of the artist and filmmaker Hito Steyerl, I explored the world of the poor quality image, an image that has been copied and pasted into different (mainly digital) channels of distribution. I found my own way of translating the visual language of an image with a past. Quoting one of the most retold (copied and pasted) plays; The Tragedy of Hamlet, Prince of Denmark written by William Shakespeare, ca 1600.

本研究之研究對象為「聽障網路使用者」。探討之主題為：一、聽障網路使用者之傳播工具使用偏好概況；二、聽障網路使用者使用網路獲取資訊之情形；三、聽障網路使用者以網路與他人互動溝通之情形；四、網路傳播對聽障者社會參與之影響。 研究設計方面，先以質化的參與觀察及深度訪談法進行前導性研究，再以量化的網頁及書面問卷進行調查研究。最後共回收307份有效樣本，進行統計分析。 綜合質化及量化研究的結果，對目前聽障傳播政策有以下建議： （一）考量不同的聽障特質，選擇適合的傳播方式 （二）電視節目加設字幕 （三）補助聽障者之簡訊及寬頻費用 （四）善用網路服務聽障市民 （五）加強社會教育、宣導「聽障傳播權」概念 （六）落實聽障傳播權法規之執行 關鍵字：聽障、聾、重聽、傳播、網路、手語、社會參與、電視字幕 / This research is to investigate communicational behavior and preference of the people with hearing-impairment, and to investigate how the people with hearing-impairment use Internet to communicate with others and to access information. This research also aims at finding out how the Internet affects social participation of people with hearing-impairment. In order to gather excessive amount of data, participant observation and in-depth interviews were used as pilot study in this research. After pilot study, a sample data about 307 people with hearing-impairment was collected via web survey and general survey. Based on the findings of this research, there are the following suggestions: 1.In order to achieve better communication effect, the government should use different communication channels when communicating to different type of people with hearing-impairment. 2.Add captions to all television programs. 3.Grant people with hearing-impairment subsidy to purchase SMS services from mobile phone companies and to purchase broadband network services. 4.0ffer services via Internet to people with hearing-impairment. 5.Propagate the communication rights of people with hearing-impairment by social education. 6.Enforce correlative rules to assure the communication rights of people with hearing-impairment. Key words: hearing-impairment, deaf, hard of hearing, communication. Internet, sign language, social participation, captions

This project discusses the importance of accurate note-taking for D/deaf and hard of hearing students who have accomodation requirements and offers innovative opportunities to improve the student experience in order to encourage more D/deaf and hard of hearing individuals to persue academia. It also includes a linguistic analysis of speech singals that correspond to transcription output errors produced by speech-to-text programs, which can be utilized to advance and improve speech recognition systems. / In hopes to encourage more D/deaf and hard of hearing (DHH) students to pursue academia, speech-to-text has been suggested to address notetaking issues. This research examined several transcripts created by two untrained speech-to-text programs, Ava and Otter, using 11 different speakers in academic contexts. Observations regarding functionality and error analysis are detailed in this thesis. This project has several objectives, including: 1) to outline how the DHH students’ experience differs from other note-taking needs; 2) to use linguistic analysis to understand how transcript accuracy converts to real-world use and to investigate why errors occur; and 3) to describe what needs to be addressed before assigning DHH students with a captioning service. Results from a focus group showed that current notetaking services are problematic, and that automatic captioning may solve some issues, but some errors are detrimental as it is particularly difficult for DHH students to identify and fix errors within transcripts. Transcripts produced by the programs were difficult to read, as outputs lacked accurate utterance breaks and contained poor punctuation. The captioning of scripted speech was more accurate than that of spontaneous speech for native and most non-native English speakers. An analysis of errors showed that some errors are less severe than others; in response, we offer an alternative way to view errors: as insignificant, obvious, or critical errors. Errors are caused by either the program’s inability to identify various items, such as word breaks, abbreviations, and numbers, or a blend of various speaker factors including: assimilation, vowel approximation, epenthesis, phoneme reduction, and overall intelligibility. Both programs worked best with intelligible speech, as measured by human perception. Speech rate trends were surprising: Otter seemed to prefer fast speech from native English speakers and Ava preferred, as expected, slow speech, but results differed between scripted and spontaneous speech. Correlations of accuracy and fundamental frequencies showed conflicting results. Some reasons for errors could not be determined without knowing more about how the systems were programed. / Thesis / Master of Science (MSc) / In hopes to encourage more D/deaf and hard of hearing (DHH) students to pursue academia, automatic captioning has been suggested to address notetaking issues. Captioning programs use speech recognition (SR) technology to caption lectures in real-time and produce a transcript afterwards. This research examined several transcripts created by two untrained speech-to-text programs, Ava and Otter, using 11 different speakers. Observations regarding functionality and error analysis are detailed in this thesis. The project has several objectives: 1) to outline how the DHH students’ experience differs from other note-taking needs; 2) to use linguistic analysis to understand how transcript accuracy converts to real-world use and to investigate why errors occur; and 3) to describe what needs to be addressed before assigning DHH students with a captioning service. Results from a focus group showed that current notetaking services are problematic, and that automatic captioning may solve some issues, but some types of errors are detrimental as it is particularly difficult for DHH students to identify and fix errors within transcripts. Transcripts produced by the programs were difficult to read, as outputs contain poor punctuation and lack breaks between thoughts. Captioning of scripted speech was more accurate than that of spontaneous speech for native and most non-native English speakers; and an analysis of errors showed that some errors are less severe than others. In response, we offer an alternative way to view errors: as insignificant, obvious, or critical errors. Errors are caused by either the program’s inability to identify various items, such as word breaks, abbreviations, and numbers, or a blend of various speaker factors. Both programs worked best with intelligible speech; One seemed to prefer fast speech from native English speakers and the other preferred slow speech; a preference of male or female voices showed conflicting results. Some reasons for errors could not be determined, as one would have to observe how the systems were programed.