Recognizing activities in real-world videos is a difficult problem exacerbated by background clutter, changes in camera angle & zoom, occlusion and rapid camera movements. Large corpora of labeled videos can be used to train automated activity recognition systems, but this requires expensive human labor and time. This thesis explores how closed captions that naturally accompany many videos can act as weak supervision that allows automatically collecting 'labeled' data for activity recognition. We show that such an approach can improve activity retrieval in soccer videos. Our system requires no manual labeling of video clips and needs minimal human supervision. We also present a novel caption classifier that uses additional linguistic information to determine whether a specific comment refers to an ongoing activity. We demonstrate that combining linguistic analysis and automatically trained activity recognizers can significantly improve the precision of video retrieval. / text

Playing and enjoying games might seem like a simple task to most people even though many games rely heavily on sensory, cognitive and motoric ability. To create games that are accessible and inclusive to a wide range of audiences, it is important that design choices are considered carefully. This thesis aims to provide an investigation into closed captioning and enhanced speech bubbles. The goal is to clarify and to compare what effects these features have on the perception of situational and affective information and how they affect the immersion of non-disabled players. Two experiments were performed with a total of 38 participants at the University of Skövde. Each experiment investigated a separate accessibility feature and evaluated their influence on the gameplay experience. By performing two experiments, it was possible to identify the strengths and weaknesses of both features in comparison to each other. The results show that closed captions can have a negative effect on the immersion of the player and that enhanced speech bubbles increase immersion and provide additional affective information to the player.

Many companies implemented remote work procedures during the pandemic and for many organizations video meetings have since remained a staple. Remote working has enabled employees to be more flexible with their schedules and technical solutions such as live captioning has been identified as potentially enabling deaf/hard-of-hearing employees during meetings. However with new procedures comes concern about how we potentially can be affected by the changes. Some earlier research has shown that speech intelligibility can be improved by the inclusion of text support, but they also raised the possibility that it could have unwanted adverse effects on cognitive abilities (Zhong, Noud et al., 2022). This study was conducted with this focus, studying the effects of text support on specifically listening effort and comprehension during normal as well as adverse conditions (featuring added noise). To investigate the effects of text support a 2 (Noise, No Noise) x 2 (Text Support, No Text Support) design was used. The participants were shown 16 short videos simulating video meetings and after each video were asked to rate their perceived listening effort as well as a comprehension question about the contents of the discussion. Each of the four conditions were equally represented but the order of the specific video files and conditions that applied were randomised for each participant to mitigate undue effects. The results of the study indicate that the presence of captions decrease effort and raise comprehension in both normal and adverse conditions. Noise was found to strongly effect the listening effort required by participants but no significant effect was found upon comprehension. Some concerns regarding the ecological validity were identified during the course of the study such as only studying energetic noise and unrealistic presentation of captions. However the results are nonetheless believed to be generalizable in most regards and showcase that captions can have a positive influence during video meetings.