There are many challenges facing society today, both socially and industrially. Whether it is to improve productivity in factories or with the intention of improving the quality of life of people in their homes, technological advances in robotics and computing have led to solutions to many problems in modern society. These areas are of great interest and are in constant development, especially in societies with a relatively ageing population. In this thesis, we address different challenges in which robotics, artificial intelligence and computer vision are used as tools to propose solutions oriented to home assistance. These tools can be organised into three main groups: “Grasping Challenges”, where we have addressed the problem of performing robot grasping in domestic environments; “Hand Interaction Challenges”, where we have addressed the detection of static and dynamic hand gestures, using approaches based on DeepLearning and GeometricLearning; and finally, “Human Behaviour Recognition”, where using a machine learning model based on hyperbolic geometry, we seek to group the actions that performed in a video sequence.

This thesis explores the use of head gesture recognition as an intuitive interface for computer interaction. This research presents a novel vision-based head gesture recognition system which utilizes contextual clues to reduce false positives. The system is used as a computer interface for answering dialog boxes. This work seeks to validate similar research, but focuses on using more efficient techniques using everyday hardware. A survey of image processing techniques for recognizing and tracking facial features is presented along with a comparison of several methods for tracking and identifying gestures over time. The design explains an efficient reusable head gesture recognition system using efficient lightweight algorithms to minimize resource utilization. The research conducted consists of a comparison between the base gesture recognition system and an optimized system that uses contextual clues to reduce false positives. The results confirm that simple contextual clues can lead to a significant reduction of false positives. The head gesture recognition system achieves an overall accuracy of 96% using contextual clues and significantly reduces false positives. In addition, the results from a usability study are presented showing that head gesture recognition is considered an intuitive interface and desirable above conventional input for answering dialog boxes. By providing the detailed design and architecture of a head gesture recognition system using efficient techniques and simple hardware, this thesis demonstrates the feasibility of implementing head gesture recognition as an intuitive form of interaction using preexisting infrastructure, and also provides evidence that such a system is desirable.

<p> The use of smart devices such as smartphones, tablets, and laptops skyrocketed in the last decade. These devices enable ubiquitous applications for entertainment, communication, productivity, and healthcare but also introduce big concern about user privacy and data security. In addition to various authentication techniques, automatic and immediate device locking based on user leaving detection is an indispensable way to secure the devices. Current user leaving detection techniques mainly rely on acoustic ranging and do not work well in environments with multiple moving objects. In this paper, we present mmLock, a system that enables faster and more accurate user leaving detection in dynamic environments. mmLock uses a mmWave FMCW radar to capture the user’s 3D mesh and detects the leaving gesture from the 3D human mesh data with a hybrid PointNet-LSTM model. Based on explainable user point clouds, mmLock is more robust than existing gesture recognition systems which can only identify the raw signal patterns. We implement and evaluate mmLock with a commercial off-the-shelf (COTS) TI mmWave radar in multiple environments and scenarios. We train the PointNet-LSTM model out of over 1 TB mmWave signal data and achieve 100% true-positive rate in most scenarios. </p>

Despite a rise in the use of “learning by doing” pedagogical methods in praxis, little is known as to how the brain benefits from these methods. Learning by doing strategies that utilize complementary information (“enrichment”) such as gestures have been shown to optimize learning outcomes in several domains including foreign language (L2) training. Here we tested the hypothesis that behavioral benefits of gesture-based enrichment are critically supported by integrity of the biological motion visual cortices (bmSTS). Prior functional neuroimaging work has implicated the visual motion cortices in L2 translation following sensorimotor-enriched training; the current study is the first to investigate the causal relevance of these structures in learning by doing contexts. Using neuronavigated transcranial magnetic stimulation and a gesture-enriched L2 vocabulary learning paradigm, we found that the bmSTS causally contributed to behavioral benefits of gesture-enriched learning. Visual motion cortex integrity benefitted both short- and long-term learning outcomes, as well as the learning of concrete and abstract words. These results adjudicate between opposing predictions of two neuroscientific learning theories: While reactivation-based theories predict no functional role of specialized sensory cortices in vocabulary learning outcomes, the current study supports the predictive coding theory view that these cortices precipitate sensorimotor-based learning benefits.:I. Abkürzungsverzeichnis II. Abbildungsverzeichnis III. Einleitung 1. Fremdsprachenlernen 1.1 Sensorische Modalitätsvergleiche 1.2 Sensomotorisches Lernen 2. Lerntheorien 2.1 Theorie des prädiktiven Kodierens 2.2 Theorie des prädiktiven Kodierens für multisensorisches Lernen 3. Sulcus temporalis superior für biologische Bewegung 4. Transkranielle Magnetstimulation 4.1 Passagere Funktionsinhibition mittels transkranieller Magnetstimulation IV. Ableitung der Rationale V. Publikationsmanuskript VI. Zusammenfassung VII. Literaturverzeichnis VIII. Appendix A. Abbildungen B. Ergänzendes Material der Publikation C. Darstellung des eigenen Beitrags D. Erklärung über die eigenständige Abfassung der Arbeit E. Lebenslauf F. Publikationen G. Danksagung

No description available.

No description available.

No description available.

No description available.

No description available.

Science, Technology, Engineering, and Mathematics (STEM) experts employ many representations that novices find hard to use because they require a critical STEM skill, interpreting two-dimensional (2D) diagrams that represent three-dimensional (3D) information. The current research focuses on learning to interpret topographic maps. Understanding topographic maps requires knowledge of how to interpret the conventions of contour lines, and skill in visualizing that information in 3D (e.g. shape of the terrain). Novices find both tasks difficult. The present study compared two interventions designed to facilitate understanding for topographic maps to minimal text-only instruction. The 3D Visualization group received instruction using 3D gestures and models to help visualize three topographic forms. The Pattern Identification group received instruction using pointing and tracing gestures to help identify the contour patterns associated with the three topographic forms. The Text-based Instruction group received only written instruction explaining topographic maps. All participants then completed a measure of topographic map use. The Pattern Identification group performed better on the map use measure than participants in the Text-based Instruction group, but no significant difference was found between the 3D Visualization group and the other two groups. These results suggest that learning to identify meaningful contour patterns is an effective strategy for learning how to comprehend topographic maps. Future research should address if learning strategies for how to interpret the information represented on a diagram (e.g. identify patterns in the contour lines), before trying to visualize the information in 3D (e.g. visualize the 3D structure of the terrain), also facilitates students' comprehension of other similar types of diagrams. / Psychology