<p>Tactile sensing is an essential human ability for understanding their surroundings. It allows humans to detect and manipulate objects that are concealed or difficult to see in low-light settings. Further, tactile sensing enables people to comprehend object and surface properties that cannot be obtained through visual feedback alone. This is achieved with gentle touches, enabling tactile exploration of fragile, sensitive objects, or living organisms. This capability could be transferred to robots through suitable hardware and algorithms. Nevertheless, current tactile sensors and skills for robotics are not comparable to the tactile sense of humans, thus resulting in inferior characterization of scenes and a risk of altering object states.</p> <p><br></p> <p>To address these limitations, this dissertation proposes a novel framework for robot active tactile exploration and object characterization. The framework combines bioinspired soft sensors and minimally invasive tactile exploration strategies to minimize perturbations to objects. This framework was achieved by: (1) an ultrasensitive whisker sensor that enables object characterization with minimal interaction forces; (2) autonomous tactile exploration skills to localize objects and then characterize their shape and surface properties; and (3) machine learning techniques to analyze contact information gathered by our tactile sensors, enabling the understanding of object attributes by tactile sensing alone. </p> <p><br></p> <p>Experiments were conducted to validate the effectiveness of the framework. In terms of object localization efficiency, informative path planners and contour exploration patterns outperformed baseline methods. Furthermore, the whisker sensor was successfully employed to characterize object surface and liquid properties. Finally, the features found through the characterization process allowed for successful classification by machine learning techniques. These results indicate that the proposed framework can effectively gather multimodal features from environments while maintaining the safety of objects. </p>

<p> This thesis demonstrates the combination of virtual reality (VR) and artificial intelligence (AI) specifically exploring the practical application of Natural Language Processing (NLP) and GPT-based models in educational VR laboratories. The objective is to design a comprehensive learning environment where users can independently engage in laboratory experiments, deriving similar educational outcomes as they would from a traditional, physical laboratory setup, particularly within the realms of Science, Technology, Engineering, and Mathematics (STEM) disciplines.</p> <p>Using machine learning techniques and authentic virtual reality simulating educational experiments, we propose an advanced learning platform—Virtual Reality Instructional Laboratory Environment (VRILE). A key feature of the VRILE is an AI-powered instructor capable of not only guiding the learners through the tasks but also responding intelligently to their actions in real time.</p> <p>The AI constituent of the VRILE uses the GPT-2 model for text generation in the field of Natural Language Processing (NLP). To ensure the generated instructions were contextually relevant and meaningful to lab participants, the model was trained on a dataset derived from an augmentation over user interactions within the VR environment.</p> <p>By pushing the boundaries of how AI can be utilized in educational VR environments, this research paves the way for broader adoption across other domains of engineering education. Furthermore, it provides a solid foundation for future research in this interdisciplinary field. It marks a significant stride in the integration of technology and education, encouraging more ventures into this promising frontier.</p>