Latency has always been one of the essential problems within Virtual Reality (VR) domain since VR is inherently an interactive paradigm which performs the real-time estimation of human motions. From the user's point of view, the latency extremely reduces the presence experience of VR systems, especially when user won’t able to perform interactions accurately. To compensate the excessive latency, different prediction methods on human motion were studied in recent years. Among them, Kalman Filter was the most popular choice. However, the effectiveness of using Kalman Filter to eliminate the latency for VR systems is not always satisfactory in practice since the accuracy of the estimation of the users’ motion depends on several factors: the linearity of the motion, the prediction time, the computational time, and the algorithm’s limitation.Therefore, this thesis presents a VR-based haptic teleoperation system to study how to effectively eliminate the latency effectively using Kalman Filter. For investigating the performances of different prediction methods for VR systems with several factors considered, two types of Kalman Filter: Linear Kalman Filter (LKF) and Unscented Kalman Filter (UKF) have been used to predict the haptic motion dataset, under different amount of simulated latencies.The result shows, both LKF and UKF provide a good performance at compensating the latency. For 200ms latency, both filters satisfactorily eliminate the latency and improve the interaction effectiveness. The comparative study shows, LKF provides better performance since the linear rotational motion dataset captured by haptic device was used; both filters show a reduced performance when the prediction time is increased. Besides, UKF requires more computational time than LKF.

Throughout history, the construction industry has been known to present many risks to its workers’ safety and is still the cause of many accidents. This thesis examines a potential solution to one of the main problems faced by this field.Teleoperation has gained popularity, especially over the last ten years, and its application can be useful to many domains. Amongst them, research done on teleoperation tools for construction sites has been growing considerably as it appears to be a solution for some of the safety problems that it’s facing. Through a literature study and the building and running of an experiment, it aims to investigate parameters that can influence the performance and user experience of participants while remotely controlling a platform. By selecting a few parameters that can play a role in video streaming, the goal is to correlate their influence on the quality of experience of participants as they experience a prototype for a construction site. After conducting the experiment with multiple test participants, the results are analyzed and can lead to conclusions. Finally, future improvements can be made for another iteration of the experiment.

[pt] Os últimos anos mostraram um interesse crescente no uso do controle háptico compartilhado (HSC) em sistemas teleoperados. No HSC, a aplicação de forças orientadoras virtuais, diminui o esforço de controle do usuário e melhora o tempo de execução em várias tarefas, apresentando uma boa alternativa em comparação com a teleoperação direta. O HSC, apesar de demonstrar bom desempenho, abre uma nova lacuna: como disenhar as forças orientadoras. Por esse motivo, o verdadeiro desafio está no desenvolvimento de controladores para fornecer as forças orientadoras virtuais, capazes de lidar com novas situações que aparecem enquanto uma tarefa está sendo executada. Este trabalho aborda esse desafio, projetando um controlador baseado no algoritmo Deep Deterministic Policy Gradient (DDPG) para fornecer assistência, e uma rede neural convolucional (CNN) para executar a detecção da tarefa. O agente aprende a minimizar o tempo que o ser humano leva para executar a tarefa desejada, minimizando simultaneamente sua resistência ao feedback fornecido. Essa resistência fornece ao algoritmo de aprendizado informações sobre a direção que o humano está tentando seguir, neste caso na tarefa pick-and-place. Diversos resultados demonstram a aplicação bem-sucedida da abordagem proposta, aprendendo políticas personalizadas para cada usuário que foi solicitado a testar o sistema. Ele exibe convergência estável e ajuda o usuário a concluir a tarefa com o menor número possível de etapas. / [en] Recent years have shown a growing interest in using haptic share control (HSC) in teleoperated systems. In HSC, the application of virtual guiding forces decreases the user’s control effort and improves execution time in various tasks, presenting a good alternative in comparison with direct teleoperation. HSC, despite demonstrating good performance, opens a new gap: how to design the guiding forces. For this reason, the real challenge lies in developing controllers to provide the virtual guiding forces, able to deal with new situations that appear while a task is being performed. This work addresses this challenge by designing a controller based on the deep deterministic policy gradient (DDPG) algorithm to provide the assistance, and a convolutional neural network (CNN) to perform the task detection. The agent learns to minimize the time it takes the human to execute the desired task, while simultaneously minimizing their resistance to the provided feedback. This resistance thus provides the learning algorithm with information about which direction the human is trying to follow, in this case, the pick-and-place task. Diverse results demonstrate the successful application of the proposed approach by learning custom policies for each user who was asked to test the system. It exhibits stable convergence and aids the user in completing the task with the least amount of steps possible.

京都大学 / 新制・課程博士 / 博士(工学) / 甲第24604号 / 工博第5110号 / 新制||工||1978(附属図書館) / 京都大学大学院工学研究科機械理工学専攻 / (主査)教授 松野 文俊, 教授 小森 雅晴, 教授 神田 崇行 / 学位規則第4条第1項該当 / Doctor of Philosophy (Engineering) / Kyoto University / DGAM

This thesis presents an interface for effectively teleoperating robots that combines an ecological display of range and video information with quickening based on dead-reckoning prediction. This display is built by viewing range and video information from a virtual camera trailing the robot. This is rendered in 3-D by using standard hardware acceleration and 3-D graphics software. Our studies demonstrate that this interface improves performance for most people, including those that do not have much previous experience with robotics. These studies involved 32 test subjects in a simulated environment and 8 in the real world. Subjects were required to drive the robot through several mazes while remembering a sequence of items. People took less time using the ecological interface and experienced fewer collisions, with a much lower workload as measured by joystick entropy. People preferred the interface over a standard interface with side-by-side range and video information and no prediction. Participants tended to rate the interface as more learn-able and more intuitive; participants also felt more confident in the robot's expected behavior.

As manufacturers are automating their production lines the use of robots isincreasing, but this does not exclude the need from humans with their nifty hands.Robots are using a certain tool for a certain job while humans have their hands forall situations. Schunk has developed a hand for these robot arms but it has yet tobe tested and implemented in a real world situation. By using a robot arm acompany can minimize the risk of damage to humans.In this work we are exploring the options of teleoperation using ros and moveitby performing a pick-and-place scenario. This task is done by a UR10 equippedwith a Schunk SIH.The results were that a ROS node could control the robot hand by sendingcommands inline or through a publishing node. This node is compatible with thewhole system and may be operated alongside a UR10 robotic arm. Moveit'smotion planning tool using inverse kinematics can drive the UR10 robot. Therobot arm was able to accomplish a pick-and-place job by placing a water bottleon top of a box.This investigation analyzes this effort from a variety of angles, one of which isan ethical one that emphasizes the need of making these advances for the sake ofhuman health.

No description available.

[pt] Telepresença refere-se a um conjunto de tecnologias que permite uma pessoa se sentir como se estivesse presente em um lugar que não seja a sua verdadeira localização. Esses equipamentos costumam usar uma câmera comum para filmar o que está acontecendo em um ambiente e transmitir ao vivo em televisões ou monitores para o usuário em outro ambiente. Algumas vezes as câmeras podem ser controladas através de dispositivos como teclado ou joysticks. Neste trabalho apresentamos uma ferramenta composta por um head-mounted display (HMD), o Oculus Rift DK1, integrado com um dispositivo nomeado de cabeça remota (CR), capaz de filmar usando uma câmera estéreo e transmitir para o Oculus Rift as imagens em 3D. Ao mesmo tempo, o giroscópio do HMD captura a orientação da cabeça do usuário e envia para a cabeça remota, que possui servo motores capazes de girá-la a fim de possibilitar o usuário movimentar a câmera estéreo sem nenhum dispositivo adicional. O objetivo é proporcionar uma experiência de telepresença imersiva, com uma interface de baixo custo e simples de usar. Foram realizados testes com usuários com cenários experimentais, que indicaram o benefício da ferramenta em videoconferências. / [en] Telepresence refers to a set of technologies that allows a person to feel as if he is in a place other than his true location. Such equipment uses an ordinary camera to film what is happening in an environment and transmits it alive on televisions or monitors to the user in another environment. Sometimes the cameras can be controlled through devices such as keyboards or joysticks. This dissertation presents a tool composed of a head-mounted display (HMD), we used the Oculus Rift DK1, integrated with a device called remote head, able to film with a stereo camera and to transmit to the Oculus Rift the images in 3D. At the same time, the HMD s gyroscope captures the user s head orientation and sends it to the remote head, which has servo motors able to rotate it in order to allow the user to move the stereo camera without any additional device. The project s goal is to provide the user an experience of immersive telepresence, with a low cost and a simple interface. Some tests with users were performed and indicated the benefit of the tool for videoconferencing.

<p>Have a nice day :)</p> / <p>This thesis presents a unified framework for coordination and control of human-in-the-loop asymmetric semi-autonomous robotic systems. It introduces a highly general teleoperation system configuration involving any number of operators, haptic interfaces, and robots with possibly different degrees of mobility. The proposed framework allows for mixed teleoperation/autonomous control of user-defined subtasks by establishing position/force tracking as well as kinematic constraints among relevant <em>teleoperation control frames</em>. Three layers of velocity-based autonomous subtasks at different priority levels with respect to human teleoperation are integrated into the control system design. The control strategy is hierarchical comprising of a high-level teleoperation coordinating controller and low-level joint velocity controllers. A Lyapunov-based adaptive joint-space velocity controller is presented as one candidate for the low-level control. The approach utilizes idempotent, generalized pseudoinverse and weighting matrices, as well as a soft-switching rank changing algorithm in order to achieve new performance objectives that are defined for such asymmetric semi-autonomous teleoperation systems. A detailed analysis of system performance and stability is presented. The proposed framework constitutes the most general formulation and solution for the teleoperation control problem to date. It yields many interesting and useful system configurations never studied before, in addition to those already considered in the literature. In particular, seven system configurations arising from the proposed teleoperation architecture are analyzed and studied in detail. Experimental results are provided to demonstrate the desired system response in these configurations. Moreover, human factors experiments are carried out to assess operator(s) performance in maneuverability and grasping under various teleoperation system configurations. The results show statistically significant performance improvement in teleoperation of a nonholonomic mobile robot and telegrasping using a twin-armed manipulator.</p> / Doctor of Philosophy (PhD)

<p>In this thesis, a human-in-the-loop control system is presented to assist an operator in teleoperation of a mobile robot. In a conventional teleoperation paradigm, the human operator would directly navigate the robot without any assistance which may result in poor performance in complex and unknown task environments due to inadequacy of visual feedback. The proposed method in this thesis builds on an earlier general control framework that systematically combines teleoperation and autonomous control subtasks. In this approach, the operator controls the mobile robot (slave) using a force-feedback haptic interface (master). Teleoperation control commands coordinate master and slave robots while an autonomous control subtask helps the operator avoid collisions with obstacles in the robot task environment by providing corrective force feedback. The autonomous collision avoidance is based on a Model Predictive Control (MPC) philosophy. The autonomous subtask control commands are generated by formulating and solving a constrained optimization problem over a rolling horizon window of time into the future using system models to predict the operator force and robot motion. The goal of the optimization is to prevent collisions within the prediction horizon by applying corrective force feedback, while minimizing interference with the operator teleoperation actions. It is assumed that the obstacles are stationary and sonar sensors mounted on the mobile robot measure the obstacle distances relative to the robot. Two formulation of MPC-based collision avoidance are proposed. The first formulation directly incorporates raw observation points as constraints in the MPC optimization problem. The second formulation relies on a line segment representation of the task environment. This thesis employs the well-known Hough transform method to effectively transform the raw sensor data into line segments. The extracted line segments constitute a compact model for the environment that is used in the formulation of collision constraints. The effectiveness of the proposed model-predictive control obstacle avoidance schemes is demonstrated in teleoperation experiments where the master robot is a 3DOF haptic interface and the slave is a P3-DX mobile robot equipped with eight (8) sonar sensors at the front.</p> / Master of Applied Science (MASc)