No description available.

In this paper a cobot system is presented, that extends a Universal Robot with Artificial Intelligence (i.e., machine learning techniques) to allow for a safe human-robot collaboration, which is one of the main technologies in Industry 4.0 and is currently significantly changing the shop floor of manufacturing companies. Typically, these cobots are equipped with a camera to dynamically adapt to new situations and actions carried out by the worker who is collaborating with the robot in the same workspace. But obviously, switching from traditional industrial robots (acting completely isolated from humans) to smart robots also requires a change concerning the skills and knowledge workers must have to be able to control, manage, and interact with such cobot systems. Therefore, the main goal of this demonstrator is to develop a hard- and software environment, enabling a variety of different training scenarios to get trainees, employees, and students familiar with the main technical aspects of such human-robot interaction. Besides hardware and software related aspects, the paper will also briefly address the learning content, which is on the one hand, the basics of robotics and machine learning based image processing, and on the other hand, the interaction of the various components to form a functional overall system.

<p>Robotsystem har använts flitigt under de senaste årtiondena för att skapa automationslösningar i ett flertal områden. De flesta nuvarande automationslösningarna är begränsade av att uppgifterna de kan lösa måste vara repetitiva och förutsägbara. En av anledningarna till detta är att dagens robotsystem saknar förmåga att förstå och resonera om omvärlden. På grund av detta har forskare inom robotik och artificiell intelligens försökt att skapa intelligentare maskiner. Trots att stora framsteg har gjorts då det gäller att skapa robotar som kan fungera och interagera i en mänsklig miljö så finns det för nuvarande inget system som kommer i närheten av den mänskliga förmågan att resonera om omvärlden.</p><p>För att förenkla problemet har vissa forskare föreslagit en alternativ lösning till helt självständiga robotar som verkar i mänskliga miljöer. Alternativet är att kombinera människors och maskiners förmågor. Exempelvis så kan en person verka på en avlägsen plats, som kanske inte är tillgänglig för personen i fråga på grund av olika orsaker, genom att använda fjärrstyrning. Vid fjärrstyrning skickar operatören kommandon till en robot som verkar som en förlängning av operatörens egen kropp.</p><p>Segmentering och identifiering av rörelser skapade av en operatör kan användas för att tillhandahålla korrekt assistans vid fjärrstyrning eller samarbete mellan människa och maskin. Assistansen sker ofta inom ramen för virtuella fixturer där eftergivenheten hos fixturen kan justeras under exekveringen för att tillhandahålla ökad prestanda i form av ökad precision och minskad tid för att utföra uppgiften.</p><p>Den här avhandlingen fokuserar på två aspekter av samarbete mellan människa och maskin. Klassificering av en operatörs rörelser till ett på förhand specificerat tillstånd under en manipuleringsuppgift och assistans under manipuleringsuppgiften baserat på virtuella fixturer. Den specifika tillämpningen som behandlas är manipuleringsuppgifter där en mänsklig operatör styr en robotmanipulator i ett fjärrstyrt eller samarbetande system.</p><p>En metod för att följa förloppet av en uppgift medan den utförs genom att använda virtuella fixturer presenteras. Istället för att följa en på förhand specificerad plan så har operatören möjlighet att undvika oväntade hinder och avvika från modellen. För att möjliggöra detta estimeras kontinuerligt sannolikheten att operatören följer en viss trajektorie (deluppgift). Estimatet används sedan för att justera eftergivenheten hos den virtuella fixturen så att ett beslut om hur rörelsen ska fixeras kan tas medan uppgiften utförs.</p><p>En flerlagers dold Markovmodell (eng. layered hidden Markov model) används för att modellera mänskliga färdigheter. En gestemklassificerare som klassificerar en operatörs rörelser till olika grundläggande handlingsprimitiver, eller gestemer, evalueras. Gestemklassificerarna används sedan i en flerlagers dold Markovmodell för att modellera en simulerad fjärrstyrd manipuleringsuppgift. Klassificeringsprestandan utvärderas med avseende på brus, antalet gestemer, typen på den dolda Markovmodellen och antalet tillgängliga träningssekvenser. Den flerlagers dolda Markovmodellen tillämpas sedan på data från en trajektorieföljningsuppgift i 2D och 3D med en robotmanipulator för att ge både kvalitativa och kvantitativa resultat. Resultaten tyder på att den flerlagers dolda Markovmodellen är väl lämpad för att modellera trajektorieföljningsuppgifter och att den flerlagers dolda Markovmodellen är robust med avseende på felklassificeringar i de underliggande gestemklassificerarna.</p> / <p>Robot systems have been used extensively during the last decades to provide automation solutions in a number of areas. The majority of the currently deployed automation systems are limited in that the tasks they can solve are required to be repetitive and predicable. One reason for this is the inability of today’s robot systems to understand and reason about the world. Therefore the robotics and artificial intelligence research communities have made significant research efforts to produce more intelligent machines. Although significant progress has been made towards achieving robots that can interact in a human environment there is currently no system that comes close to achieving the reasoning capabilities of humans.</p><p>In order to reduce the complexity of the problem some researchers have proposed an alternative to creating fully autonomous robots capable of operating in human environments. The proposed alternative is to allow <i>fusion </i>of human and machine capabilities. For example, using teleoperation a human can operate at a remote site, which may not be accessible for the operator for a number of reasons, by issuing commands to a remote agent that will act as an extension of the operator’s body.</p><p>Segmentation and recognition of operator generated motions can be used to provide appropriate assistance during task execution in teleoperative and human-machine collaborative settings. The assistance is usually provided in a virtual fixture framework where the level of compliance can be altered online in order to improve the performance in terms of execution time and overall precision. Acquiring, representing and modeling human skills are key research areas in teleoperation, programming-by-demonstration and human-machine collaborative settings. One of the common approaches is to divide the task that the operator is executing into several sub-tasks in order to provide manageable modeling.</p><p>This thesis is focused on two aspects of human-machine collaborative systems.<i> Classfication </i>of an operator’s motion into a predefined state of a manipulation task and assistance during a manipulation task based on <i>virtual fixtures</i>. The particular applications considered consists of manipulation tasks where a human operator controls a robotic manipulator in a cooperative or teleoperative mode.</p><p>A method for online task tracking using <i>adaptive virtual fixtures</i> is presented. Rather than executing a predefined plan, the operator has the ability to avoid unforeseen obstacles and deviate from the model. To allow this, the probability of following a certain trajectory sub-task) is estimated and used to automatically adjusts the compliance of a virtual fixture, thus providing an online decision of how to fixture the movement.</p><p>A layered hidden Markov model is used to model human skills. A gestem classifier that classifies the operator’s motions into basic action-primitives, or gestemes, is evaluated. The gestem classifiers are then used in a layered hidden Markov model to model a simulated teleoperated task. The classification performance is evaluated with respect to noise, number of gestemes, type of the hidden Markov model and the available number of training sequences. The layered hidden Markov model is applied to data recorded during the execution of a trajectory-tracking task in 2D and 3D with a robotic manipulator in order to give qualitative as well as quantitative results for the proposed approach. The results indicate that the layered hidden Markov model is suitable for modeling teleoperative trajectory-tracking tasks and that the layered hidden Markov model is robust with respect to misclassifications in the underlying gestem classifiers.</p>

Competitive systems for manufacturing, especially assembly systems, have to cope with frequent changes of external as well as internal demands. A proactive behaviour in an assembly system should make it capable of rapid changes and have an ability to handle frequent changes and disturbances. During recent decades several different system theories have occurred of which the majority remained theories never taken to actual production solutions. The thesis presents results from four case studies.   It is suggested, that the proactivity of an assembly system is strongly influenced by the system’s ability to change the three parameters: 1) level of automation, 2) level of information, 3) level of competence[l3]  (among the operators in a defined work area).   Proactivity is not easy to describe. However, this[l4]  thesis has taken a step in that direction. A general definition of proactivity is “taking action by causing change towards a state and not only reacting to change when it happens”. Another way to phrase this is "to be anticipatory and taking charge of situations”. Proactivity can be described as the ability of preparation for: - Changes and disturbance during operation; - Planned long-term evolution for a sustainable and perfect production system.   Such a system consists of technical components efficiently integrated with human operators and has the ability to handle frequent changes. Proactivity in an assembly system is dependent on the following factors: -         Continuous changes; -         Mandate to allocate resources; -         Mandate to do short term planning. The thesis presents a first model for evaluation of different technical resources that contributes to an overall proactive system behaviour. The model has been published but not yet tested. / QC 20120119

Robotsystem har använts flitigt under de senaste årtiondena för att skapa automationslösningar i ett flertal områden. De flesta nuvarande automationslösningarna är begränsade av att uppgifterna de kan lösa måste vara repetitiva och förutsägbara. En av anledningarna till detta är att dagens robotsystem saknar förmåga att förstå och resonera om omvärlden. På grund av detta har forskare inom robotik och artificiell intelligens försökt att skapa intelligentare maskiner. Trots att stora framsteg har gjorts då det gäller att skapa robotar som kan fungera och interagera i en mänsklig miljö så finns det för nuvarande inget system som kommer i närheten av den mänskliga förmågan att resonera om omvärlden. För att förenkla problemet har vissa forskare föreslagit en alternativ lösning till helt självständiga robotar som verkar i mänskliga miljöer. Alternativet är att kombinera människors och maskiners förmågor. Exempelvis så kan en person verka på en avlägsen plats, som kanske inte är tillgänglig för personen i fråga på grund av olika orsaker, genom att använda fjärrstyrning. Vid fjärrstyrning skickar operatören kommandon till en robot som verkar som en förlängning av operatörens egen kropp. Segmentering och identifiering av rörelser skapade av en operatör kan användas för att tillhandahålla korrekt assistans vid fjärrstyrning eller samarbete mellan människa och maskin. Assistansen sker ofta inom ramen för virtuella fixturer där eftergivenheten hos fixturen kan justeras under exekveringen för att tillhandahålla ökad prestanda i form av ökad precision och minskad tid för att utföra uppgiften. Den här avhandlingen fokuserar på två aspekter av samarbete mellan människa och maskin. Klassificering av en operatörs rörelser till ett på förhand specificerat tillstånd under en manipuleringsuppgift och assistans under manipuleringsuppgiften baserat på virtuella fixturer. Den specifika tillämpningen som behandlas är manipuleringsuppgifter där en mänsklig operatör styr en robotmanipulator i ett fjärrstyrt eller samarbetande system. En metod för att följa förloppet av en uppgift medan den utförs genom att använda virtuella fixturer presenteras. Istället för att följa en på förhand specificerad plan så har operatören möjlighet att undvika oväntade hinder och avvika från modellen. För att möjliggöra detta estimeras kontinuerligt sannolikheten att operatören följer en viss trajektorie (deluppgift). Estimatet används sedan för att justera eftergivenheten hos den virtuella fixturen så att ett beslut om hur rörelsen ska fixeras kan tas medan uppgiften utförs. En flerlagers dold Markovmodell (eng. layered hidden Markov model) används för att modellera mänskliga färdigheter. En gestemklassificerare som klassificerar en operatörs rörelser till olika grundläggande handlingsprimitiver, eller gestemer, evalueras. Gestemklassificerarna används sedan i en flerlagers dold Markovmodell för att modellera en simulerad fjärrstyrd manipuleringsuppgift. Klassificeringsprestandan utvärderas med avseende på brus, antalet gestemer, typen på den dolda Markovmodellen och antalet tillgängliga träningssekvenser. Den flerlagers dolda Markovmodellen tillämpas sedan på data från en trajektorieföljningsuppgift i 2D och 3D med en robotmanipulator för att ge både kvalitativa och kvantitativa resultat. Resultaten tyder på att den flerlagers dolda Markovmodellen är väl lämpad för att modellera trajektorieföljningsuppgifter och att den flerlagers dolda Markovmodellen är robust med avseende på felklassificeringar i de underliggande gestemklassificerarna. / Robot systems have been used extensively during the last decades to provide automation solutions in a number of areas. The majority of the currently deployed automation systems are limited in that the tasks they can solve are required to be repetitive and predicable. One reason for this is the inability of today’s robot systems to understand and reason about the world. Therefore the robotics and artificial intelligence research communities have made significant research efforts to produce more intelligent machines. Although significant progress has been made towards achieving robots that can interact in a human environment there is currently no system that comes close to achieving the reasoning capabilities of humans. In order to reduce the complexity of the problem some researchers have proposed an alternative to creating fully autonomous robots capable of operating in human environments. The proposed alternative is to allow fusion of human and machine capabilities. For example, using teleoperation a human can operate at a remote site, which may not be accessible for the operator for a number of reasons, by issuing commands to a remote agent that will act as an extension of the operator’s body. Segmentation and recognition of operator generated motions can be used to provide appropriate assistance during task execution in teleoperative and human-machine collaborative settings. The assistance is usually provided in a virtual fixture framework where the level of compliance can be altered online in order to improve the performance in terms of execution time and overall precision. Acquiring, representing and modeling human skills are key research areas in teleoperation, programming-by-demonstration and human-machine collaborative settings. One of the common approaches is to divide the task that the operator is executing into several sub-tasks in order to provide manageable modeling. This thesis is focused on two aspects of human-machine collaborative systems. Classfication of an operator’s motion into a predefined state of a manipulation task and assistance during a manipulation task based on virtual fixtures. The particular applications considered consists of manipulation tasks where a human operator controls a robotic manipulator in a cooperative or teleoperative mode. A method for online task tracking using adaptive virtual fixtures is presented. Rather than executing a predefined plan, the operator has the ability to avoid unforeseen obstacles and deviate from the model. To allow this, the probability of following a certain trajectory sub-task) is estimated and used to automatically adjusts the compliance of a virtual fixture, thus providing an online decision of how to fixture the movement. A layered hidden Markov model is used to model human skills. A gestem classifier that classifies the operator’s motions into basic action-primitives, or gestemes, is evaluated. The gestem classifiers are then used in a layered hidden Markov model to model a simulated teleoperated task. The classification performance is evaluated with respect to noise, number of gestemes, type of the hidden Markov model and the available number of training sequences. The layered hidden Markov model is applied to data recorded during the execution of a trajectory-tracking task in 2D and 3D with a robotic manipulator in order to give qualitative as well as quantitative results for the proposed approach. The results indicate that the layered hidden Markov model is suitable for modeling teleoperative trajectory-tracking tasks and that the layered hidden Markov model is robust with respect to misclassifications in the underlying gestem classifiers. / QC 20101102

No description available.

<p dir="ltr">Bridge safety has been a longstanding priority for civil engineers. Engineers and researchers devote significant effort toward establishing the ability to detect and monitor damage and to manage the performance of our bridges over their lifecycle. However, current practices, heavily reliant on manual involvement, still present challenges in efficiency and effectiveness. For instance, rapid bridge vulnerability assessment methods are being developed, but these methods normally require information about the bridge, e.g., substructure type, which is not readily available in most current bridge databases (e.g., National Bridge Inventory). Manually collecting the necessary information for each bridge is time-consuming and would influence the use of those rapid bridge vulnerability assessment methods. Similarly, routine bridge inspection, mandated every two years in the United States, requires organizing inspection photos, evaluating bridge condition, etc., which are also time-consuming tasks. A two-year inspection cycle may be overly cautious, especially in the early stages of its life when there is typically little degradation. Furthermore, when conducting an inspection there is no reference for when to use advanced inspection techniques, so visual inspection, the simplest method, is adopted for most bridges. Given these challenges in bridge asset management, the integration of machine learning to use the data from historical records of bridge performance can aid in and serve to expedite the above tasks.</p><p dir="ltr">The objective of this research is to develop machine learning-based methods that can assist humans in completing certain tasks associated with bridge asset management. Towards this objective, the following research tasks are carried out. In Task 1, a CNN-based bridge substructure identifier is developed to automatically recognize the bridge substructure type from an inspection image. A method is developed to set a rational budget for this work based on risk tolerance. In Task 2, the automated bridge inspection image organization tool (ABIRT) is developed to automate the process of organizing inspection images and generating an inspection report. In Task 3, a technique for machine-assisted bridge damage analysis using visual data is developed and validated. Additionally, a decision-making method is established to assist bridge inspectors in adopting this technique, with a focus on managing costs and minimizing risks. And in Task 4, reinforcement learning-based approach is developed to manage the bridge inspection process. Through this research, several key machine learning techniques are explored to assist bridge managers with the more tedious steps involved in the seismic vulnerability analysis and condition management tasks, allowing the engineer to dedicate more time to making decisions. This highly cross-disciplinary research is scalable and expandable to many other applications and will serve to improve the future safety and reliability of our infrastructure.</p>