<p>Object rearrangement is a fundamental problem in robotics with various practical applications ranging from managing warehouses to cleaning and organizing home kitchens. While existing research has primarily focused on single-agent solutions, real-world scenarios often require multiple robots to work together on rearrangement tasks. We propose a comprehensive learning-based framework for multi-agent object rearrangement planning, addressing the challenges of task sequencing and path planning in complex environments. The proposed method iteratively selects objects, determines their relocation regions, and pairs them with available robots under kinematic feasibility and task reachability for execution to achieve the target arrangement. Our experiments on a diverse range of environments demonstrate the effectiveness and robustness of the proposed framework. Furthermore, results indicate improved performance in terms of traversal time and success rate compared to baseline approaches. The videos and supplementary material are available at https://sites.google.com/view/maner-supplementary</p>

Networked telerobots are operated by humans through remote interactions and have found applications in unstructured environments, such as outer space, underwater, telesurgery, manufacturing etc. In precision agricultural robotics, target monitoring, recognition and detection is a complex task, requiring expertise, hence more efficiently performed by collaborative human-robot systems. A HUB is an online portal, a platform to create and share scientific and advanced computing tools. HUB-CI is a similar tool developed by PRISM center at Purdue University to enable cyber-augmented collaborative interactions over cyber-supported complex systems. Unlike previous HUBs, HUB-CI enables both physical and virtual collaboration between several groups of human users along with relevant cyber-physical agents. This research, sponsored in part by the Binational Agricultural Research and Development Fund (BARD), implements the HUB-CI model to improve the Collaborative Intelligence (CI) of an agricultural telerobotic system for early detection of anomalies in pepper plants grown in greenhouses. Specific CI tools developed for this purpose include: (1) Spectral image segmentation for detecting and mapping to anomalies in growing pepper plants; (2) Workflow/task administration protocols for managing/coordinating interactions between software, hardware, and human agents, engaged in the monitoring and detection, which would reliably lead to precise, responsive mitigation. These CI tools aim to minimize interactions’ conflicts and errors that may impede detection effectiveness, thus reducing crops quality. Simulated experiments performed show that planned and optimized collaborative interactions with HUB-CI (as opposed to ad-hoc interactions) yield significantly fewer errors and better detection by improving the system efficiency by between 210% to 255%. The anomaly detection method was tested on the spectral image data available in terms of number of anomalous pixels for healthy plants, and plants with stresses providing statistically significant results between the different classifications of plant health using ANOVA tests (P-value = 0). Hence, it improves system productivity by leveraging collaboration and learning based tools for precise monitoring for healthy growth of pepper plants in greenhouses.

<p>By using flexible and stretchable materials in place of fixed components, soft robots can materially adapt or change to their environment, providing built-in safeties for robotic operation around humans or fragile, delicate objects. And yet, building a robot out of only soft and flexible materials can be a significant challenge depending on the tasks that the robot needs to perform, for example if the robot were to need to exert higher forces (even temporarily) or self-report its current state (as it deforms unexpectedly around external objects). Thus, the appeal of multifunctional materials for soft robots, wherein the materials used to build the body of the robot also provide actuation, sensing, or even simply electrical connections, all while maintaining the original vision of environmental adaptability or safe interactions. Multifunctional material systems are explored throughout the body of this dissertation in three ways: (1) Sensor integration into high strain actuators for state estimation and closed-loop control. (2) Simplified control of multifunctional material systems by enabling multiple functions through a single input stimulus (<i>i.e.</i>, only requiring one source of input power). (3) Presenting a solution for the open challenge of controlling both well established and newly developed thermally-responsive soft robotic materials through an on-body, high strain, uniform, Joule-heating energy source. Notably, these explorations are not isolated from each other as, for example, work towards creating a new material for thermal control also facilitated embedded sensory feedback. The work presented in this dissertation paves a way forward for multifunctional material integration, towards the end-goal of full-functioning soft robots, as well as (more broadly) design methodologies for other safety-forward or adaptability-forward technologies.</p>

The onset of Industry 4.0 has provided considerable benefits to Intelligent Cyber-Physical Systems (ICPS), with technologies such as internet of things, wireless sensing, cognitive computing and artificial intelligence to improve automation and control. However, with increasing automation, the “human” element in industrial systems is often overlooked for the sake of standardization. While automation aims to redirect the workload of human to standardized and programmable entities, humans possess qualities such as cognitive awareness, perception and intuition which cannot be automated (or programmatically replicated) but can provide automated systems with much needed robustness and sustainability, especially in unstructured and dynamic environments. Incorporating tangible human skills and knowledge within industrial environments is an essential function of “Human-in-the-loop” (HITL) Systems, a term for systems powerfully augmented by different qualities of human agents. The primary challenge, however, lies in the realistic modelling and application of these qualities; an accurate human model must be developed, integrated and tested within different cyber-physical workflows to 1) validate the assumed advantages, investments and 2) ensure optimized collaboration between entities. Agricultural Robotic Systems (ARS) are an example of such cyber-physical systems (CPS) which, in order to reduce reliance on traditional human-intensive approaches, leverage sensor networks, autonomous robotics and vision systems and for the early detection of diseases in greenhouse plants. Complete elimination of humans from such environments can prove sub-optimal given that greenhouses present a host of dynamic conditions and interactions which cannot be explicitly defined or managed automatically. Supported by efficient algorithms for sampling, routing and search, HITL augmentation into ARS can provide improved detection capabilities, system performance and stability, while also reducing the workload of humans as compared to traditional methods. This research thus studies the modelling and integration of humans into the loop of ARS, using simulation techniques and employing intelligent protocols for optimized interactions. Human qualities are modelled in human “classes” within an event-based, discrete time simulation developed in Python. A logic controller based on collaborative intelligence (HUB-CI) efficiently dictates workflow logic, owing to the multi-agent and multi-algorithm nature of the system. Two integration hierarchies are simulated to study different types of integrations of HITL: Sequential, and Shared Integration. System performance metrics such as costs, number of tasks and classification accuracy are measured and compared for different collaboration protocols within each hierarchy, to verify the impact of chosen sampling and search algorithms. The experiments performed show the statistically significant advantages of HUB-CI based protocol over traditional protocols in terms of collaborative task performance and disease detectability, thus justifying added investment due to the inclusion of HITL. The results also discuss the competitive factors between both integrations, laying out the relative advantages and disadvantages and the scope for further research. Improving human modelling and expanding the range of human activities within the loop can help to improve the practicality and accuracy of the simulation in replicating an HITL-ARS. Finally, the research also discusses the development of a user-interface software based on ARS methodologies to test the system in the real-world.<br>

<p> </p> <p>The usage of robots has increased in different areas of society and human work, including medicine, transportation, education, space exploration, and the service industry. This phenomenon has generated a sudden enthusiasm to develop more intelligent robots that are better equipped to perform tasks in a manner that is equivalently good as those completed by humans. Such jobs require human involvement as operators or teammates since robots struggle with automation in everyday settings. Soon, the role of humans will be far beyond users or stakeholders and include those responsible for training such robots. A popular teaching form is to allow robots to mimic human behavior. This method is intuitive and natural and does not require specialized knowledge of robotics. While there are other methods for robots to complete tasks effectively, collaborative tasks require mutual understanding and coordination that is best achieved by mimicking human motion. This mimicking problem has been tackled through skill imitation, which reproduces human-like motion during a task shown by a trainer. Skill imitation builds on faithfully replicating the human pose and requires two steps. In the first step, an expert's demonstration is captured and pre-processed, and motion features are obtained; in the second step, a learning algorithm is used to optimize for the task. The learning algorithms are often paired with traditional control systems to transfer the demonstration to the robot successfully. However, this methodology currently faces a generalization issue as most solutions are formulated for specific robots or tasks. The lack of generalization presents a problem, especially as the frequency at which robots are replaced and improved in collaborative environments is much higher than in traditional manufacturing. Like humans, we expect robots to have more than one skill and the same skills to be completed by more than one type of robot. Thus, we address this issue by proposing a human motion imitation framework that can be efficiently computed and generalized for different kinematic structures (e.g., different robots).</p> <p> </p> <p>This framework is developed by training an algorithm to augment collaborative demonstrations, facilitating the generalization to unseen scenarios. Later, we create a model for pose imitation that converts human motion to a flexible constraint space. This space can be directly mapped to different kinematic structures by specifying a correspondence between the main human joints (i.e., shoulder, elbow, wrist) and robot joints. This model permits having an unlimited number of robotic links between two assigned human joints, allowing different robots to mimic the demonstrated task and human pose. Finally, we incorporate the constraint model into a reward that informs a Reinforcement Learning algorithm during optimization. We tested the proposed methodology in different collaborative scenarios. Thereafter, we assessed the task success rate, pose imitation accuracy, the occlusion that the robot produces in the environment, the number of collisions, and finally, the learning efficiency of the algorithm.</p> <p> </p> <p>The results show that the proposed framework creates effective collaboration in different robots and tasks.</p>

<p dir="ltr">This dissertation presents the findings and research conducted during my Ph.D. study, which focuses on developing grasp policies for modular end-effectors on mobile manipulation platforms operating in cluttered environments. The primary objective of this research is to enhance the performance and accuracy of robotic manipulation systems in complex, real-world scenarios. The work has potential implications for various domains, including the rapidly growing Industry 4.0 and the advancement of autonomous systems in space habitats.</p><p dir="ltr">The dissertation offers a comprehensive literature review, emphasizing the challenges faced by mobile manipulation platforms in cluttered environments and the state-of-the-art techniques for grasping and manipulation. It showcases the development and evaluation of a Modular End-Effector System (MEES) for mobile manipulation platforms, which includes the investigation of object 6D pose estimation techniques, the generation of a deep learning-based grasping dataset for MEES, the development of a suction cup gripper grasping policy (Sim-Suction), the development of a two-finger grasping policy (Sim-Grasp), and the integration of Modular End-Effector System grasping policy (Sim-MEES). The proposed methodology integrates hardware designs, control algorithms, data-driven methods, and large language models to facilitate adaptive grasping strategies that consider the unique constraints and requirements of cluttered environments.</p><p dir="ltr">Furthermore, the dissertation discusses future research directions, such as further investigating the Modular End-Effector System grasping policy. This Ph.D. study aims to contribute to the advancement of robotic manipulation technology, ultimately enabling more versatile and robust mobile manipulation platforms capable of effectively interacting with complex environments.</p>

Agent-based systems usually operate in real-time, stochastic and dynamic environments. Many theoretical and applied techniques have been applied to the investigation of agent architecture with respect to communication, cooperation, and learning, in order to provide a framework for implementing artificial intelligence and computing techniques. Intelligent agents are required to be able to adapt and learn in uncertain environments via communication and collaboration (in both competitive and cooperative situations). The ability of reasoning and learning is one fundamental feature for intelligent agents. Due to the inherent complexity, however, it is difficult to verify the properties of the complex and dynamic environments a priori. Since analytic techniques are inadequate for solving these problems, reinforcement learning (RL) has appeared as a popular approach by mapping states to actions, so as to maximise the long-term rewards. Computer simulation is needed to replicate an experiment for testing and verifying the efficiency of simulation-based optimisation techniques. In doing so, a simulation testbed called robot soccer is used to test the learning algorithms in the specified scenarios. This research involves the investigation of simulation-based optimisation techniques in agent-based systems. Firstly, a hybrid agent teaming framework is presented for investigating agent team architecture, learning abilities, and other specific behaviors. Secondly, the novel reinforcement learning algorithms to verify goal-oriented agents; competitive and cooperative learning abilities for decision-making are developed. In addition, the function approximation technique known as tile coding (TC), is used to avoid the state space growing exponentially with the curse of dimensionality. Thirdly, the underlying mechanism of eligibility traces is analysed in terms of on-policy algorithm and off-policy algorithm, accumulating traces and replacing traces. Fourthly, the "design of experiment" techniques, such as Simulated Annealing method and Response Surface methodology, are integrated with reinforcement learning techniques to enhance the performance. Fifthly, a methodology is proposed to find the optimal parameter values to improve convergence and efficiency of the learning algorithms. Finally, this thesis provides a serious full-fledged numerical analysis on the efficiency of various RL techniques.

<div>There are millions of individuals in the world who currently experience limited mobility as a result of aging, stroke, injuries to the brain or spinal cord, and certain neurological diseases. Robotic Assistive Devices (RADs) have shown superiority in helping people with limited mobility by providing physical movement assistance. However, RADs currently existing on the market for people with limited mobility are still far from intelligent.</div><div><br></div><div>Learning control strategies are developed in this study to make a Cable-Driven Assistive Device (CDAD) intelligent in assisting a human joint (e.g., a knee joint, an ankle joint, or a wrist joint). CDADs are a type of RADs designed based on Cable-Driven Parallel Robots (CDPRs). A PID–FNN control strategy and DDPG-based strategies are proposed to allow a CDAD to learn physical human-robot interactions when controlling the pose of the human joint. Both pose-tracking and trajectory-tracking tasks are designed to evaluate the PID–FNN control strategy and the DDPG-based strategies through simulations. Simulations are conducted in the Gazebo simulator using an example CDAD with three degrees of freedom and four cables. Simulation results show that the proposed PID–FNN control strategy and DDPG-based strategies work in controlling a CDAD with proper learning.</div>

<div>Self-organization in a multi-robot system is a spontaneous process where some form of overall order arises from local interactions between robots in an initially disordered system. Cooperative coordination strategies for self-organization promote teamwork to complete a task while increasing the total utility of the system. In this dissertation, we apply prosocial behavioral concepts such as altruism and cooperation in multi-robot systems and investigate their effects on overall system performance on given tasks. We stress the significance of this research in long-term applications involving minimal to no human supervision, where self-sustainability of the multi-robot group is of utmost importance for the success of the mission at hand and system re-usability in the future.</div><div><br></div><div>For part of the research, we take bio-inspiration of cooperation from the huddling behavior of Emperor Penguins in the Antarctic which allows them to share body heat and survive one of the harshest environments on Earth as a group. A cyclic energy sharing concept is proposed for a convoying structured multi-robot group inspired from penguin movement dynamics in a huddle with carefully placed induction coils to facilitate directional energy sharing with neighbors and a position shuffling algorithm, allowing long-term survival of the convoy as a group in the field. Simulation results validate that the cyclic process allows individuals an equal opportunity to be at the center of the group identified as the most energy conserving position, and as a result robot groups were able to travel over 4 times the distance during convoying with the proposed method without any robot failing as opposed to without the shuffling and energy sharing process. </div><div><br></div><div>An artificial potential based Adaptive Inter-agent Spacing (AIS) control law is also proposed for efficient energy distribution in an unstructured multi-robot group aimed at long-term survivability goals in the field. By design, as an altruistic behavior higher energy bearing robots are dispersed throughout the group based on their individual energy levels to counter skewed initial distributions for faster group energy equilibrium attainment. Inspired by multi-huddle merging and splitting behavior of Emperor Penguins, a clustering and sequential merging based systematic energy equilibrium attainment method is also proposed as a supplement to the AIS controller. The proposed system ensures that high energy bearing agents are not over crowded by low energy bearing agents. The AIS controller proposed for the unstructured energy sharing and distribution process yielded 55%, 42%, 23% and 33% performance improvements in equilibrium attainment convergence time for skewed, bi-modal, normal and random initial agent resource level distributions respectively on a 2D plane using the proposed energy distribution method over the control method of no adaptive spacing. Scalability analysis for both energy sharing concepts confirmed their application with consistently improved performances different sized groups of robots. Applicability of the AIS controller as a generalized resource distribution method under certain constraints is also discussed to establish its significance in various multi-robot applications.</div><div><br></div><div>A concept of group based survival from damaging directional external stimuli is also adapted from the Emperor Penguin huddling phenomenon where individuals on the damaging stimuli side continuously relocate to the leeward side of the group following the group boundary using Gaussian Processes Machine Learning based global health-loss rate minima estimations in a distributed manner. The method relies on cooperation from all robots where individuals take turns being sheltered by the group from the damaging external stimuli. The distributed global health loss rate minima estimation allowed the development of two settling conditions. The global health loss rate minima settling method yielded 12.6%, 5.3%, 16.7% and 14.2% improvement in average robot health over the control case of no relocation, while an optimized health loss rate minima settling method further improved on the global health loss rate settling method by 3.9%, 1.9%, 1.7% and 0.6% for robot group sizes 26, 35, 70 and 107 respectively.</div><div><br></div><div>As a direct application case study of collaboration in multi-robot systems, a distributed shape formation strategy is proposed where robots act as beacons to help neighbors settle in a prescribed formation by local signaling. The process is completely distributed in nature and does not require any external control due to the cooperation between robots. Beacon robots looking for a robot to settle as a neighbor and continue the shape formation process, generates a surface gradient throughout the formed shape that allow robots to determine the direction of the structure forming frontier along the dynamically changing structure surface and eventually reach the closest beacon. Simulation experiments validate complex shape formation in 2D and 3D using the proposed method. The importance of group collaboration is emphasized in this case study without which the shape formation process would not be possible, without a centralized control scheme directing individual agents to specific positions in the structure. </div><div> </div><div>As the final application case study, a collaborative multi-agent transportation strategy is proposed for unknown objects with irregular shape and uneven weight distribution. Although, the proposed system is robust to single robot object transportation, the proposed methodology of transport is focused on robots regulating their effort while pushing objects from an identified pushing location hoping other robots support the object moment on the other end of the center of mass to prevent unintended rotation and create an efficient path of the object to the goal. The design of the object transportation strategy takes cooperation cues from human behaviors when coordinating pushing of heavy objects from two ends. Collaboration is achieved when pushing agents can regulate their effort with one another to maintain an efficient path for the object towards the set goal. Numerous experiments of pushing simple shapes such as disks and rectangular boxes and complex arbitrary shapes with increasing number of robots validate the significance and effectiveness of the proposed method. Detailed robustness studies of changing weight of objects during transportation portrayed the importance of cooperation in multi-agent systems in countering unintended drift effects of the object and maintain a steady efficient path to the goal. </div><div><br></div><div>Each case study is presented independent of one another with the Penguin huddling based self-organizations in response to internal and external stimuli focused on fundamental self-organization methods, and the structure formation and object transportation strategies focused on cooperation in specific applications. All case studies are validated by relevant simulation and experiments to establish the effectiveness of altruistic and cooperative behaviors in multi-robot systems.</div>

Game balance has been a challenge for game developers since the time games have become more complex. There have been a handful of proposals for game balancing processes outside the manual labor-intensive play testing methods, which most game developers often are forced to use simply due to the lack of better methods. Simple solutions, like restrictive game play, are limited because of their inability to provide insight on interdependencies among the mechanisms in the game. Complex techniques framed around the potential of AI algorithms are limited by computational budgets or cognition inability to assess human actions. In order to find a middle ground we investigate Game Theory and Mechanism Design concepts. Both have proven to be effective tools to analyse strategic situations among interacting participants, or in this case `players'. We test the feasibility of using these techniques in an Real Time Strategy (RTS) game domain to understand game balance. MicroRTS, a small and simple execution of an RTS game is employed as our model. The results provide promising insight on the effectiveness of the method in detecting imbalances and further inspection to find the cause. An additional benefit out of this technique, besides detecting for game imbalances, the approach can be leveraged to create imbalances. This is useful when the designer or player desires to do so.