<p>Autonomous Mobile Robots (AMR) and IoT (Internet of Things) devices have substantial potential to increase productivity in Small and Medium Enterprises (SME). However, factors such as high costs involved in setting up and maintaining AMR automation and deficit of programming expertise among factory workers render AMR automation uneconomical for SMEs. </p><p><br></p><p></p><p>Visual Programming can be used in the spatial context of the operating environment to enable closer mental models at the right level of abstraction for novice programmers. In this thesis, we develop a system for deploying a Spatially Situated Visual Language (SSVL) for programming robot-IoT workflows. To address the challenges of interoperability, scalability, and versatility of industrial machinery, we develop a novel communication protocol based on the Resource Description Framework (RDF). Our communication protocol forms the basis of a system architecture that integrates SSVL, IoT devices, and mobile robots that enables factory workers to program AMR workflows. We design, implement and validate the system and the architecture following the systems engineering approach. To evaluate the system developed in an industrial context, we deploy the system to author AMR workflows for material handling application for an SME painting factory.</p><p></p>

With current state-of-the-art approaches, development of a single mobile manipulation capability can be a labor-intensive process that presents an impediment to the creation of general purpose household robots. At the same time, we expect that involving a larger community of non-roboticists can accelerate the creation of new novel behaviors. We introduce the use of a software authoring environment called ROS Commander (ROSCo) allowing end-users to create, refine, and reuse robot behaviors with complexity similar to those currently created by roboticists. Akin to Photoshop, which provides end-users with interfaces for advanced computer vision algorithms, our environment provides interfaces to mobile manipulation algorithmic building blocks that can be combined and configured to suit the demands of new tasks and their variations. As our system can be more demanding of users than alternatives such as using kinesthetic guidance or learning from demonstration, we performed a user study with 11 able-bodied participants and one person with quadriplegia to determine whether computer literate non-roboticists will be able to learn to use our tool. In our study, all participants were able to successfully construct functional behaviors after being trained. Furthermore, participants were able to produce behaviors that demonstrated a variety of creative manipulation strategies, showing the power of enabling end-users to author robot behaviors. Additionally, we introduce how using autonomous robot learning, where the robot captures its own training data, can complement human authoring of behaviors by freeing users from the repetitive task of capturing data for learning. By taking advantage of the robot's embodiment, our method creates classifiers that predict using visual appearances 3D locations on home mechanisms where user constructed behaviors will succeed. With active learning, we show that such classifiers can be learned using a small number of examples. We also show that this learning system works with behaviors constructed by non-roboticists in our user study. As far as we know, this is the first instance of perception learning with behaviors not hand-crafted by roboticists.