Over the past few decades the use of industrial robots has increased the efficiency as well as competitiveness of many companies. Despite this fact, in many cases, robot automation investments are considered to be technically challenging. In addition, for most small and medium sized enterprises (SME) this process is associated with high costs. Due to their continuously changing product lines, reprogramming costs are likely to exceed installation costs by a large margin. Furthermore, traditional programming methods for industrial robots are too complex for an inexperienced robot programmer, thus assistance from a robot programming expert is often needed.  We hypothesize that in order to make industrial robots more common within the SME sector, the robots should be reprogrammable by technicians or manufacturing engineers rather than robot programming experts. In this thesis we propose a high-level natural language framework for interacting with industrial robots through an instructional programming environment for the user.  The ultimate goal of this thesis is to bring robot programming to a stage where it is as easy as working together with a colleague.In this thesis we mainly address two issues. The first issue is to make interaction with a robot easier and more natural through a multimodal framework. The proposed language architecture makes it possible to manipulate, pick or place objects in a scene through high level commands. Interaction with simple voice commands and gestures enables the manufacturing engineer to focus on the task itself, rather than programming issues of the robot. This approach shifts the focus of industrial robot programming from the coordinate based programming paradigm, which currently dominates the field, to an object based programming scheme.The second issue addressed is a general framework for implementing multimodal interfaces. There have been numerous efforts to implement multimodal interfaces for computers and robots, but there is no general standard framework for developing them. The general framework proposed in this thesis is designed to perform natural language understanding, multimodal integration and semantic analysis with an incremental pipeline and includes a novel multimodal grammar language, which is used for multimodal presentation and semantic meaning generation. / robot colleague project

Visual scripting languages are used as alternatives to text programming to make coding easier. Visual programming languages provide a structure and a guidance that does not exist in text programming, which should make them easier to code with. Some users do however find that the structure in visual scripting languages makes it cumbersome to code. To find a design of visual scripting that subvert this and has a better user experience than contemporary designs, a prototype of a visual scripting interface was developed using an iterative design and testing cycle. When a final prototype had been developed, it was tested to see how it compared to text programming. From the tests performed, a few teachings were discovered. If-statements that grow perpendicularly to the rest of the code fit more information on the screen at the same time and can feel more natural and easier to parse for some users. Having a help menu with syntax-help makes it so that users do not have to leave the program, which increases programming speed. The visual coding elements in a visual scripting language need to be coloured such that the most important parts are the most visible; otherwise users have a hard time parsing the code. Showing existing variables that are in scope gives the user a good overview of what variables they can use. Having help menus where elements can be clicked to insert them at the user's text cursor reduces the chance of misspelling variables and gives the user confidence in the correctness of the code. Having visual coding elements that can change depending on context or by using toggles can make coding more intuitive and faster. / <p>Examensarbetet är utfört vid Institutionen för teknik och naturvetenskap (ITN) vid Tekniska fakulteten, Linköpings universitet</p>