The aim of this thesis is to design and evaluate an ant algorithm for a certain class of (hybrid) flowshop problems. These problems consist of a two-staged production process with two identical machines on the second station and buffers in between the stages. For a given number of jobs, each consisting of a processing time and a deadline, the goal is to create a feasible partition and permutation of these jobs which minimizes the total tardiness. Furthermore, a theoretical investigation on the placement and size of the buffers and its impact on the resulting solution space was done. In order to further improve the swarm intelligence it was combined with a set of different heuristics and pheromone evaluation rules. The resulting variations of the algorithm were later tested and evaluated on a set of benchmark instances. Subsequently, the obtained theoretical insights on the flowshop problems were analyzed in a practical matter and proved to be valuable. Finally, the quality of the solutions found by the designed ant algorithm was analyzed on smaller pro- blem instances. It became apparent that at least for such smaller problems the algorithm creates very good solutions which only differ little from the optimum.

In this thesis, we focus on building reliable hybrid human-machine classifiers to be deployed in cost-sensitive classification tasks. The objective is to assess ML quality in hybrid classification contexts and design the appropriate metrics, thereby knowing whether we can trust the model predictions and identifying the subset of items on which the model is well-calibrated and trustworthy. We start by discussing the key concepts, research questions, challenges, and architecture to design and implement an effective hybrid classification service. We then present a deeper investigation of each service component along with our solutions and results. We mainly contribute to cost-sensitive hybrid classification, selective classification, model calibration, and active learning. We highlight the importance of model calibration in hybrid classification services and propose novel approaches to improve the calibration of human-machine classifiers. In addition, we argue that the current accuracy-based metrics are misaligned with the actual value of machine learning models and propose a novel metric ``value". We further test the performance of SOTA machine learning models in NLP tasks with a cost-sensitive hybrid classification context. We show that the performance of the SOTA models in cost-sensitive tasks significantly drops when we evaluate them according to value rather than accuracy. Finally, we investigate the quality of hybrid classifiers in the active learning scenarios. We review the existing active learning strategies, evaluate their effectiveness, and propose a novel value-aware active learning strategy to improve the performance of selective classifiers in the active learning of cost-sensitive tasks.

Artificial intelligence (AI) is revolutionizing contemporary industries and being applied in application domains ranging from recommendation systems to self-driving cars. In scenarios in which humans are interacting with an AI, inaccurate algorithms could lead to human mistreatment or even harmful events. Human-in-the-loop computing is a machine learning approach desiring hybrid intelligence, the combination of human and machine intelligence, to achieve accurate and interpretable results. This thesis applies human-in-the-loop computing in a Design Science Research project with a Swedish manufacturing company to make operational processes more efficient. The thesis aims to investigate emerging design principles useful for designing machine learning algorithms of hybrid intelligence. Hereby, the thesis has two key contributions: First, a theoretical framework is built that comprises general design knowledge originating from Information Systems (IS) research. Second, the analysis of empirical findings leads to the review of general IS design principles and to the formulation of useful design principles for human-in-the-loop computing. Whereas the principle of AI-readiness improves the likelihood of strategical AI success, the principle of hybrid intelligence shows how useful it can be to trigger a demand for human-in-the-loop computing in involved stakeholders. The principle of use case-marketing might help designers to promote the customer benefits of applying human-in-the-loop computing in a research setting. By utilizing the principle of power relationship and the principle of human-AI trust, designers can demonstrate the humans’ power over AI and build a trusting human-machine relationship. Future research is encouraged to extend and specify the formulated design principles and employ human-in-the-loop computing in different research settings. With regard to technological advancements in brain-machine interfaces, human-in-the-loop computing might even become much more critical in the future.