Classification is a pervasive problem in research that aims at grouping items in categories according to established criteria. There are two prevalent ways to classify items of interest: i) to train and exploit machine learning (ML) algorithms or ii) to resort to human classification (via experts or crowdsourcing). Machine Learning algorithms have been rapidly improving with an impressive performance in complex problems such as object recognition and natural language understanding. However, in many cases they cannot yet deliver the required levels of precision and recall, typically due to difficulty of the problem and (lack of) availability of sufficiently large and clean datasets. Research in crowdsourcing has also made impressive progress in the last few years, and the crowd has been shown to perform well even in difficult tasks [Callaghan et al., 2018; Ranard et al., 2014]. However, crowdsourcing remains expensive, especially when aiming at high levels of accuracy, which often implies collecting more votes per item to make classification more robust to workers' errors. Recently, we witness rapidly emerging the third direction of hybrid crowd-machine classification that can achieve superior performance by combining the cost-effectiveness of automatic machine classifiers with the accuracy of human judgment. In this thesis, we focus on designing crowdsourcing strategies and hybrid crowd-machine approaches that optimize the item classification problem in terms of results and budget. We start by investigating crowd-based classification under the budget constraint with different loss implications, i.,e., when false positive and false negative errors carry different harm to the task. Further, we propose and validate a probabilistic crowd classification algorithm that iteratively estimates the statistical parameters of the task and data to efficiently manage the accuracy vs. cost trade-off. We then investigate how the crowd and machines can support each other in tackling classification problems. We present and evaluate a set of hybrid strategies balancing between investing money in building machines and exploiting them jointly with crowd-based classifiers. While analyzing our results of crowd and hybrid classification, we found it is relevant to study the problem of quality of crowd observations and their confusions as well as another promising direction of linking entities from structured and unstructured sources of data. We propose crowd and neural network grounded algorithms to cope with these challenges followed by rich evaluation on synthetic and real-world datasets.

While many human computation (human-in-the-loop) systems exist in the field of Artificial Intelligence (AI) to solve problems that can't be solved by computers alone, comparatively fewer platforms exist for collecting human knowledge, and evaluation of various techniques for harnessing human insights in improving forecasting models for infectious diseases, such as Influenza and Ebola. In this thesis, we present the design and implementation of My4Sight, a human computation system developed to harness human insights and intelligence to improve forecasting models. This web-accessible system simplifies the collection of human insights through the careful design of the following two tasks: (i) asking users to rank system-generated forecasts in order of likelihood; and (ii) allowing users to improve upon an existing system-generated prediction. The structured output collected from querying human computers can then be used in building better forecasting models. My4Sight is designed to be a complete end-to- end analytical platform, and provides access to data collection features and statistical tools that are applied to the collected data. The results are communicated to the user, wherever applicable, in the form of visualizations for easier data comprehension. With My4Sight, this thesis makes a valuable contribution to the field of epidemiology by providing the necessary data and infrastructure platform to improve forecasts in real time by harnessing the wisdom of the crowd. / Master of Science

Most phenomena in engineering fields involve physical variables that can potentially be predicted using simple or complex mathematical models. However, traffic engineers and researchers are faced with a complex challenge since they have to deal with the human element. For instance, it can be stated that the biggest challenge facing researchers in the area of car-following theory relates to accounting for the human-in-the-loop while modeling the longitudinal motion of the vehicles. In fact, a major drawback of existing car-following models is that the human-in-the-loop is not modeled explicitly. This is specifically important since the output from car-following models directly impacts several other factors and measures of effectiveness, such as vehicle emissions and fuel consumption levels. The main contribution of this research relates to modeling and incorporating, in an explicit and independent manner, the human-in-the-loop component in car-following theory in such a way that it can be either activated or deactivated depending on if a human driver is in control of the vehicle. That would ensure that a car-following model is able to reflect the different control and autonomy levels that a vehicle could be operated under. Besides that, this thesis offers a better understanding of how humans behave and differ from each other. In fact, through the implementation of explicit parameters representing the human-in-the-loop element, the heterogeneity of human behavior, in terms of driving patterns and styles, is captured. To achieve its contributions, the study starts by modifying the maximum acceleration vehicle-dynamics model by explicitly incorporating parameters that aim to model driver behavior in its expression making it suitable for the representation of typical acceleration behavior. The modified variant of the model is demonstrated to have a flexible shape that allows it to model different types of variations that drivers can generate, and to be superior to other similar models in that it predicts more accurate acceleration levels in all domains. The resulting model is then integrated in the Rakha-Pasumarthy-Adjerid car-following model, which uses a steady-state formulation along with acceleration and collision avoidance constraints to model the longitudinal motion of vehicles. The validation of the model using a naturalistic dataset found that the modified formulation successfully integrated the human behavior component in the model and that the new formulation decreases the modeling error. Thereafter, this dissertation proposes a new car-following model, which we term the Fadhloun-Rakha model. Even though structurally different, the developed model incorporates the key components of the Rakha-Pasumarthy-Adjerid model in that it uses the same steady state formulation, respects vehicle dynamics, and uses very similar collision-avoidance strategies to ensure safe following distances between vehicles. Besides offering a better fit to empirical data, the Fadhloun-Rakha model is inclusive of the following characteristics: (1) it models the driver throttle and brake pedal input; (2) it captures driver variability; (3) it allows for shorter than steady-state following distances when following faster leading vehicles; (4) it offers a much smoother acceleration profile; and (5) it explicitly captures driver perception and control inaccuracies and errors. Through a quantitative and qualitative evaluation using naturalistic data, the new model is demonstrated to outperform other state-of-the-practice car-following models. In fact, the model is proved to result in a significant decrease in the modeling error, and to generate trajectories that are highly consistent with the observed car-following behavior. The final part of this study investigates a case in which the driver is excluded and the vehicles are operating in a connected environment. This section aims to showcase a scenario in which the human-in-the-loop is deactivated through the development of a platooning strategy that governs the motion of connected cooperative multi-vehicle platoons. / Doctor of Philosophy / Even though the study of the longitudinal motion of vehicles spanned over several decades leading to the development of more precise and complex car-following models, an important aspect was constantly overlooked in those models. In fact, due to the complexity of modeling the human-in-the-loop, the vehicle and the driver were almost always assumed to represent a single entity. More specifically, ignoring driver behavior and integrating it to the vehicle allowed avoiding to deal with the challenges related to modeling human behavior. The difficulty of mathematically modeling the vehicle and the driver as two independent components rather than one unique system is due to two main reasons. First, there are numerous car models and types that make it difficult to determine the different parameters impacting the performance of the vehicle as they differ from vehicle to vehicle. Second, different driving patterns exist and the fact that they are mostly dependent on human behavior and psychology makes them very difficult to replicate mathematically. The research presented in this thesis provides a comprehensive investigation of the human-in-the-loop component in car-following theory leading to a better understanding of the human-vehicle interaction. This study was initiated due to the noticeable overlooking of driver behavior in the existing literature which, as a result, fails to capture the effect of human control and perception errors.

Advances in Information and Communication Technology (ICT) paved the way for decentralized Heating, Ventilation, and Air-Conditioning (HVAC) HVAC operations. It has been envisioned that development of personal thermal comfort profiles leads to accurate predictions of each occupant's thermal comfort state and such information is employed in context-aware HVAC operations for energy efficiency. This dissertation has three key contributions in realizing this envisioned HVAC operation. First, it presents a systematic review of research trends and developments in context-aware HVAC operations. Second, it contributes to expanding the feasibility of the envisioned HVAC operation by introducing novel sensing technologies. Third, it contributes to shedding light on viability and potentials of comfort-aware operations (i.e., integrating personal thermal comfort models into HVAC control logic) through a comprehensive assessment of energy efficiency implications. In the first contribution, by developing a taxonomy, two major modalities – occupancy-driven and comfort-aware operations – in Human-In-The-Loop (HITL) HVAC operations were identified and reviewed quantitatively and qualitatively. The synthesis of previous studies has indicated that field evaluations of occupancy-driven operations showed lower potentials in energy saving, compared to the ones with comfort-aware operations. However, the results in comfort-aware operations could be biased given the small number of explorations. Moreover, required data representation schema have been presented to foster constructive performance assessments across different research efforts. In the end, the current state of research and future directions of HITL HVAC operations were discussed to shed light on future research need. As the second contribution, moving toward expanding the feasibility of comfort-aware operations, novel and smart sensing solutions have been introduced. It has been noted that, in order to have high accuracy in predicting individual's thermal comfort state (≥90%), user physiological response data play a key part. However, the limited number of applicable sensing technologies (e.g., infrared cameras) has impeded the potentials of implementation. After defining required characteristics in physiological sensing solutions in context of comfort-aware operations (applicability, sensitivity, ubiquity, and non-intrusiveness), the potentials of RGB cameras, Doppler radar sensors, and heat flux sensors were evaluated. RGB cameras, available in many smart computing devices, could be a ubiquitous solution in quantifying thermoregulation states. Leveraging the mechanism of skin blood perfusion, two thermoregulation state quantification methods have been developed. Then, applicability and sensitivity were checked with two experimental studies. In the first experimental study aiming to see applicability (distinguishing between 20 and 30C with fully acclimated human bodies), for 16 out of 18 human subjects, an increase in their blood perfusion was observed. In the second experimental study aiming to evaluate sensitivity (distinguishing responses to a continuous variation of air temperature from 20 to 30C), 10 out of 15 subjects showed a positive correlation between blood perfusion and thermal sensations. Also, the superiority of heat flux data, compared to skin temperature data, has been demonstrated in predicting personal thermal comfort states through the developments of machine-learning-based prediction models with feature engineering. Specifically, with random forest classifier, the median value of prediction accuracy was improved by 3.8%. Lastly, Doppler radar sensors were evaluated for their capability of quantifying user thermoregulation states leveraging the periodic movement of the chest/abdomen area induced by respiration. In an experimental study, the results showed that, with sufficient acclimation time, the DRS-based approach could show distinction between respiration states for two distinct air temperatures (20 and 30C). On the other hand, in a transient temperature without acclimation time, it was shown that, some of the human subjects (38.9%) used respiration as an active means of heat exchange for thermoregulation. Lastly, a comprehensive evaluation of comfort-aware operations' performance was carried out with a diverse set of contextual and operational factors. First, a novel comfort-aware operation strategy was introduced to leverage personal sensitivity to thermal comfort (i.e., different responses to temperature changes; e.g., sensitive to being cold) in optimization. By developing an agent-based simulation framework and thorough diverse scenarios with different numbers and combinations of occupants (i.e., human agents in the simulation), it was shown that this approach is superior in generating collectively satisfying environments against other approaches focusing on individual preferred temperatures in selection of optimized setpoints. The energy implications of comfort-aware operations were also evaluated to understand the impact from a wide range of factors (e.g., human and building factors) and their combinatorial effect given the uncertainty of multioccupancy scenarios. The results demonstrated that characteristics of occupants' thermal comfort profiles are dominant in impacting the energy use patterns, followed by the number of occupants, and the operational strategies. In addition, when it comes to energy efficiency, more occupants in a thermal zone/building result in reducing the efficacy of comfort-driven operation (i.e., the integration of personal thermal comfort profiles). Hence, this study provided a better understanding of true viability of comfort-driven HVAC operations and provided the probabilistic bounds of energy saving potentials. These series of studies have been presented as seven journal articles and they are included in this dissertation. / Doctor of Philosophy / With vision of a smart built environment, capable of understanding the contextual dynamics of built environment and adaptively adjusting its operation, this dissertation contributes to context-aware/decentralized HVAC operations. Three key contributions in realization of this goal include: (1) a systematic review of research trends and developments in the last decade, (2) enhancing the feasibility of quantifying personal thermal comfort by presenting novel sensing solutions, and (3) a comprehensive assessment of energy efficiency implications from comfort-aware HVAC operations with the use of personal comfort models. Starting from identifying two major modalities of context-aware HVAC operations, occupancy-driven and comfort-aware, the first part of this dissertation presents a quantitative and qualitative review and synthesis of the developments, trends, and remaining research questions in each modality. Field evaluation studies using occupancy-driven operations have shown median energy savings between 6% and 15% depending on the control approach. On the other hand, the comfort-aware HVAC operations have shown 20% energy savings, which were mainly derived from small-scale test beds in similar climate regions. From a qualitative technology development standpoint, the maturity of occupancy-driven technologies for field deployment could be interpreted to be higher than comfort-aware technologies while the latter has shown higher potentials. Moreover, by learning from the need for comparing different methods of operations, required data schemas have been proposed to foster better benchmarking and effective performance assessment across studies. The second part of this dissertation contributes to the cornerstone of comfort-aware operations by introducing novel physiological sensing solutions. Previous studies demonstrated that, in predicting individual's thermal comfort states, using physiological data in model development plays a key role in increasing accuracy (>90%). However, available sensing technologies in this context have been limited. Hence, after identifying essential characteristics for sensing solutions (applicability, sensitivity, ubiquity, and non-intrusiveness), the potentials of RGB cameras, heat flux sensors, and Doppler radar sensors were evaluated. RGB cameras, available in many smart devices, could be programmed to measure the level of blood flow to skin, regulated by the human thermoregulation mechanism. Accordingly, two thermoregulation states' quantification methods by using RGB video images have been developed and assessed under two experimental studies: (i) capturing subjects' facial videos in two opposite temperatures with sufficient acclimation time (20 and 30C), and (ii) capturing facial videos when subjects changed their thermal sensations in a continuous variation of air temperature from 20 to 30C. Promising results were observed in both situations. The first study had subjects and 16 of them showed an increasing trend in blood flow to skin. In the second study, posing more challenges due to insufficient acclimation time, 10 subjects had a positive correlation between the level of blood flow to skin with thermal sensation. With the assumption that heat flux sensing will be a better reflection of thermoregulation sates, a machine learning framework was developed and tested. The use of heat flux sensing showed an accuracy of 97% with an almost 4% improvement compared to skin temperature. Lastly, Doppler radar sensors were evaluated for their capability of quantifying thermoregulation states by detecting changes in breathing patterns. In an experimental study, the results showed that, with sufficient acclimation time, the DRS-based approach could show distinction between respiration states for two distinct air temperatures (20 and 30C). However, using a transient temperature was proven to be more challenging. It was noted that for some of the human subjects (38.9%), respiration was detected as an active means of heat exchange. It was concluded that specialized artifact removal algorithms might help improve the detection rate. The third component of the dissertation contributed by studying the performance of comfort-driven operations (i.e., using personal comfort preferences for HVAC operations) under a diverse set of contextual and operational factors. Diverse scenarios for interaction between occupants and building systems were evaluated by using different numbers and combinations of occupants, and it was demonstrated that an approach of addressing individual's thermal comfort sensitivity (personal thermal-comfort-related responses to temperature changes) outperforms other approaches solely focusing on individual preferred temperatures. The energy efficiency implications of comfort-driven operations were then evaluated by accounting for the impact of human and building factors (e.g., number of thermal zones) and their combinations. The results showed that characteristics of occupants' thermal comfort profiles are dominant in driving the energy use patterns, followed by the number of occupants, and operational strategies. As one of the main outcomes of this study, the energy saving and efficiency (energy use for comfort improvement) potentials and probabilistic bounds of comfort-driven operations were identified. It was shown that keeping the number of occupants low (under 6) in a thermal zone/building, boosts the energy saving potentials of comfort-driven operations. These series of studies have been presented as seven journal articles, included in this dissertation.

Humans learn to interact with many complex dynamic systems such as helicopters, bicycles, and automobiles. This dissertation develops a subsystem identification method to model the control strategies that human subjects use in experiments where they interact with dynamic systems. This work provides new results on the control strategies that humans learn. We present a novel subsystem identification algorithm, which can identify unknown linear time-invariant feedback and feedforward subsystems interconnected with a known linear time-invariant subsystem. These subsystem identification algorithms are analyzed in the cases of noiseless and noisy data. We present results from human-in-the-loop experiments, where human subjects in- teract with a dynamic system multiple times over several days. Each subject’s control behavior is assumed to have feedforward (or anticipatory) and feedback (or reactive) components, and is modeled using experimental data and the new subsystem identifi- cation algorithms. The best-fit models of the subjects’ behavior suggest that humans learn to control dynamic systems by approximating the inverse of the dynamic system in feedforward. This observation supports the internal model hypothesis in neuro- science. We also examine the impact of system zeros on a human’s ability to control a dynamic system, and on the control strategies that humans employ.

abstract: Allocating tasks for a day's or week's schedule is known to be a challenging and difficult problem. The problem intensifies by many folds in multi-agent settings. A planner or group of planners who decide such kind of task association schedule must have a comprehensive perspective on (1) the entire array of tasks to be scheduled (2) idea on constraints like importance cum order of tasks and (3) the individual abilities of the operators. One example of such kind of scheduling is the crew scheduling done for astronauts who will spend time at International Space Station (ISS). The schedule for the crew of ISS is decided before the mission starts. Human planners take part in the decision-making process to determine the timing of activities for multiple days for multiple crew members at ISS. Given the unpredictability of individual assignments and limitations identified with the various operators, deciding upon a satisfactory timetable is a challenging task. The objective of the current work is to develop an automated decision assistant that would assist human planners in coming up with an acceptable task schedule for the crew. At the same time, the decision assistant will also ensure that human planners are always in the driver's seat throughout this process of decision-making. The decision assistant will make use of automated planning technology to assist human planners. The guidelines of Naturalistic Decision Making (NDM) and the Human-In-The -Loop decision making were followed to make sure that the human is always in the driver's seat. The use cases considered are standard situations which come up during decision-making in crew-scheduling. The effectiveness of automated decision assistance was evaluated by setting it up for domain experts on a comparable domain of scheduling courses for master students. The results of the user study evaluating the effectiveness of automated decision support were subsequently published. / Dissertation/Thesis / Masters Thesis Computer Science 2019

We present INCREMENT, a cluster refinement algorithm which utilizes user feedback to refine clusterings. INCREMENT is capable of improving clusterings produced by arbitrary clustering algorithms. The initial clustering provided is first sub-clustered to improve query efficiency. A small set of select instances from each of these sub-clusters are presented to a user for labelling. Utilizing the user feedback, INCREMENT trains a feature embedder to map the input features to a new feature space. This space is learned such that spatial distance is inversely correlated with semantic similarity, determined from the user feedback. A final clustering is then formed in the embedded space. INCREMENT is tested on 9 datasets initially clustered with 4 distinct clustering algorithms. INCREMENT improved the accuracy of 71% of the initial clusterings with respect to a target clustering. For all the experiments the median percent improvement is 27.3% for V-Measure and is 6.08% for accuracy.

Humans learn to interact with many complex physical systems. For example, humans learn to fly aircraft, operate drones, and drive automobiles. We present results from human-in-the-loop (HITL) experiments, where human subjects interact with dynamic systems while performing command-following tasks multiple times over a one-week period. We use a new subsystem identification (SSID) algorithm to estimate the control strategies (feedforward, feedforward delay, feedback, and feedback delay) that human subjects use during their trials. We use experimental and SSID results to examine the effects of system characteristics (e.g., system zeros, relative degree, system order, phase lag, time delay), reference command, and command-following objectives on humans command-following performance and on the control strategies that the humans learn. Results suggest that nonminimum-phase zeros, relative degree, phase lag, and time delay tend to make dynamic systems difficult for human to control. Subjects can generalize their control strategies from one task to another and use prediction of the reference command to improve their command-following performance. However, this dissertation also provides evidence that humans can learn to improve performance without prediction. This dissertation also presents a new SSID algorithm to model the control strategies that human subjects use in HITL experiments where they interact with dynamic systems. This SSID algorithm uses a two-candidate-pool multi-convex-optimization approach to identify feedback-and-feedforward subsystems with time delay that are interconnected in closed loop with a known subsystem. This SSID method is used to analyze the human control behavior in the HITL experiments discussed above.

<p dir="ltr"> Most industrial processes are not fully-automated. Although reference tracking can be handled by low-level controllers, initializing and adjusting the reference, or setpoint, values, are commonly tasks assigned to human operators. A major challenge that arises, though, is control policy variation among operators which in turn results in inconsistencies in the final product. In order to guide operators to pursue better and more consistent performance, researchers have explored the optimal control policy through different approaches. Although in different applications, researchers use different approaches, an accurate process model is still crucial to the approaches. However, for a highly transient process (e.g., the startup of a manufacturing process), modeling can be challenging and inaccurate, and approaches highly relying on a process model may not work well. One example is process startup in a twin-roll steel strip casting process and motivates this work. </p><p dir="ltr"><br></p><p dir="ltr"> In this dissertation, I propose three offline reinforcement learning (RL) algorithms which require the RL agent to learn a control policy from a fixed dataset that is pre-collected by human operators during operations of the twin-roll casting process. Compared to existing offline RL algorithms, the proposed algorithms focus on exploiting the best control policy used by human operators rather than exploring new control policies constrained by the existing policies. In addition, in existing offline RL algorithms, there is not enough consideration of the imbalanced dataset problem. In the second and the third proposed algorithms, I leverage the idea of cost sensitive learning to incentivize the RL agent to learn the most valuable control policy, rather than the most common one represented in the dataset. In addition, since the process model is not available, I propose a performance metric that does not require a process model or simulator for agent testing. The third proposed algorithm is compared with benchmark offline RL algorithms and achieves better and more consistent performance.</p>

With the increase of robotic presence in our homes and work environment, it has become imperative to consider human-in-the-loop systems when designing robotic controllers. This includes both a physical presence of humans as well as interaction on a decision and control level. One important aspect of this is to design controllers which are guaranteed to satisfy specified safety constraints. At the same time we must minimize the risk of not finding solutions, which would force the system to stop. This require some room for relaxation to be put on the specifications. Another aspect is to design the system to be adaptive to the human and its environment. In this thesis we approach the problem by considering control synthesis for multi-agent systems under hard and soft constraints, where the human has direct impact on how the soft constraint is violated. To handle the multi-agent structure we consider both a classical centralized automata based framework and a decentralized approach with collision avoidance. To handle soft constraints we introduce a novel metric; hybrid distance, which quantify the violation. The hybrid distance consists of two types of violation; continuous distance or missing deadlines, and discrete distance or spacial violation. These distances are weighed against each other with a weight constant we will denote as the human preference constant. For the human impact we consider two types of feedback; direct feedback on the violation in the form of determining the human preference constant, and direct control input through mixed-initiative control where the human preference constant is determined through an inverse reinforcement learning algorithm based on the suggested and followed paths. The methods are validated through simulations. / I takt med att robotar blir allt vanligare i våra hem och i våra arbetsmiljöer, har det blivit allt viktigare att ta hänsyn till människan plats i systemen när regulatorerna för robotorna designas. Detta innefattar både människans fysiska närvaro och interaktion på besluts- och reglernivå. En viktig aspekt i detta är att designa regulatorer som garanterat uppfyller givna villkor. Samtidigt måste vi minimera risken att ingen lösning hittas, eftersom det skulle tvinga systemet till ett stopp. För att uppnå detta krävs det att det finns rum för att mjuka upp villkoren. En annan aspekt är att designa systemet så att det är anpassningsbart till människan och miljön. I den här uppsatsen närmar vi oss problemet genom att använda regulator syntes för multi-agent system under hårda och mjuka villkor där människan har direkt påverkan på hur det svaga villkoret överträds. För att hantera multi-agent strukturen undersöker vi både det klassiska centraliserade automata-baserade ramverket och ett icke-centraliserat tillvägagångsätt med krockundvikning. För att hantera mjuka villkor introducerar vi en metrik; hybrida avståndet, som kvantifierar överträdelsen. Det hybrida avståndet består av två typer av överträdelse (kontinuerligt avstånd eller missandet av deadlines, och diskret avstånd eller rumsliga överträdelser) som vägs mot varandra med en vikt konstant som vi kommer att kalla den mänskliga preferens kontanten. Som mänsklig påverkan överväger vi direkt feedback på överträdelsen genom att hon bestämmer värdet på den mänskliga preferens kontanten, och direkt påverkan på regulatorn där den mänskliga preferens konstanten bestäms genom en inverserad förstärkt inlärnings algoritm baserad på de föreslagna och följda vägarna. Metoderna valideras genom simuleringar. / <p>QC20190517</p>