The advent of fast and economical computers with large electronic storage has led to a large volume of data, most of which is unlabeled. While computers provide expeditious, accurate and low-cost computation, they still lag behind in many tasks that require human intelligence such as labeling medical images, videos or text. Consequently, current research focuses on a combination of computer accuracy and human intelligence to complete labeling task. In most cases labeling needs to be done by domain experts, however, because of the variability in expertise, experience, and intelligence of human beings, experts can be scarce. As an alternative to using domain experts, help is sought from non-experts, also known as Crowd, to complete tasks that cannot be readily automated. Since crowd labelers are non-expert, multiple labels per instance are acquired for quality purposes. The final label is obtained by com- bining these multiple labels. It is very common that the ground truth, instance difficulty, and the labeler ability are unknown entities. Therefore, the aggregation task becomes a “chicken and egg” problem to start with. Despite the fact that much research using machine learning and statistical techniques has been conducted in this area (e.g., [Dekel and Shamir, 2009; Hovy et al., 2013a; Liu et al., 2012; Donmez and Carbonell, 2008]), many questions remain unresolved, these include: (a) What are the best ways to evaluate labelers? (b) It is common to use expert-labeled instances (ground truth) to evaluate la- beler ability (e.g., [Le et al., 2010; Khattak and Salleb-Aouissi, 2011; Khattak and Salleb-Aouissi, 2012; Khattak and Salleb-Aouissi, 2013]). The question is, what should be the cardinality of the set of expert-labeled instances to have an accurate evaluation? (c) Which factors other than labeler expertise (e.g., difficulty of instance, prevalence of class, bias of a labeler toward a particular class) can affect the labeling accuracy? (d) Is there any optimal way to combine multiple labels to get the best labeling accuracy? (e) Should the labels provided by oppositional/malicious labelers be dis- carded and blocked? Or is there a way to use the “information” provided by oppositional/malicious labelers? (f) How can labelers and instances be evaluated if the ground truth is not known with certitude? In this thesis, we investigate these questions. We present methods that rely on few expert-labeled instances (usually 0.1% -10% of the dataset) to evaluate various parameters using a frequentist and a Bayesian approach. The estimated parameters are then used for label aggregation to produce one final label per instance. In the first part of this thesis, we propose a method called Expert Label Injected Crowd Esti- mation (ELICE) and extend it to different versions and variants. ELICE is based on a frequentist approach for estimating the underlying parameters. The first version of ELICE estimates the pa- rameters i.e., labeler expertise and data instance difficulty, using the accuracy of crowd labelers on expert-labeled instances [Khattak and Salleb-Aouissi, 2011; Khattak and Salleb-Aouissi, 2012]. The multiple labels for each instance are combined using weighted majority voting. These weights are the scores of labeler reliability on any given instance, which are obtained by inputting the pa- rameters in the logistic function. In the second version of ELICE [Khattak and Salleb-Aouissi, 2013], we introduce entropy as a way to estimate the uncertainty of labeling. This provides an advantage of differentiating between good, random and oppositional/malicious labelers. The aggregation of labels for ELICE version 2 flips the label (for binary classification) provided by the oppositional/malicious labeler thus utilizing the information that is generally discarded by other labeling methodologies. Both versions of ELICE have a cluster-based variant in which rather than making a random choice of instances from the whole dataset, clusters of data are first formed using any clustering approach e.g., K-means. Then an equal number of instances from each cluster are chosen randomly to get expert-labels. This is done to ensure equal representation of each class in the test dataset. Besides taking advantage of expert-labeled instances, the third version of ELICE [Khattak and Salleb-Aouissi, 2016], incorporates pairwise/circular comparison of labelers to labelers and in- stances to instances. The idea here is to improve accuracy by using the crowd labels, which unlike expert-labels, are available for the whole dataset and may provide a more comprehensive view of the labeler ability and instance difficulty. This is especially helpful for the case when the domain experts do not agree on one label and ground truth is not known for certain. Therefore, incorporating more information beyond expert labels can provide better results. We test the performance of ELICE on simulated labels as well as real labels obtained from Amazon Mechanical Turk. Results show that ELICE is effective as compared to state-of-the-art methods. All versions and variants of ELICE are capable of delaying phase transition. The main contribution of ELICE is that it makes the use of all possible information available from crowd and experts. Next, we also present a theoretical framework to estimate the number of expert-labeled instances needed to achieve certain labeling accuracy. Experiments are presented to demonstrate the utility of the theoretical bound. In the second part of this thesis, we present Crowd Labeling Using Bayesian Statistics (CLUBS) [Khattak and Salleb-Aouissi, 2015; Khattak et al., 2016b; Khattak et al., 2016a], a new approach for crowd labeling to estimate labeler and instance parameters along with label aggregation. Our approach is inspired by Item Response Theory (IRT). We introduce new parameters and refine the existing IRT parameters to fit the crowd labeling scenario. The main challenge is that unlike IRT, in the crowd labeling case, the ground truth is not known and has to be estimated based on the parameters. To overcome this challenge, we acquire expert-labels for a small fraction of instances in the dataset. Our model estimates the parameters based on the expert-labeled instances. The estimated parameters are used for weighted aggregation of crowd labels for the rest of the dataset. Experiments conducted on synthetic data and real datasets with heterogeneous quality crowd-labels show that our methods perform better than many state-of-the-art crowd labeling methods. We also conduct significance tests between our methods and other state-of-the-art methods to check the significance of the accuracy of these methods. The results show the superiority of our method in most cases. Moreover, we present experiments to demonstrate the impact of the accuracy of final aggregated labels when used as training data. The results essentially emphasize the need for high accuracy of the aggregated labels. In the last part of the thesis, we present past and contemporary research related to crowd la- beling. We conclude with future of crowd labeling and further research directions. To summarize, in this thesis, we have investigated different methods for estimating crowd labeling parameters and using them for label aggregation. We hope that our contribution will be useful to the crowd labeling community.

The present study used an experimental design to investigate the efficacy of using short (12 words or less), prominently placed supplemental labels to increase the effectiveness of select extant labels in museum exhibits. The experimenter-developed supplemental labels were designed to leverage exogenous/bottom-up and endogenous/top-down sources of influence on selective attention. Measures of patron behavior, knowledge retention, and attitude found no significant differences between group means under control and treatment conditions. These outcomes were surprising and inconsistent with findings from similar research conducted by Hirschi and Screven. The supplemental labels in the present study might have failed to capture attention because they were not sufficiently visually stimulating, they did not sufficiently tap internal motivations, or perhaps patrons experienced innattentional blindness in regards to them.

This research explored social worker’s perception of the services received among patients labeled with a diagnosis or labeled negatively, such as non-compliant, in comparison to non-labeled patients in a medical setting. Data for this project were gathered through seven in person interviews with social workers. The participants were all social workers in a medical setting at different DaVita Dialysis centers throughout San Bernardino County. The participants experience and education level ranged from master level social work interns to licensed clinical social workers. The findings indicated that the social workers do believe patients with diagnoses do not receive the same level of care as patients without a diagnosis or label.

Thesis(Ph.D.)--Case Western Reserve University, 2010 / Title from PDF (viewed on 2010-01-28) Department of Social Welfare Includes abstract Includes bibliographical references and appendices Available online via the OhioLINK ETD Center

Thesis (Ph. D.)--University of Washington, 2003. / Vita. Includes bibliographical references (leaves 148-174).

This study examined the relationship between ethnic identity and ethnic self-labels, and their associations with adolescents’ psychosocial outcomes (perceived self-concept, self-reported grades, conduct disorder symptoms), in a diverse sample of 759 adolescents (52% females; 46% Latino, 35% Black, 13% White, and 6% of another race) aged 12 to 20 years in New York City). To further elucidate these relationships, the role of parental ethnic/racial socialization and age were also examined. Regression analyses revealed that ethnic identity, parental socialization, and age all had significant associations with each other and with psychosocial outcomes. There were no significant associations between ethnic self-labels and ethnic identity, behavioral symptoms or social competence. However, adolescents who used hyphenated ethnic self-labels reported higher academic achievement. Though the hypothesis that ethnic self-labels would predict adolescent outcomes was not supported, they suggest the need for greater accuracy in determining ethnic self-labels and delineation of ethnic identity. / text

Currently, nutritional labelling is difficult to interpret and time-consuming to read. This is a major problem as many consumers are overweight and resort to eating readymade meals and snacks. These are likely to be energy-dense food and beverages that are high in fat, sugar and artificial preservatives. Simplifying nutritional labels could help stem rising obesity rates. Front-of-pack labels are a tool to help overcome this problem by providing consumers with understandable, visible information to aid them into making healthier food choices. This study expands on past research by evaluating 7 separate pre-existing, proposed and fictitious front-of-pack nutritional labels. It includes Information, Image or a combination of both Information and Image based labels. Plus No label, which is a control variable to determine the effectiveness of each label. The nutritional labels were placed on a chicken salad sandwich which was kept consistent for all 14 manipulations. The nutritional components were altered to reflect either an Unhealthy or Healthy sandwich. The design of this experiment is a 2 (nutritional level: Healthy, Unhealthy) X7 (labelling system: Traffic Light, Star, Running, Walking, Third Party, Daily Intake, Caloric, None) between subjects design. The results provide evidence of the urgent need to communicate nutritional information more effectively. Images, simplicity, colour and reliability, are determining label elements that influence consumption behaviour. The results from this study help to understand behaviours associated to labels. This study draws differences between those who partake in health behaviours and those who do not. This information could help to trigger support for a new, more effective front-of-pack labelling system to be put in place globally to guide consumers in making healthier food choices.

Information regarding the structure and function of living tissues and cells is instrumental to the advancement of cell biology and biophysics. Nonlinear optical microscopy can provide such information, but only certain biological structures generate nonlinear optical signals. Therefore, structural specificity can be achieved by introducing labels for nonlinear optical microscopy. Few studies exist in the literature about labels that facilitate harmonic generation, coined "harmonophores". This thesis consists of the first major investigation of harmonophores for third harmonic generation (THG) microscopy. Carotenoids and chlorophylls were investigated as potential harmonophores. Their nonlinear optical properties were studied by the THG ratio technique. In addition, a tunable refractometer was built in order to determine their second hyperpolarizability (γ). At 830 nm excitation wavelength, carotenoids and chlorophylls were found to have large negative γ values however, at 1028 nm, the sign of γ reversed for carotenoids and remained negative for chlorophylls. Consequently, at 1028 nm wavelength, THG signal is canceled with mixtures of carotenoids and chlorophylls. Furthermore, when such molecules are covalently bonded as dyads or interact within photosynthetic pigment-protein complexes, it is found that additive effects with the γ values still play a role, however, the overall γ value is also influenced by the intra-pigment and inter-pigment interaction. The nonlinear optical properties of aggregates containing chlorophylls and carotenoids were the target of subsequent investigations. Carotenoid aggregates were imaged with polarization-dependent second harmonic generation and THG microscopy. Both techniques revealed crystallographic information pertaining to H and J aggregates and β-carotene crystalline aggregates found in orange carrot. In order to demonstrate THG enhancement due to labeling, cultured cells were labeled with carotenoid incorporated liposomes. In addition, Drosophila melanogaster larvae muscle as well as keratin structures in the hair cortex were labeled with β-carotene. Polarization-dependent THG studies may be particularly useful in understanding the structural organization that occurs within biological structures containing carotenoids and chlorophylls such as photosynthetic pigment-protein complexes and carotenoid aggregates in plants and alga. Further, artificial labeling with carotenoids and chlorophylls may be useful in clinical applications since they are nontoxic, nutritionally valuable, and they can aid in visualizing structural changes in cellular components.

Thesis (M. Consumer Science)--North-West University, Potchefstroom Campus, 2009.

Thesis (M. Consumer Science)--North-West University, Potchefstroom Campus, 2009.