Deep learning has achieved classification performance matching or exceeding the human one, as long as plentiful labeled training samples are available. However, the performance on few-shot learning, where the classifier had seen only several or possibly only one sample of the class is still significantly below human performance. Recently, a type of algorithm called meta-learning achieved impressive performance for few-shot learning. However, meta-learning requires a large dataset of labeled tasks closely related to the test task. The work described in this dissertation outlines techniques that significantly reduce the need for expensive and scarce labeled data in the meta-learning phase. Our insight is that meta-training datasets require only in-class samples (samples belonging to the same class) and out-of-class samples. The actual labels associated with the classes are not relevant, as they are not retained in the meta-learning process. First, we propose an algorithm called UMTRA that generates out-of-class samples using random sampling from an unlabeled dataset, and generates in-class samples using augmentation. We show that UMTRA achieves a large fraction of the accuracy of supervised meta-learning, while using orders of magnitudes less labeled data. Second, we note that the augmentation step in UMTRA works best when an augmentation technology specific to the domain is used. In many practical cases it is easier to train a generative model for a domain than to find an augmentation algorithm. From this idea, we design a new unsupervised meta-learning algorithm called LASIUM, where the in- and out-of-class samples for the meta-learning step are generated by choosing appropriate points in the latent space of a generative model (such as a variational autoencoder or generative adversarial network). Finally, we describe work that makes progress towards a next step in meta-learning, the ability to draw the meta-training samples from a different domain from the target task's domain.

Intelligent virtual agents (IVAs) have been researched for years and recently many of these IVAs have become commercialized and widely used by many individuals as intelligent personal assistants. The majority of these IVAs are anthropomorphic, and many are developed to resemble real humans entirely. However, real humans do not interact only with other humans in the real world, and many benefit from interactions with non-human entities. A prime example is human interactions with animals, such as dogs. Humans and dogs share a historical bond that goes back thousands of years. In the past 30 years, there has been a great deal of research to understand the effects of human-dog interaction, with research findings pointing towards the physical, mental, and social benefits to humans when interacting with dogs. However, limitations such as allergies, stress on dogs, and hygiene issues restrict some needy individuals from receiving such benefits. More recently, advances in augmented and virtual reality technology provide opportunities for realizing virtual dogs and animals, allowing for their three-dimensional presence in the users' real physical environment or while users are immersed in virtual worlds. In this dissertation, I utilize the findings from human-dog interaction research and conduct a systematic literature review on embodied IVAs to define a research scope to understand virtual dogs' social and behavioral influences in augmented and virtual reality. I present the findings of this systematic literature review that informed the creation of the research scope and four human-subjects studies. Through these user studies, I found that virtual dogs bring about a sense of comfort and companionship for users in different contexts. In addition, their responsiveness plays an important role in enhancing users' quality of experience, and they can be effectively utilized as attention guidance mechanisms and social priming stimuli.

The success of deep neural networks in a variety of computer vision tasks heavily relies on large- scale datasets. However, it is expensive to manually acquire labels for large datasets. Given the human annotation cost and scarcity of data, the challenge is to learn efficiently with insufficiently labeled data. In this dissertation, we propose several approaches towards data-efficient learning in the context of few-shot learning, long-tailed visual recognition, and unsupervised and semi-supervised learning. In the first part, we propose a novel paradigm of Task-Agnostic Meta- Learning (TAML) algorithms to improve few-shot learning. Furthermore, in the second part, we analyze the long-tailed problem from a domain adaptation perspective and propose to augment the classic class-balanced learning for longtails by explicitly estimating the differences between the class-conditioned distributions with a meta-learning approach. Following this, we propose our lazy approach based on an intuitive teacher-student scheme to enable the gradient-based meta- learning algorithms to explore long horizons. Finally, in the third part, we propose a novel face detector adaptation approach that is applicable whenever the target domain supplies many representative images, no matter they are labeled or not. Experiments on several benchmark datasets verify the efficacy of the proposed methods under all settings.

During the past decade, there has been a resurgence of interest in applying mathematical methods to problems in artificial intelligence. Much work has been done in the field of machine learning, but it is not always clear how the results of this research should be applied to practical problems. Our aim is to help bridge the gap between theory and practice by addressing the question: "If we are given a machine learning algorithm, how should we go about formally analyzing it?" as opposed to the usual question: "how do we write a learning algorithm we can analyze?".;We will consider algorithms that accept randomly drawn training data as input, and produce classification rules as their outputs. For the most part our analyses will be based on the syntactic structure of these classification rules; for example, if we know that the algorithm we want to analyze will only output logical expressions that are conjunctions of variables, we can use this fact to facilitate our analysis.;We use a probabilistic framework for machine learning, often called the pac model. In this framework, one asks whether or not a machine learning algorithm has a high probability of generating classification rules that "usually" make the right classification (pac means probably approximately correct). Research in the pac framework can be divided into two subfields. The first field is concerned with the amount of training data that is needed for successful learning to take place (success being defined in terms of generalization ability); the second field is concerned with the computational complexity of learning once the training data have been selected. Since most existing algorithms use heuristics to deal with the problem of complexity, we are primarily concerned with the amount of training data that algorithms require.

Adoption of deep neural networks (DNNs) into safety-critical and high-assurance systems has been hindered by the inability of DNNs to handle adversarial and out-of-distribution input. State-of-the-art DNNs misclassify adversarial input and give high confidence output for out-of-distribution input. We attempt to solve this problem by employing two approaches, first, by detecting adversarial input and, second, by developing a confidence metric that can indicate when a DNN system has reached its limits and is not performing to the desired specifications. The effectiveness of our method at detecting adversarial input is demonstrated against the popular DeepFool adversarial image generation method. On a benchmark of 50,000 randomly chosen ImageNet adversarial images generated for CaffeNet and GoogLeNet DNNs, our method can recover the correct label with 95.76% and 97.43% accuracy, respectively. The proposed attribution-based confidence (ABC) metric utilizes attributions used to explain DNN output to characterize whether an output corresponding to an input to the DNN can be trusted. The attribution based approach removes the need to store training or test data or to train an ensemble of models to obtain confidence scores. Hence, the ABC metric can be used when only the trained DNN is available during inference. We test the effectiveness of the ABC metric against both adversarial and out-of-distribution input. We experimental demonstrate that the ABC metric is high for ImageNet input and low for adversarial input generated by FGSM, PGD, DeepFool, CW, and adversarial patch methods. For a DNN trained on MNIST images, ABC metric is high for in-distribution MNIST input and low for out-of-distribution Fashion-MNIST and notMNIST input.

Interactive Virtual Humans (VHs) are increasingly used to replace or assist real humans in various applications, e.g., military and medical training, education, or entertainment. In most VH research, the perceived social presence with a VH, which denotes the user's sense of being socially connected or co-located with the VH, is the decisive factor in evaluating the social influence of the VH—a phenomenon where human users' emotions, opinions, or behaviors are affected by the VH. The purpose of this dissertation is to develop new knowledge about how characteristics and behaviors of a VH in a Mixed Reality (MR) environment can affect the perception of and resulting behavior with the VH, and to find effective and efficient ways to improve the quality and performance of social interactions with VHs. Important issues and challenges in real–virtual human interactions in MR, e.g., lack of physical–virtual interaction, are identified and discussed through several user studies incorporating interactions with VH systems. In the studies, different features of VHs are prototyped and evaluated, such as a VH's ability to be aware of and influence the surrounding physical environment, while measuring objective behavioral data as well as collecting subjective responses from the participants. The results from the studies support the idea that the VH's awareness and influence of the physical environment can improve not only the perceived social presence with the VH, but also the trustworthiness of the VH within a social context. The findings will contribute towards designing more influential VHs that can benefit a wide range of simulation and training applications for which a high level of social realism is important, and that can be more easily incorporated into our daily lives as social companions, providing reliable relationships and convenience in assisting with daily tasks.

In this dissertation, we propose novel machine learning algorithms for high-risk medical imaging applications. Specifically, we tackle current challenges in radiology screening process and introduce cutting-edge methods for image-based diagnosis, detection and segmentation. We incorporate expert knowledge through eye-tracking, making the whole process human-centered. This dissertation contributes to machine learning, computer vision, and medical imaging research by: 1) introducing a mathematical formulation of radiologists level of attention, and sparsifying their gaze data for a better extraction and comparison of search patterns. 2) proposing novel, local and global, image analysis algorithms. Imaging based diagnosis and pattern analysis are "high-risk" Artificial Intelligence applications. A standard radiology screening procedure includes detection, diagnosis and measurement (often done with segmentation) of abnormalities. We hypothesize that having a true collaboration is essential for a better control mechanism, in such applications. In this regard, we propose to form a collaboration medium between radiologists and machine learning algorithms through eye-tracking. Further, we build a generic platform consisting of novel machine learning algorithms for each of these tasks. Our collaborative algorithm utilizes eye tracking and includes an attention model and gaze-pattern analysis, based on data clustering and graph sparsification. Then, we present a semi-supervised multi-task network for local analysis of image in radiologists' ROIs, extracted in the previous step. To address missing tumors and analyze regions that are completely missed by radiologists during screening, we introduce a detection framework, S4ND: Single Shot Single Scale Lung Nodule Detection. Our proposed detection algorithm is specifically designed to handle tiny abnormalities in lungs, which are easy to miss by radiologists. Finally, we introduce a novel projective adversarial framework, PAN: Projective Adversarial Network for Medical Image Segmentation, for segmenting complex 3D structures/organs, which can be beneficial in the screening process by guiding radiologists search areas through segmentation of desired structure/organ.

The recent advances in the field of Artificial Intelligence (AI), particularly Generative Adversarial Networks (GANs) and an abundance of training samples along with robust computational resources have significantly propelled the field of AI-generated fake information in all kinds, e.g., deepfakes. Deepfakes are among the most sinister types of misinformation, posing large-scale and severe security and privacy risks targeting critical governmental institutions and ordinary people across the world. The fact that deepfakes are AI-generated digital content and not actual events captured by a camera implies that they still can be detected using advanced AI models. Although the deepfake detection task has gained massive attention within the last couple of years, the mainstream detection frameworks mainly rely on Convolutional Neural Networks (CNNs). In deepfake detection tasks, it is critically important to successfully identify forged pixels to extract better discriminative features in a scalable manner. One of our works demonstrated that the performance of the CNN models could be improved through attention-based mechanisms, which forces the model to learn more discriminative features. Although CNNs have proven themselves solid candidates for learning local information of the image, they still miss capturing pixels' spatial interdependence due to constrained receptive fields. While CNNs fail to learn relative spatial information and lose essential data in pooling layers, vision transformers' global attention mechanism enables the network to learn higher-level information much faster. Therefore, a multi-stream deepfake detection framework is presented that incorporates pixels' spatial interdependence in a global context with local image features in a scalable scheme using unique characteristics of transformer models on learning the global relationship of pixels. Furthermore, this work proposes a framework at the intersection of graph theory, attention analysis, and vision transformers to overcome the shortcomings of previous approaches. The successful outcome of this study will help better detection of deepfakes in less computational cost compared to previous studies.

Conditional preference networks (CP-nets) model user preferences over objects described in terms of values assigned to discrete features, where the preference for one feature may depend on the values of other features. Most existing algorithms for learning CP-nets from the user's choices assume that the user chooses between pairs of objects. However, many real-world applications involve the the user choosing from all combinatorial possibilities or a very large subset. We introduce a CP-net learning algorithm for the latter type of choice, and study its properties formally and empirically.

The advancements on the Internet have enabled connecting more devices into this technology every day. This great connectivity has led to the introduction of the internet of things (IoTs) that is a great bed for engagement of all new technologies for computing devices and systems. Nowadays, the IoT devices and systems have applications in many sensitive areas including military systems. These challenges target hardware and software elements of IoT devices and systems. Integration of hardware and software elements leads to hardware systems and software systems in the IoT platforms, respectively. A recent trend for the hardware systems is making them trustworthy and energy efficient. On the other hand, the trend for software systems is making them intelligent and secure. The hardware elements are made energy efficient through implementation of them using emerging transistor and memory technologies. The artificial intelligence (AI) techniques can be utilized in the design and development of software elements. In order to enhance security of the software and the hardware elements, possible threats and countermeasures for them need to be researched and introduced into the community. Globalization of the computing systems and making them Internet-connected introduces diverse set of security threats and malicious activities. These security problems bring detrimental impacts and catastrophic consequences into the networks and systems. In this regard, we address these problems in the IoT world from both hardware and software perspectives. In order to address the emerging security problems in the hardware, we design and develop threats and countermeasures for two different types of Analog to Digital Converter (ADC). This is the first attempt in introducing ADC into the security context. Our findings show that lack of considering the security of ADCs, their performance and functionality can be remarkably degraded due to the payloads of possible attacks. For addressing the ongoing software security problems, we propose: (a) the AI-based software that can help in countering certain attacks; and (b) the techniques for protecting AI-based software against launching attacks on them. We found enhancing the defense systems with AI caters major improvements in detecting malicious information and recognizing the identities. Additionally, we found protection of the AI-based software against functionality manipulative data (a.k.a. adversarial examples) is realized through engaging multiple elements in system training and improving its classification knowledge.