<p>Neural networks provide state-of-the-art results for most machine learning tasks. Unfortunately, neural networks are vulnerable to test-time evasion attacks adversarial examples): inputs specifically designed by an adversary to cause a neural network to misclassify them. This makes applying neural networks in security-critical areas concerning. In this dissertation, we introduce a general framework for evaluating the robustness of neural network through optimization-based methods. We apply our framework to two different domains, image recognition and automatic speech recognition, and find it provides state-of-the-art results for both. To further demonstrate the power of our methods, we apply our attacks to break 14 defenses that have been proposed to alleviate adversarial examples. We then turn to the problem of designing a secure classifier. Given this apparently-fundamental vulnerability of neural networks to adversarial examples, instead of taking an existing classifier and attempting to make it robust, we construct a new classifier which is provably robust by design under a restricted threat model. We consider the domain of malware classification, and construct a neural network classifier that is can not be fooled by an insertion adversary, who can only insert new functionality, and not change existing functionality. We hope this dissertation will provide a useful starting point for both evaluating and constructing neural networks robust in the presence of an adversary.

<p> Social dynamics is concerned with the interactions of individuals and the resulting group behaviors. It models the temporal evolution of social systems via the interactions of the individuals within these systems. The availability of large-scale data in social networks and sensor networks offers an unprecedented opportunity to predict state changing events at the individual level. Examples of such events are disease infection, rumor propagation and opinion transition in elections, etc. Unlike previous research focusing on the collective effects of social systems, we want to make efficient inferences on the individual level.</p><p> Two main challenges are addressed: temporal modeling and computational complexity. The interaction pattern for each individual keeps changing over the time, i.e., an individual interacts with different individuals at different times. Second, as the number of tracked individual increases, the computational complexity grows exponentially with traditional sequential data analysis. </p><p> The contributions are: (i) leverage social networks and sensor networks data to make tractable inferences on both individual behaviors and collective effects in social dynamics. (ii) use the stochastic kinetic model to summarize dynamic interactions among individuals and simplify the state transition probabilities. (iii) propose an efficient variational inference algorithm whose complexity grows <i>linearly</i> with the number of tracked individuals <i> M</i>. Given the state space <i>K</i> of a single individual and the total number of time steps <i>T</i>, the complexity of naive brute-force approach is <i>O(K^MT)</i> and the complexity of existing exact inference approach is <i>O(K^MT)</i>. In comparison, the complexity of the proposed algorithm is<i> O(K²MT)</i>. In practice, it requires several iterations to converge. </p><p> In the empirical study concerning epidemics dynamics, given wireless sensor network data collected from more than ten thousand people (M = 13,888) over three years (T = 3465), we use the proposed algorithm to track disease transmission, and predict the probability of infection for each individual (K = 2) along the time until convergence (I=5). It is more efficient than state of the art sampling methods, i.e., MCMC and particle filter, while achieving high accuracy.</p>

<p> Automatic age estimation from real-world and wild face images is a challenging task and has an increasing importance due to its wide range of applications in current and future lifestyles. As a result of increasing age specific human-computer interactions, it is expected that computerized systems should be capable of estimating the age from face images and respond accordingly. Over the past decade, many research studies have been conducted on automatic age estimation from face images. </p><p> In this research, new approaches for enhancing age classification of a person from face images based on deep neural networks (DNNs) are proposed. The work shows that pre-trained CNNs which were trained on large benchmarks for different purposes can be retrained and fine-tuned for age estimation from unconstrained face images. Furthermore, an algorithm to reduce the dimension of the output of the last convolutional layer in pre-trained CNNs to improve the performance is developed. Moreover, two new jointly fine-tuned DNNs frameworks are proposed. The first framework fine-tunes tow DNNs with two different feature sets based on the element-wise summation of their last hidden layer outputs. While the second framework fine-tunes two DNNs based on a new cost function. For both frameworks, each has two DNNs, the first DNN is trained by using facial appearance features that are extracted by a well-trained model on face recognition, while the second DNN is trained on features that are based on the superpixels depth and their relationships. </p><p> Furthermore, a new method for selecting robust features based on the power of DNN and <i>l₂₁-norm</i> is proposed. This method is mainly based on a new cost function relating the DNN and the L21 norm in one unified framework. To learn and train this unified framework, the analysis and the proof for the convergence of the new objective function to solve minimization problem are studied. Finally, the performance of the proposed jointly fine-tuned networks and the proposed robust features are used to improve the age estimation from the facial images. The facial features concatenated with their corresponding robust features are fed to the first part of both networks and the superpixels features concatenated with their robust features are fed to the second part of the network </p><p> Experimental results on a public database show the effectiveness of the proposed methods and achieved the state-of-art performance on a public database. </p><p>

<p> Historically, most virtual human character research focuses on realism/emotions, interaction with humans, and discourse. The majority of the spatial positioning of characters has focused on one-on-one conversations with humans or placing virtual characters side-by-side when talking. These rely on conversation space as the main driver (if any) for character placement.</p><p> Movies and games rely on motion capture (mocap) files and hard-coded functions to perform spatial movements. These require extensive technical knowledge just to have a character move from one place to another. Other methods involve the use of Behavior Markup Language (BML), a form of XML, which describes character behaviors. BML Realizers take this BML and perform the requested behavior(s) on the character(s). Also, there are waypoint and other spatial navigation schemes, but they primarily focus on traversals and not correct positioning. Each of these require a fair amount of low-level detail and knowledge to write, plus BML realizers are still in their early stages of development. </p><p> Theatre, movies, and television all utilize a form of play-scripts, which provide detailed information on what the actor must do spatially, and when for a particular scene (that is spatio-temporal direction). These involve annotations, in addition to the speech, which identify scene setups, character movements, and entrances /exits. Humans have the ability to take these play-scripts and easily perform a believable scene. </p><p> This research focuses on utilizing play-scripts to provide spatio-temporal direction to virtual characters within a scene. Because of the simplicity of creating a playscript, and our algorithms to interpret the scripts, we are able to provide a quick method of blocking scenes with virtual characters.</p><p> We focus on not only an all-virtual cast of characters, but also human-controlled characters intermixing with the virtual characters for the scene. The key here is that human-controlled characters introduce a dynamic spatial component that affects how the virtual characters should perform the scene to ensure continuity, cohesion, and inclusion with the human-controlled character.</p><p> The algorithms to accomplish the blocking of a scene from a standard play-script are the core research contribution. These techniques include some part of speech tagging, named entity recognition, a rules engine, and strategically designed force-directed graphs. With these methods, we are able to similarly map any play-script&rsquo;s spatial positioning of characters to a human-performed version of the same playscript. Also, human-based evaluations indicate these methods provide a qualitatively good performance.</p><p> Potential applications include: a rehearsal tool for actors; a director tool to help create a play-script; a controller for virtual human characters in games or virtual environments; or a planning tool for positioning people in an industrial environment.</p><p>

<p> Natural Language Understanding (NLU) has been one of the longest-running and the most challenging areas in artificial intelligence. For any natural language comprehension system having a basic understanding of entities and concepts is a primary requirement. Comparison, where we name the similarities and differences between entities and concepts, is a unique cognitive ability in humans which requires memorizing facts, experiencing things and integration of concepts of the world. Clearly, developing NLU systems that are capable of comprehending comparison is a crucial step forward in AI. In this thesis, I will present my research on developing systems that are capable of comprehending comparison, through which, systems can learn world knowledge and perform basic commonsense reasoning.</p><p>

<p> Building systems that have natural language understanding capabilities has been one of the oldest and the most challenging pursuits in AI. In this thesis, we present our research on modeling language in terms of `events' and how they interact with each other in time, mainly in the domain of stories. </p><p> Deep language understanding, which enables inference and commonsense reasoning, requires systems that have large amounts of knowledge which would enable them to connect surface language to the concepts of the world. A part of our work concerns developing approaches for learning semantically rich knowledge bases on events. First, we present an approach to automatically acquire conceptual knowledge about events in the form of inference rules, which can enable commonsense reasoning. We show that the acquired knowledge is precise and informative which can be employed in different NLP tasks. </p><p> Learning stereotypical structure of related events, in the form of narrative structures or scripts, has been one of the major goals in AI. The research on narrative understanding has been hindered by the lack of a proper evaluation framework. We address this problem by introducing a new framework for evaluating story understanding and script learning: the 'Story Cloze Test (SCT)&rsquo;. In this test, the system is posed with a short four-sentence narrative context along with two alternative endings to the story, and is tasked with choosing the right ending. Along with the SCT, We have worked on developing the ROCStories corpus of about 100K commonsense short stories, which enables building models for story understanding and story generation. We present various models and baselines for tackling the task of SCT and show that human can perform with an accuracy of 100%. </p><p> One prerequisite for understanding and proper modeling of events and their interactions is to develop a comprehensive semantic framework for representing their variety of relations. We introduce `Causal and Temporal Relation Scheme (CaTeRS)' which is a rich semantic representation for event structures, with an emphasis on the domain of stories. The impact of the SCT and the ROCStories project goes beyond this thesis, where numerous teams and individuals across academia and industry have been using the evaluation framework and the dataset for a variety of purposes. We hope that the methods and the resources presented in this thesis will spur further research on building systems that can effectively model eventful context, understand, and generate logically-sound stories. </p><p>

<p> Deep learning, in particular neural networks, achieved remarkable success in the recent years. However, most of it is based on supervised learning, and relies on ever larger datasets, and immense computing power. One step towards general artificial intelligence is to build a model of the world, with enough knowledge to acquire a kind of ``common sense''. Representations learned by such a model could be reused in a number of other tasks. It would reduce the requirement for labelled samples and possibly acquire a deeper understanding of the problem. The vast quantities of knowledge required to build common sense precludes the use of supervised learning, and suggests to rely on unsupervised learning instead. </p><p> The concept of <i>uncertainty</i> is central to unsupervised learning. The task is usually to learn a complex, multimodal distribution. Density estimation and generative models aim at representing the whole distribution of the data, while predictive learning consists of predicting the state of the world given the context and, more often than not, the prediction is not unique. That may be because the model lacks the capacity or the computing power to make a certain prediction, or because the future depends on parameters that are not part of the observation. Finally, the world can be chaotic of truly stochastic. Representing complex, multimodal continuous distributions with deep neural networks is still an open problem. </p><p> In this thesis, we first assess the difficulties of representing probabilities in high dimensional spaces, and review the related work in this domain. We then introduce two methods to address the problem of video prediction, first using a novel form of linearizing auto-encoders and latent variables, and secondly using Generative Adversarial Networks (GANs). We show how GANs can be seen as trainable loss functions to represent uncertainty, then how they can be used to disentangle factors of variation. Finally, we explore a new non-probabilistic framework for GANs.</p><p>

<p> In this work, we address the problem of understanding what may have happened in a goal-based deliberative agent's environment after the occurrence of exogenous actions and events. Such an agent observes, periodically, information about the state of the world, but this information is incomplete, and reasons for state changes are not observed. We propose methods a goal-based agent can use to construct internal, causal explanations of its observations based on a model of its environment. These explanations comprise a series of inferred actions and events that have occurred and continue to occur in its world, as well as assumptions about the initial state of the world. We show that an agent can more accurately predict future events and states by reference to these explanations, and thereby more reliably achieve its goals. This dissertation presents the following novel contributions: (1) a formalization of the problems of achieving goals, understanding what has happened, and updating an agent's model in a partially observable, dynamic world with partially known dynamics; (2) a complete agent (DHA<p style="font-variant: small-caps">GENT</p>) that achieves goals in such environments more reliably than existing agents; (3) a novel algorithm (D<p style="font-variant: small-caps">ISCOVER</p>H<p style="font-variant: small-caps">ISTORY</p>) and technique (D<p style="font-variant: small-caps">ISCOVER </p>H<p style="font-variant: small-caps">ISTORY</p> search) for rapidly and accurately iteratively constructing causal explanations of what may have happened in these environments; (4) an examination of formal properties of these techniques; (5) a novel method (EML), capable of inferring improved models of an environment based on a small number of training scenarios; (6) experiments supporting performance claims about the novel methods described; and (7) an analysis of the efficiency of two D<p style="font-variant: small-caps">ISCOVER</p>H<p style="font-variant: small-caps">ISTORY</p> algorithm implementations. </p><p>

Among the most impressive of aspects of human intelligence is skill acquisition—the ability to identify important behavioral components, retain them as skills, refine them through practice, and apply them in new task contexts. Skill acquisition underlies both our ability to choose to spend time and effort to specialize at particular tasks, and our ability to collect and exploit previous experience to become able to solve harder and harder problems over time with less and less cognitive effort. Hierarchical reinforcement learning provides a theoretical basis for skill acquisition, including principled methods for learning new skills and deploying them during problem solving. However, existing work focuses largely on small, discrete problems. This dissertation addresses the question of how we scale such methods up to high-dimensional, continuous domains, in order to design robots that are able to acquire skills autonomously. This presents three major challenges; we introduce novel methods addressing each of these challenges. First, how does an agent operating in a continuous environment discover skills? Although the literature contains several methods for skill discovery in discrete environments, it offers none for the general continuous case. We introduce skill chaining, a general skill discovery method for continuous domains. Skill chaining incrementally builds a skill tree that allows an agent to reach a solution state from any of its start states by executing a sequence (or chain) of acquired skills. We empirically demonstrate that skill chaining can improve performance over monolithic policy learning in the Pinball domain, a challenging dynamic and continuous reinforcement learning problem. Second, how do we scale up to high-dimensional state spaces? While learning in relatively small domains is generally feasible, it becomes exponentially harder as the number of state variables grows. We introduce abstraction selection, an efficient algorithm for selecting skill-specific, compact representations from a library of available representations when creating a new skill. Abstraction selection can be combined with skill chaining to solve hard tasks by breaking them up into chains of skills, each defined using an appropriate abstraction. We show that abstraction selection selects an appropriate representation for a new skill using very little sample data, and that this leads to significant performance improvements in the Continuous Playroom, a relatively high-dimensional reinforcement learning problem. Finally, how do we obtain good initial policies? The amount of experience required to learn a reasonable policy from scratch in most interesting domains is unrealistic for robots operating in the real world. We introduce CST, an algorithm for rapidly constructing skill trees (with appropriate abstractions) from sample trajectories obtained via human demonstration, a feedback controller, or a planner. We use CST to construct skill trees from human demonstration in the Pinball domain, and to extract a sequence of low-dimensional skills from demonstration trajectories on a mobile robot. The resulting skills can be reliably reproduced using a small number of example trajectories. Finally, these techniques are applied to build a mobile robot control system for the uBot-5, resulting in a mobile robot that is able to acquire skills autonomously. We demonstrate that this system is able to use skills acquired in one problem to more quickly solve a new problem.

This thesis examines decentralized meta-reasoning. For a single agent or multiple agents, it may not be enough for agents to compute correct decisions if they do not do so in a timely or resource efficient fashion. The utility of agent decisions typically increases with decision quality, but decreases with computation time. The reasoning about one's computation process is referred to as meta-reasoning. Aspects of meta-reasoning considered in this thesis include the reasoning about how to allocate computational resources, including when to stop one type of computation and begin another, and when to stop all computation and report an answer. Given a computational model, this translates into computing how to schedule the basic computations that solve a problem. This thesis constructs meta-reasoning strategies for the purposes of monitoring and control in multi-agent settings, specifically settings that can be modeled by the Decentralized Partially Observable Markov Decision Process (Dec-POMDP). It uses decision theory to optimize computation for efficiency in time and space in communicative and non-communicative decentralized settings. Whereas base-level reasoning describes the optimization of actual agent behaviors, the meta-reasoning strategies produced by this thesis dynamically optimize the computational resources which lead to the selection of base-level behaviors.