<p> Object understanding is a fundamental problem in computer vision and it has been extensively researched in recent years thanks to the availability of powerful GPUs and labelled data, especially in the context of images. However, 3D object understanding is still not on par with its 2D domain and deep learning for 3D has not been fully explored yet. In this dissertation, I work on two approaches, both of which advances the state-of-the-art results in 3D classification and segmentation.</p><p> The first approach, called MVRNN, is based multi-view paradigm. In contrast to MVCNN which does not generate consistent result across different views, by treating the multi-view images as a temporal sequence, our MVRNN correlates the features and generates coherent segmentation across different views. MVRNN demonstrated state-of-the-art performance on the Princeton Segmentation Benchmark dataset.</p><p> The second approach, called PointGrid, is a hybrid method which combines points and regular grid structure. 3D points can retain fine details but irregular, which is challenge for deep learning methods. Volumetric grid is simple and has regular structure, but does not scale well with data resolution. Our PointGrid, which is simple, allows the fine details to be consumed by normal convolutions under a coarser resolution grid. PointGrid achieved state-of-the-art performance on ModelNet40 and ShapeNet datasets in 3D classification and object part segmentation. </p><p>

<p> Being able to accomplish tasks with multiple learners through learning has long been a goal of the multiagent systems and machine learning communities. One of the main approaches people have taken is reinforcement learning, but due to certain conditions and restrictions, applying reinforcement learning in a multiagent setting has not achieved the same level of success when compared to its single agent counterparts. </p><p> This thesis aims to make coordination better for agents in cooperative games by improving on reinforcement learning algorithms in several ways. I begin by examining certain pathologies that can lead to the failure of reinforcement learning in cooperative games, and in particular the pathology of <i> relative overgeneralization</i>. In relative overgeneralization, agents do not learn to optimally collaborate because during the learning process each agent instead converges to behaviors which are robust in conjunction with the other agent's exploratory (and thus random), rather than optimal, choices. One solution to this is so-called <i>lenient learning</i>, where agents are forgiving of the poor choices of their teammates early in the learning cycle. In the first part of the thesis, I develop a lenient learning method to deal with relative overgeneralization in independent learner settings with small stochastic games and discrete actions. </p><p> I then examine certain issues in a more complex multiagent domain involving parameterized action Markov decision processes, motivated by the RoboCup 2D simulation league. I propose two methods, one batch method and one actor-critic method, based on state of the art reinforcement learning algorithms, and show experimentally that the proposed algorithms can train the agents in a significantly more sample-efficient way than more common methods. </p><p> I then broaden the parameterized-action scenario to consider both repeated and stochastic games with continuous actions. I show how relative overgeneralization prevents the multiagent actor-critic model from learning optimal behaviors and demonstrate how to use Soft Q-Learning to solve this problem in repeated games. </p><p> Finally, I extend imitation learning to the multiagent setting to solve related issues in stochastic games, and prove that given the demonstration from an expert, multiagent Imitation Learning is exactly the multiagent actor-critic model in Maximum Entropy Reinforcement Learning framework. I further show that when demonstration samples meet certain conditions the relative overgeneralization problem can be avoided during the learning process.</p><p>

Thesis (Ph.D.)--University of Illinois at Urbana-Champaign, 2009. / Source: Dissertation Abstracts International, Volume: 70-06, Section: B, page: 3593. Adviser: Jiawei Han. Includes bibliographical references (leaves 93-99) Available on microfilm from Pro Quest Information and Learning.

Thesis (Ph. D.)--Syracuse University, 2009. / "Publication number: AAT 3381974 ."

Thesis (Ph.D.)--University of Illinois at Urbana-Champaign, 2009. / Source: Dissertation Abstracts International, Volume: 70-06, Section: B, page: 3605. Adviser: Gerald DeJong. Includes bibliographical references (leaves 107-112) Available on microfilm from Pro Quest Information and Learning.

Thesis (M.S.)--Lehigh University, 2009. / Adviser: Hector Munoz-Avila.

<p> In this dissertation, I introduce modularization as a means of efficiently solving problems represented by dynamic Bayesian networks and study the properties and effects of modularization relative to traditional solutions. Modularizing a Bayesian filter allows its results to be calculated faster than a traditional Bayesian filter. Traditional Bayesian filters can have issues when large problems must be solved within a short period of time. Modularization addresses this issue by dividing the full problem into a set of smaller problems that can then be solved with separate Bayesian filters. Since the time complexity of Bayesian filters is greater than linear, solving several smaller problems is cheaper than solving a single large problem. The cost of reassembling the results from the smaller problems is comparable to the cost of the smaller problems. This document introduces the concept of both exact and approximate modular Bayesian filters and describes how to design each of the elements of a modular Bayesian filters. These concepts are clarified by using a series of examples from the realm of vehicle state estimation and include the results of each stage of the algorithm creation in a simulated environment. A final section shows the implementation of a modular Bayesian filter in a real-world problem tasked with addressing the problem of vehicle state estimation in the face of transitory sensor failure. This section also includes all of the attending algorithms that allow the problem to be solved accurately and in real-time.</p>

<p> In computer vision, grammatical models are models that represent objects hierarchically as compositions of sub-objects. This allows us to specify rich object models in a standard Bayesian probabilistic framework. In this thesis, we formulate shape grammars, a probabilistic model of curve formation that allows for both continuous variation and structural variation. We derive an EM-based training algorithm for shape grammars. We demonstrate the effectiveness of shape grammars for modeling human silhouettes, and also demonstrate their effectiveness in classifying curves by shape. We also give a general method for heuristically speeding up a large class of dynamic programming algorithms. We provide a general framework for discussing coarse-to-fine search strategies, and provide proofs of correctness. Our method can also be used with inadmissible heuristics. </p><p> Finally, we give an algorithm for doing approximate context-free parsing of long strings in linear time. We define a notion of approximate parsing in terms of restricted families of decompositions, and construct small families which can approximate arbitrary parses.</p>

<p> Fast and accurate recognition of offline Chinese characters is a problem significantly more difficult than the recognition of the English alphabet. The vastly larger set of characters and noise in handwriting require more sophisticated normalization, feature extraction, and classification methods. This thesis explores the feasibility of a fast and accurate classification and translation retrieval system. An ensemble classifier composed of k-nearest neighbors and support vector machines is used as the basis of a fast classifier to recognize Chinese and Japanese characters. In contrast to other models, this classifier incorporates contextual N-gram information directly into the classification task to increase the accuracy of the classifier.</p>

<p> With billions of internet users, online media services have become commonplace. Prediction and recommendation for online media are fundamental problems in various applications, including recommender systems and information retrieval. As an example, accurately predicting user behaviors improves user experiences through more intelligent user interfaces. On the other hand, user behavior prediction in online media is also strongly related to behavior targeting and online advertisement which is the major business for most consumer internet services. Estimating and understanding users' click behaviors is a critical problem in online advertising. In this dissertation, we investigate the prediction and recommendation problems in various online media. We find a number of challenges: high order relations, temporal dynamics, complexity of network structure, high data sparsity and coupled social media activities. We consider user behavior understanding and prediction in four areas: tag prediction in a social tagging system, link prediction in microblogging services, multi-context modeling in online social media and click prediction in sponsored search. In such topics, based on real world data, we analyze user behaviors and discover patterns, properties and challenges. Subsequently, we design specific models for online user behavior prediction in various online media: a probabilistic model for personalized tag prediction, a user-tag-specific temporal interests model for tracking users' interests over time in tagging systems, a personalized structure based link prediction model for micro-blogging systems, a generalized latent factor model and Bayesian treatment for modeling across multiple contexts in online social media, a context-aware click model and framework for estimating ad group performance in sponsored search. Our extensive experiments on large-scale real-world datasets show our novel models advance the state-of-the-art.</p>