This is a project with the ambition of demonstrating the possibilities and applications of the quantum approximation optimization algorithm (QAOA). Throughout the paper discussions on the theoretical background and fundamentals of the algorithm will be done by examining the relevant nomenclature. Then a set of possible application problems will be considered where it will be discussed why this specific algorithm is of interest for each individual problem. In the fourth section these problems will concretely be tested via simulations of the QAOA and lastly an analysis of the outcomes will be done.

In this thesis, we study the intersection of optimization and machine learning, especially how to use machine learning and optimization tools to make decisions. In Chapter 1, we propose a novel approach for accurate policy evaluation in personalized pricing. We solve an optimization problem to evaluate new pricing strategies, while searching over some worst case revenue functions. In Chapter 2, we consider problems where parameters are predicted using a machine learning model to be used for downstream optimization tasks. Recent works have proposed an integrated approach, accounting for how predictions are used in the downstream optimization problem, instead of just minimizing prediction error. We analyze the asymptotic performance of methods under the integrated and traditional approaches, in the sense of first-order stochastic. We argue that when the model class is rich enough to cover the ground truth, the traditional predict-then-optimize approach outperforms the integrated approach, and the performance ordering between the two approaches is reversed when the model is misspecified. In Chapter 3, we present a new class of architectures for reinforcement learning, Implicit Two-Tower (ITT) policies, where the actions are chosen based on the attention scores of their learnable latent representations with those of the input states. We show that ITT-architectures are particularly suited for evolutionary optimization and the corresponding policy training algorithms outperform their vanilla unstructured implicit counterparts as well as commonly used explicit policies. In Chapter 4, we consider an active learning problem, in which the learner has the ability to sequentially select unlabeled samples for labeling. A typical active learning algorithm would sample more points at “difficult" regions in the feature space to more efficiently use the sampling budget and reduce excess risk. For nonparametric classification with smooth regression functions, we show that nuances in notions of margin that involves the uniqueness of the Bayes classifier, having no apparent effect on rates in passive learning, determine whether or not any active learner can outperform passive learning rates.

京都大学 / 新制・課程博士 / 博士(工学) / 甲第24581号 / 工博第5087号 / 新制||工||1974(附属図書館) / 京都大学大学院工学研究科材料工学専攻 / (主査)教授 田中 功, 教授 安田 秀幸, 教授 中村 裕之 / 学位規則第4条第1項該当 / Doctor of Philosophy (Engineering) / Kyoto University / DFAM

<p>Real-world datasets are rarely clean. This causes the discrepancy between the claimed performance of machine learning algorithms on paper, and their actual performance on real-world problems. When dealing with missing or hidden information in a dataset, researchers have been using heuristic imputation methods since the first day of machine learning. However, it is known that many imputation methods do not have theoretical guarantees in various machine learning tasks, including clustering, community detection, sparsity recovery, to name a few. On the other hand, theoretical machine learning papers often follow simplistic assumptions, which are rarely fulfilled in real-world datasets. My research focuses on developing statistically and computationally efficient learning algorithms with provable guarantees under novel incomplete and latent assumptions. We consider problems with arguably more realistic incomplete and latent assumptions.We provide analysis to community detection in various network models, inference with latent variables in an arbitrary planted model, federated myopic community detection, and high-order tensor models. We analyze the interaction between the missing or latent structures and the inference / recoverability conditions, and proposed algorithms to solve the problems efficiently. <br> <br> Our main contributions in this thesis are as follows.<br> </p> <ol> <li>We analyze the information-theoretic limits for the recovery of node labels in several network models. We analyze the information-theoretic limits for community detection. We carefully construct restricted ensembles for a subclass of network models, and provide a series of novel results. </li> <li>We analyze the necessary and sufficient conditions for exact inference of a latent model. We show that exact inference can be achieved using a semidefinite programming approach without knowing either the latent variables or their domain. Our analysis predicts the experimental correctness of SDP with high accuracy, showing the suitability of our focus on the Karush-Kuhn-Tucker conditions and the spectrum of a properly defined matrix.</li> <li>We study the problem of recovering the community structure of a network under federated myopic learning. Under this paradigm, we have several clients, each of them having a myopic view, i.e., observing a small subgraph of the network. Each client sends a censored evidence graph to a central server. We provide an efficient algorithm, which computes a consensus signed weighted graph from clients evidence, and recovers the underlying network structure in the central server. We analyze the topological structure conditions of the network, as well as the signal and noise levels of the clients that allow for recovery of the network structure. Our analysis shows that exact recovery is possible and can be achieved in polynomial time.</li> <li>We study the problem of exact partitioning of high-order models. We consider two different high-order assumptions, and show that exact partitioning of high-order planted models is achievable through solving a convex optimization problem with a novel Carathéodory symmetric tensor cone in one case, and with a tensor nuclear norm constraint in the other.</li> <li>We study the problem of inference in high-order structured prediction tasks. We apply a generative model approach to study the problem of high-order inference, and provide a two-stage convex optimization algorithm for exact label recovery. We also connect the performance of our algorithm and the hyperedge expansion property using a novel hypergraph Cheeger-type inequality.</li> <li>We study the problem of partial recovery through semidefinite programming. We are interested in the scenarios in which the SDP returns a solution that is partially correct without any rounding. We analyze the optimality condition of partial recovery and provide statistical and topological guarantees. </li> </ol>

Worst-case analysis (WCA) has been the dominant tool for understanding the performance of the lion share of algorithmic arsenal of theoretical computer science. While WCA has provided us a thorough picture for a variety of problems over the last few decades, the advent of Machine Learn- ing era renewed our interest in several important optimization problems whose actual complexity have been elusive for empirical and real-world instances. More interestingly, while state-of-the- art ML models become deeper, larger in scale, sequential and highly nonconvex, the backbone of modern learning algorithms are simple algorithms such as Local Search, Gradient Descent and Follow The Leader variations (in the case of multi-agent tasks). Thus, a basic question endures: Why do simple algorithms work so well even in these challenging settings? A rapidly developing recent line of research, the so-called beyond worst-case analysis of algo- rithms (BWCA), aims to bridge this gap between theory and practice considering the design and analysis of algorithms using more realistic models or using natural structural properties. In this the- sis, we continue the line of work of BWCA by making contributions in several areas. Specifically, we focus on four main problems and models: 1. In combinatorial optimization, can simple flip local search methods compute local optima efficiently? 2. In continuous optimization, are the gradients necessary to compute efficiently local optima in the general nonconvex setting? 3. In multi-agent optimization, what is the computational complexity of generalized Nash Equi- libria and how simple dynamics behave on them? 4. In the special case of nonconvex-nonconcave minmax optimization, are there rich classes of ML well-motivated games with an effectively unique game theoretic solution that is selected by standard optimization techniques (e.g gradient descent) ? Leveraging machinery like the celebrated Smoothed Analysis of Spielman and Tang and the widely studied Lyapunov’s Stability Analysis, in this thesis we show that although the standard versions of these classical algorithms do not enjoy good theoretical properties in the worst case, simple modifications are sufficient to grant them desirable behaviors, which explain the underlying mechanisms behind their favorable performance in practice.

[pt] Compreensão de dados, codificação digital da fala, telecomunicações via celular, correção de erros de transmissão, são algumas das aplicações práticas do estudo dos códigos de cobertura, um importante ramo da área da matemática denominada teoria dos códigos. Neste trabalho são abordados dois problemas de códigos de cobertura: o problema clássico de códigos de cobertura e o recente problema denominado de códigos curtos de cobertura. Apresenta-se uma aplicação da metaeurística Busca Tabu Reativa, uma importante variação da Busca Tabu clássica, para os problemas citados. Além disto, apresenta-se uma nova técnica heurística para resolução de problemas de otimização combinatória denominada Heurística de Melhoria via Geração de Colunas (HMGC), juntamente com uma aplicação da mesma aos problemas em questão. A HMGC combina a geração atrasada de colunas, técnica usada na resolução de problemas com um grande número de variáveis de decisão (colunas), e heurísticas de busca local. É feita uma comparação dos resultados obtidos pela Busca Tabu Reativa, a Busca Tabu sem o mecanismo de reação e a HMGC, de forma a avaliar a qualidade das heurísticas apresentadas. / [en] Data compression, speech coding, móbile telecommunications and error-corretion are some of the practical apllications of the covering codes study, an important field of coding theory. This work addresses two problems of covering codes: the classic code covering problem and the recent short code covering problem. It presents an application of Reactive Tabu Search (RTS) metaheuristic for the problems cited, the RTS is an important variation of the classic Tabu Search. Moreover, it presents a new heuristic technique for solving combinatorial optimization problems named Column Generation Improbement Heuristic (CGIH). It also presents an application of CGIH for the covering codes problems. The CGIH combines the delayed column generation, technique used to solve problems with a large number of decision variables (columns), and local search heuristics. A comparison of results obtained by the Reactive Tabu Search, the Tabu Search without the reaction mechanism and the CGIH is also presented in order to assess the effectivenss of the presented heuristics.

In countries like the US, public school systems function through school districts, which are geographical areas where schools share the same administrative structure and are often coterminous with the boundary of a city or a county. School districts play an important role in the functioning of society. In a well-run school district with safe and well-functioning schools, graduating enough students can enhance the quality of life in its area. Conversely, a poorly run district may cause growth in the area to be far less than surrounding areas, or even a decline in population over time. To promote the efficient functioning of the school district, the boundaries of public schools are redrawn from time to time by the school board/planning officials. In the majority of the cases, this process of redrawing the school boundaries, also called school redistricting or school boundary formation, is done manually by the planners and involves hand-drawn maps. Given the rapid advancements in GIS made in the last decade and the availability of high-quality geospatial data, we opine that an objective treatment of the school redistricting problem by a data-driven model can assist the school board/ decision-makers by providing them with automated plans. These automated plans may serve as possible suggestions to the planners, who can adapt them to prepare their own plans in the way they see fit based on their subjective knowledge and expertise. In this dissertation, we propose algorithmic techniques for solving the problem of (school) redistricting, which is an NP-hard problem. We primarily investigate optimization-based algorithms for solving the problem. Our approaches include (i) clustering, (ii) local search, and (iii) memetic algorithms. We also propose ways of solving the problem using exact methods and fair redistricting techniques based on ethical considerations. The techniques developed here are generic enough to be applied to other redistricting problems with some degree of modification in the objective function and constraint-handling techniques. The source code and corresponding datasets are available at https://github.com/subhodipbiswas/schoolredistricting. / Doctor of Philosophy / In many countries, public school systems function through school districts, which are geographical areas where schools share the same administrative structure and are often coterminous with the boundary of a city or a county. To promote efficient functioning of the school district, the boundaries of public schools are redrawn from time to time by the school board/planning officials. In the majority of the cases, this process of redrawing the school boundaries, also called school redistricting, is done manually by the planners and involves hand-drawn maps. Given the rapid advancements in GIS made in the last decade and the availability of high-quality geospatial data, we opine that an objective treatment of the school redistricting problem by a data-driven model can assist the school board/ decision-makers by providing them with automated plans. In this presentation, we propose algorithmic techniques for solving the school redistricting problem. Our approaches include (i) clustering, (ii) local search, and (iii) memetic algorithms. We also show that MCMC-based techniques can aid in enabling exact methods to operate on this problem. Lastly, we briefly highlight ethical considerations involved in the process of school redistricting and throw light on some ways to devise more ethically-aware strategies for doing school redistricting. The results indicate that the proposed methods could be a valuable decision-making tool for school officials.

No description available.

In this dissertation, the vehicle routing problem and one of its variants, the vehicle routing problem with simultaneous pick up and deliveries (VRPSPD) are studied. The traditional vehicle routing problem (VRP) consists of constructing minimum cost routes for the vehicles to follow so that the set of customers are visited only once. A lot of effort has been devoted to research on developing fast and effective solution methods for many different versions of this problem by different majors of engineering profession. Thus, a structuring effort is needed to organize and document the vast literature so far has accumulated in this field. Over its lifespan the VRP literature has become quite disjointed and disparate. Keeping track of its development has become difficult because its subject matter transcends several academic disciplines and professions that range from algorithm design to traffic management. Consequently, this dissertation begins with defining VRP's domain in its entirety, accomplishes an allencompassing taxonomy for the VRP literature, and delineates all of VRP's facets in a parsimonious and discriminating manner. Sample articles chosen for their disparity are classified to illustrate the descriptive power and parsimony of the taxonomy. Next, a more detailed version of the original problem, the VRPSPD is examined and a more abstract taxonomy is proposed. Additionally, two other existing classification methodologies are used to distinguish all published VRPSPD papers on their respective research strategies and solution methods. By using well-organized methods this study provides a solid multidimensional identification of all VRPSPD studies? attributes thus synthesizing knowledge in the filed. Finally, a hybrid metaheuristic solution algorithm for the VRPSPD problem is presented. To solve this NP-hard vehicle routing problem a GRASP initiated hybrid genetic algorithm is developed. The algorithm is tested on two sets of benchmark problems from the literature with respect to computational efficiency and solution quality. The effect of starting with a better initial population for the genetic algorithm is further investigated by comparing the current results with previously generated ones. The experimental results indicate that the proposed algorithm produces relatively good quality solutions and a better initial population yields a reduction in processing cycles.

Traditionally, optimization problems in operations research have been studied in a complete information setting; the input/data is collected and made fully accessible to the user, before an algorithm is sequentially run to generate the optimal output. However, the growing magnitude of treated data and the need to make immediate decisions are increasingly shifting the focus to optimizing under incomplete information settings. The input can be partially inaccessible to the user either because it is generated continuously, contains some uncertainty, is too large and cannot be stored on a single machine, or has a hidden structure that is costly to unveil. Many problems providing a context for studying algorithms when the input is not entirely accessible emanate from the field of matching theory, where the objective is to pair clients and servers or, more generally, to group clients in disjoint sets. Examples include ride-sharing and food delivery platforms, internet advertising, combinatorial auctions, and online gaming. In this thesis, we study three different novel problems from the theory of matchings. These problems correspond to situations where the input is hidden, spread across multiple processors, or revealed in two stages with some uncertainty. In particular, we present in Chapter 1 the necessary definitions and terminology for the concepts and problems we cover. In Chapter 2, we consider a two-stage robust optimization framework that captures matching problems where one side of the input includes some future demand uncertainty. We propose two models to capture the demand uncertainty: explicit and implicit scenarios. Chapters 3 and 4 see us switch our attention to matchings in hypergraphs. In Chapter 3, we consider the problem of learning hidden hypergraph matchings through membership queries. Finally, in Chapter 4, we study the problem of finding matchings in uniform hypergraphs in the massively parallel computation (MPC) model where the data (e.g. vertices and edges) is distributed across the machines and in each round, a machine performs local computation on its fragment of data, and then sends messages to other machines for the next round.