In this thesis, a class of combinatorial optimization methods rooted in statistical mechanics and their use in signal processing applications will be discussed. The thesis consists of two separate parts. The first part deals with the rationale for my work and also covers the background information necessary to put the second part, which consists of a number of papers, in context. There are (at least) two sides to an optimization problem---the problem statement arising from an application or a design and the selection of an algorithm to solve the problem. In this work the problem statements are practical problems, of combinatorial nature, frequently encountered in signal processing and the algorithms of choice are annealing based algorithms, founded in statistical mechanics. From my work, it is my experience that solving a particular problem often leads to new developments on the part of the algorithm which, in turn, open up possibilities to apply the modified algorithm to a new set of problems, leading to a continuously improving algorithm and a growing field of applications. The included papers deal with the application of annealing optimization methods to the problems of configuring active noise and vibration control systems, digital filter design and adaptive filtering. They also describe the successive development of a highly efficient entropy-directed deterministic annealing (EDDA) optimization algorithm detailed in the final paper.

Combinatorial optimization problems are of high academical as well as practical importance. Many instances of relevant combinatorial optimization problems are, due to their dimensions, intractable for complete methods such as branch and bound. Therefore, approximate algorithms such as metaheuristics received much attention in the past 20 years. Examples of metaheuristics are simulated annealing, tabu search, and evolutionary computation. One of the most recent metaheuristics is ant colony optimization (ACO), which was developed by Prof. M. Dorigo (who is the supervisor of this thesis) and colleagues. This thesis deals with theoretical as well as practical aspects of ant colony optimization. * A survey of metaheuristics. Chapter 1 gives an extensive overview on the nowadays most important metaheuristics. This overview points out the importance of two important concepts in metaheuristics: intensification and diversification. * The hyper-cube framework. Chapter 2 introduces a new framework for implementing ACO algorithms. This framework brings two main benefits to ACO researchers. First, from the point of view of the theoretician: we prove that Ant System (the first ACO algorithm to be proposed in the literature) in the hyper-cube framework generates solutions whose expected quality monotonically increases with the number of algorithm iterations when applied to unconstrained problems. Second, from the point of view of the experimental researcher, we show through examples that the implementation of ACO algorithms in the hyper-cube framework increases their robustness and makes the handling of the pheromone values easier. * Deception. In the first part of Chapter 3 we formally define the notions of first and second order deception in ant colony optimization. Hereby, first order deception corresponds to deception as defined in the field of evolutionary computation and is therefore a bias introduced by the problem (instance) to be solved. Second order deception is an ACO-specific phenomenon. It describes the observation that the quality of the solutions generated by ACO algorithms may decrease over time in certain settings. In the second part of Chapter 3 we propose different ways of avoiding second order deception. * ACO for the KCT problem. In Chapter 4 we outline an ACO algorithm for the edge-weighted k-cardinality tree (KCT) problem. This algorithm is implemented in the hyper-cube framework and uses a pheromone model that was determined to be well-working in Chapter 3. Together with the evolutionary computation and the tabu search approaches that we develop in Chapter 4, this ACO algorithm belongs to the current state-of-the-art algorithms for the KCT problem. * ACO for the GSS problem. Chapter 5 describes a new ACO algorithm for the group shop scheduling (GSS) problem, which is a general shop scheduling problem that includes among others the well-known job shop scheduling (JSS) and the open shop scheduling (OSS) problems. This ACO algorithm, which is implemented in the hyper-cube framework and which uses a new pheromone model that was experimentally tested in Chapter 3, is currently the best ACO algorithm for the JSS as well as the OSS problem. In particular when applied to OSS problem instances, this algorithm obtains excellent results, improving the best known solution for several OSS benchmark instances. A final contribution of this thesis is the development of a general method for the solution of combinatorial optimization problems which we refer to as Beam-ACO. This method is a hybrid between ACO and a tree search technique known as beam search. We show that Beam-ACO is currently a state-of-the-art method for the application to the existing open shop scheduling (OSS) problem instances.

The <i>mixed postman problem</i> consists of finding a minimum cost tour of a mixed graph <i>M</i> = (<i>V</i>,<i>E</i>,<i>A</i>) traversing all its edges and arcs at least once. We prove that two well-known linear programming relaxations of this problem are equivalent. The <i>extra cost</i> of a mixed postman tour <i>T</i> is the cost of <i>T</i> minus the cost of the edges and arcs of <i>M</i>. We prove that it is <i>NP</i>-hard to approximate the minimum extra cost of a mixed postman tour. A related problem, known as the <i>windy postman problem</i>, consists of finding a minimum cost tour of an undirected graph <i>G</i>=(<i>V</i>,<i>E</i>) traversing all its edges at least once, where the cost of an edge depends on the direction of traversal. We say that <i>G</i> is <i>windy postman perfect</i> if a certain <i>windy postman polyhedron O</i> (<i>G</i>) is integral. We prove that series-parallel undirected graphs are windy postman perfect, therefore solving a conjecture of Win. Given a mixed graph <i>M</i> = (<i>V</i>,<i>E</i>,<i>A</i>) and a subset <i>R</i> &#8838; <i>E</i> &#8746; <i>A</i>, we say that a mixed postman tour of <i>M</i> is <i>restricted</i> if it traverses the elements of <i>R</i> exactly once. The <i>restricted mixed postman problem</i> consists of finding a minimum cost restricted tour. We prove that this problem is <i>NP</i>-hard even if <i>R</i>=<i>A</i> and we restrict <i>M</i> to be planar, hence solving a conjecture of Veerasamy. We also prove that it is <i>NP</i>-complete to decide whether there exists a restricted tour even if <i>R</i>=<i>E</i> and we restrict <i>M</i> to be planar. The <i>edges postman problem</i> is the special case of the restricted mixed postman problem when <i>R</i>=<i>A</i>. We give a new class of valid inequalities for this problem. We introduce a relaxation of this problem, called the <i>b-join problem</i>, that can be solved in polynomial time. We give an algorithm which is simultaneously a 4/3-approximation algorithm for the edges postman problem, and a 2-approximation algorithm for the extra cost of a tour. The <i>arcs postman problem</i> is the special case of the restricted mixed postman problem when <i>R</i>=<i>E</i>. We introduce a class of necessary conditions for <i>M</i> to have an arcs postman tour, and we give a polynomial-time algorithm to decide whether one of these conditions holds. We give linear programming formulations of this problem for mixed graphs arising from windy postman perfect graphs, and mixed graphs whose arcs form a forest.

Um emparelhamento em um grafo é um conjunto de arestas duas a duas não adjacentes. Dado um grafo G com pesos em suas arestas, o problema do emparelhamento de peso é máximo é encontrar um emparelhamento cuja soma dos pesos de suas arestas é máxima. Neste trabalho estudamos diferentes soluções para esse problema. Estudamos algoritmos combinatórios que resolvem o problema no caso em que G é bipartido e no caso geral. O algoritmo de Edmonds é um algoritmo polinomial cuja complexidade de tempo é O(n^4), onde n é o número de vértices do grafo G. Discutimos nesse trabalho nossa implementação desse algoritmo. Num trabalho de 1985, Grötschel e Holland propuseram o uso de ferramentas de programação linear para resolver o mesmo problema. O método chamado de planos-de-corte baseia-se em um resultado de Padberg e Rao de que o problema da separação associado ao poliedro dos emparelhamentos pode ser resolvido em tempo polinomial. Neste trabalho fizemos implementações dos dois métodos e os utilizamos para resolver diversos tipos de instâncias do problema. Nossa conclusão é que o método poliédrico, apesar de utilizar ferramentas genéricas, é bastante eficiente na prática. / A matching in a graph G is a set of pairwise disjoint edges of G. Given a graph G with edge weights, we define the maximum weight matching problem as that of finding a matching which maximizes the sum of its weights. In this thesis we study different solutions to this problem. We studied combinatorial algorithms that solve this problem in the case where G is bipartite and also in the general case. Edmonds algorithm [Edm65a] is a polynomial time algorithm with complexity O(n4 ), where n is the number of vertices in the graph G. We discuss in this document our implementation of this algorithm. In a paper from 1985, Gr tschel & Holland [GH85] discussed the use of linear programming o tools for solving the maximum weight matching problem. This so called cut-plane method relies on a result by Padberg & Rao [PR82] that proves that the separation problem associated with matching polyhedron is solvable in polinomial time. In this work we implemented both methods and used then to solve different instances of the problem. Our conclusion is that the polyhedral method, although using generical tools is very efficient in practice.

O empacotamento irregular de fita é um grupo de problemas na área de corte e empacotamento, cuja aplicação é observada nas indústrias têxtil, moveleira e construção naval. O problema consiste em definir uma configuração de itens irregulares de modo que o comprimento do contêiner retangular que contém o leiaute seja minimizado. A solução deve ser válida, isto é, não deve haver sobreposição entre os itens, que não devem extrapolar as paredes do contêiner. Devido a aspectos práticos, são admitidas até quatro orientações para o item. O volume de material desperdiçado está diretamente relacionado à qualidade do leiaute obtido e, por este motivo, uma solução eficiente pressupõe uma vantagem econômica e resulta em um menor impacto ambiental. O objetivo deste trabalho consiste na geração automática de leiautes de modo a obter níveis de compactação e tempo de processamento compatíveis com outras soluções na literatura. A fim de atingir este objetivo, são realizadas duas propostas de solução. A primeira consiste no posicionamento sequencial dos itens de modo a maximizar a ocorrência de posições de encaixe, que estão relacionadas à restrição de movimento de um item no leiaute. Em linhas gerais, várias sequências de posicionamentos são exploradas com o objetivo de encontrar a solução mais compacta. Na segunda abordagem, que consiste na principal proposta deste trabalho, métodos rasterizados são aplicados para movimentar itens de acordo com uma grade de posicionamento, admitindo sobreposição. O método é baseado na estratégia de minimização de sobreposição, cujo objetivo é a eliminação da sobreposição em um contêiner fechado. Ambos os algoritmos foram testados utilizando o mesmo conjunto de problemas de referência da literatura. Foi verificado que a primeira estratégia não foi capaz de obter soluções satisfatórias, apesar de fornecer informações importantes sobre as propriedades das posições de encaixe. Por outro lado, a segunda abordagem obteve resultados competitivos. O desempenho do algoritmo também foi compatível com outras soluções, inclusive em casos nos quais o volume de dados era alto. Ademais, como trabalho futuro, o algoritmo pode ser estendido de modo a possibilitar a entrada de itens de geometria genérica, o que pode se tornar o grande diferencial da proposta. / Irregular nesting belongs to the area of cutting and packing problems and are employed in the textile, wood and shipbuilding industries. The problem consists in determining a configuration for a set of irregular items which minimizes the length of the rectangular container in which the layout is located. The solution must be feasible, i.e., items must not overlap nor protrude the container walls. Due to practical reasons, up to four orientations are allowed for an item. The volume of wasted material is directly affected by the quality (density) of the layout. Thus, an efficient solution produces a positive economic and environmental impact. In this work, the objective is to automatically obtain layouts such that their density and the performance of the algorithm are competitive with other solutions in literature. So as to achieve this goal, two approaches are proposed. The first method uses a special sequential placement heuristic such that the algorithm maximizes exact placements, which consist of constrained positions for items. In general terms, a search is performed in the placement sequence in order to obtain a compact layout. In the second approach, which is the main subject of this work, raster methods are employed to guide the translation of items, which are free to move within the layout, and may overlap other items. The method is based on overlap minimization techniques, in which the objective is to eliminate the overlap in a fixed dimensions container. Both algorithms were tested using benchmark problems from the literature. The first strategy yielded unsatisfactory results, though it provided important information about the properties of exactly fitting placements. On the other hand, the main approach was able to produce competitive solutions. The performance was also compatible with other solutions, even in cases which the data volume was high. Moreover, as a future work, an extension for the algorithm can be developed such that items with generic geometry can be considered, which would be an important advance in research terms.

Abstract not available

Un des buts principaux de la biologie évolutive et de la médecine moléculaire consiste à reconstruire les relations phylogénétiques entre organismes à partir de leurs séquences moléculaires. En littérature, cette question est connue sous le nom d’inférence phylogénétique et a d'importantes applications dans la recherche médicale et pharmaceutique, ainsi que dans l’immunologie, l’épidémiologie, et la dynamique des populations. L’accumulation récente de données de séquences d’ADN dans les bases de données publiques, ainsi que la facilité relative avec laquelle des données nouvelles peuvent être obtenues, rend l’inférence phylogénétique particulièrement difficile (l'inférence phylogénétique est un problème NP-Hard sous tous les critères d’optimalité connus), de telle manière que des nouveaux critères et des algorithmes efficaces doivent être développés. Cette thèse a pour but: (i) d’analyser les limites mathématiques et biologiques des critères utilisés en inférence phylogénétique, (ii) de développer de nouveaux algorithmes efficaces permettant d’analyser de plus grands jeux de données.

We study the complexity of finding so called spanner paths between arbitrary nodes in Euclidean graphs. We study both general Euclidean graphs and a special type of graphs called Integer Graphs. The problem is proven NP-complete for general Euclidean graphs with non-constant stretches (e.g. (2n)^(3/2) where n denotes the number of nodes in the graph). An algorithm solving the problem in O(2^(0.822n)) is presented. Integer graphs are simpler and for these special cases a better algorithm is presented. By using a partial order of so called Images the algorithm solves the spanner path problem using O(2^(c(\log n)^2)) time, where c is a constant depending only on the stretch.

One of the simplest, but still NP-hard, routing problems is the Traveling Salesman Problem (TSP). In the TSP, one is given a set of cities and a way of measuring the distance between cities. One has to find the shortest tour that visits all cities exactly once and returns back to the starting city. In state-of-the-art algorithms, they all assume that a complete graph is given as an input. However, for very large graphs, generating all edges in a complete graph, which corresponds to finding shortest paths for all city pairs, could be time-consuming. This is definitely a major obstacle for some real-life applications, especially when the tour needs to be generated in real-time. The objective, in this thesis, is to find a near-optimal TSP tour with a reduced set of edges in the complete graph. In particular, the following problems are investigated: which subset of edges can be produced in a shorter time comparing to the time for generating the complete graph? Is there a subset of edges in the complete graph that results in a better near-optimal tour than other sets? With a non-complete graph, which improvement algorithms work better? In this thesis, we study six algorithms to generate subsets of edges in a complete graph. To evaluate the proposed algorithms, extensive experiments are conducted with the well-known TSP data in a TSP library. In these experiments, we evaluate these algorithms in terms of tour quality, time and scalability.

The <i>mixed postman problem</i> consists of finding a minimum cost tour of a mixed graph <i>M</i> = (<i>V</i>,<i>E</i>,<i>A</i>) traversing all its edges and arcs at least once. We prove that two well-known linear programming relaxations of this problem are equivalent. The <i>extra cost</i> of a mixed postman tour <i>T</i> is the cost of <i>T</i> minus the cost of the edges and arcs of <i>M</i>. We prove that it is <i>NP</i>-hard to approximate the minimum extra cost of a mixed postman tour. A related problem, known as the <i>windy postman problem</i>, consists of finding a minimum cost tour of an undirected graph <i>G</i>=(<i>V</i>,<i>E</i>) traversing all its edges at least once, where the cost of an edge depends on the direction of traversal. We say that <i>G</i> is <i>windy postman perfect</i> if a certain <i>windy postman polyhedron O</i> (<i>G</i>) is integral. We prove that series-parallel undirected graphs are windy postman perfect, therefore solving a conjecture of Win. Given a mixed graph <i>M</i> = (<i>V</i>,<i>E</i>,<i>A</i>) and a subset <i>R</i> &#8838; <i>E</i> &#8746; <i>A</i>, we say that a mixed postman tour of <i>M</i> is <i>restricted</i> if it traverses the elements of <i>R</i> exactly once. The <i>restricted mixed postman problem</i> consists of finding a minimum cost restricted tour. We prove that this problem is <i>NP</i>-hard even if <i>R</i>=<i>A</i> and we restrict <i>M</i> to be planar, hence solving a conjecture of Veerasamy. We also prove that it is <i>NP</i>-complete to decide whether there exists a restricted tour even if <i>R</i>=<i>E</i> and we restrict <i>M</i> to be planar. The <i>edges postman problem</i> is the special case of the restricted mixed postman problem when <i>R</i>=<i>A</i>. We give a new class of valid inequalities for this problem. We introduce a relaxation of this problem, called the <i>b-join problem</i>, that can be solved in polynomial time. We give an algorithm which is simultaneously a 4/3-approximation algorithm for the edges postman problem, and a 2-approximation algorithm for the extra cost of a tour. The <i>arcs postman problem</i> is the special case of the restricted mixed postman problem when <i>R</i>=<i>E</i>. We introduce a class of necessary conditions for <i>M</i> to have an arcs postman tour, and we give a polynomial-time algorithm to decide whether one of these conditions holds. We give linear programming formulations of this problem for mixed graphs arising from windy postman perfect graphs, and mixed graphs whose arcs form a forest.