Το κυρίως αντικείμενο της διατριβής αυτής είναι ο σχεδιασμός και η ανάλυση αποκεντρωμένων τεχνικών ελέγχου για επίτευξη μέγιστης κάλυψης από κινούμενα δίκτυα αισθητήρων. Λόγω των πολλών εφαρμογών αυτών σε αποστολές σχετιζόμενες με εξερεύνηση περιοχών ενδιαφέροντος, περιβαλλοντική δειγματοληψία, φύλαξη ή ακόμα και θέματα ασφάλειας, μία μεγάλη μερίδα της επιστημονικής κοινότητας έχει στρέψει το ενδιαφέρον της στην ανάπτυξη μεθόδων για βέλτιστη (ει δυνατόν) περιβαλλοντική αντίληψη μέσω αισθητήρων από αυτόνομες ομάδες ρομποτικών συστημάτων. Τέτοιες ομάδες, συνήθως τοποθετούμενες αρχικώς στις περιοχές ενδιαφέροντος, σχεδιάζονται με στόχο τον αποκεντρωμένο έλεγχό τους, αντί ενός καθολικού εποπτικού συστήματος, με στόχο να επιτύχουν στην εκάστοτε αποστολή. Στα πρώτα στάδια της διατριβής αυτής, το πρόβλημα της κάλυψης μιας περιοχής ενδιαφέροντος από μία ομάδα όμοιων κόμβων αναλύεται από υπολογιστική σκοπιά. Οι κινούμενοι κόμβοι υποθέτονται ότι υπακούν σε απλοϊκό κινηματικό μοντέλο διακριτού χρόνου, ενώ η αισθητήρια επίδοσή τους θεωρείται ακτινική, περιορισμένης εμβέλειας, ομοιόμορφη γύρω από τον κόμβο. Σαν πρώτη προσέγγιση, η κατεύθυνση σε κάθε χρονική στιγμή για βέλτιστη κάλυψη καθορίζεται βάσει τεχνικών διαμέρισης του χώρου βασιζόμενες στην έννοια της απόστασης. Η αναπτυσσόμενη στρατηγική επιτρέπει σταδιακή αύξηση της καλυπτόμενης επιφάνειας μεταξύ διαδοχικών βημάτων, ενώ έχει ως απαίτηση την κίνηση ενός μόνο επιτρεπτού κόμβου τη φορά. Στη συνέχεια, το προαναφερθέν σχέδιο επεκτείνεται για την περίπτωση ετερογενών δικτύων, όπου η ετερογένεια αντικατοπτρίζεται στις άνισες εμβέλειες απόδοσης αίσθησης των κόμβων. Επιπροσθέτως, επέκταση σε μοντέλο συνεχούς χρόνου επιτρέπει την κίνηση όλων των κόμβων του δικτύου ταυτόχρονα, αυξάνοντας ιδιαίτερα τον χρόνο σύγκλισης προς την βέλτιστη κατάσταση, ειδικά για μεγάλης κλίμακας δίκτυα. Μία εναλλακτική διαμέριση του χώρου αναπτύσσεται, η οποία βασίζεται κυρίως στα αισθητήρια μοτίβα των κόμβων, παρά στις θέσεις των κόμβων καθεαυτές. Τα παραγόμενα κελιά του χώρου ανατιθέμενα στους κόμβους αποτελούν τον βασικό πυρήνα του αλγόριθμου οργάνωσης, με στόχο την αποκεντρωμένη οργάνωση της κινούμενης ομάδας, ώστε να επιτύχει βέλτιστη απόδοση κάλυψης. Υποκινούμενοι από την υψηλού–βαθμού ανισοτροπία που χαρακτηρίζει κάποιους τύπους αισθητήρων, όπως κατευθυντικά μικρόφωνα για ανίχνευση ήχου σε εφαρμογές ασφάλειας, ή ακόμα μοτίβα εκπομπής/λήψης κατευθυντικών κεραιών σε σενάρια τηλεπικοινωνιακής κάλυψης, η έρευνά μας επεκτείνεται πέραν του κλασσικού ακτινικού μοντέλου δίσκου αίσθησης. Βασιζόμενοι σε συγκεκριμένες ιδιότητες για επίπεδες κυρτές καμπύλες, μια αποκεντρωμένη στρατηγική οργάνωσης αναπτύχθηκε για δίκτυα που χαρακτηρίζονται από κυρτά αισθητήρια μοτίβα ίδιας κατευθυντικότητας. Παρότι η κυρτότητα των συνόλων αίσθησης φαίνεται να θέτει ένα μεγάλου βαθμού περιορισμό στο συνολικό πρόβλημα, στην πραγματικότητα προσπερνάται μέσω ανάθεσης αυτών ως το μέγιστο κυρτό χωρίο που εγγράφεται στο πρωταρχικώς ανισοτροπικό μοτίβο. Το σχήμα ελέγχου επεκτείνεται στη συνέχεια για την περίπτωση όπου εισάγουμε ένα επιπλέον βαθμό ελευθερίας στις κινηματικές ικανότητες των κόμβων, ενσωματώνοντας έτσι διαφορετικές και χρονικά μεταβαλλόμενες κατευθυντικότητες μεταξύ των μοτίβων αυτών. Το παραγόμενο πλάνο ελέγχου αποδεικνύεται ότι οδηγεί ανισοτροπικά δίκτυα σε βέλτιστες τοπολογίες, αναφορικά με τα αισθητήρια μοτίβα τους, ελέγχοντας κατάλληλα ταυτόχρονα την θέση και προσανατολισμό, μέσω ενός καινοτόμου σχήματος κατακερματισμού του χώρου βασιζόμενο στο εκάστοτε μοτίβο. Η διατριβή κλείνει με την μελέτη δικτύων με περιορισμούς στην εμβέλεια επικοινωνίας αναφορικά με την μετάδοση πληροφοριών μεταξύ των κόμβων. Στην πλειονότητα των σχετικών εργασιών, το ζήτημα αυτό προσπερνάται επιτρέποντας στην εμβέλεια επικοινωνίας να είναι τουλάχιστον διπλάσια αυτής της (ομοιόμορφης) αίσθησης, εγγυώντας έτσι την αποκεντρωμένη φύση των πλάνων ελέγχου. Ο προτεινόμενος έλεγχος επιτρέπει την αποσύζευξη μεταξύ των δύο αυτών εμβελειών, οδηγώντας το δίκτυο στην βέλτιστη κατάσταση, μέσω ταυτόχρονου σεβασμού του εκάστοτε, εκ των προτέρων δοσμένου, περιορισμού στην εμβέλεια επικοινωνίας. Συγκεντρωτικά συμπεράσματα και συγκριτική ανάλυση παρουσιάζονται στο τελευταίο κεφάλαιο, ενώ προτείνονται μελλοντικά πλάνα επέκτασης των τεχνικών αυτών. / The main scope of this thesis is the design and analysis of distributed control strategies for achieving optimum area coverage in mobile sensor networks. Due to the numerous applications of the latter in missions as area exploration, environmental sampling, patrolling, or even security, a large part of the scientific community has turned its interest on developing methods for achieving optimum, if possible, sensing environmental perception by groups of autonomous mobile agents. Such robotic teams, randomly deployed in areas of interest initially, are designed to coordinate their motion in a distributed manner, rather than via a global supervisory system, in order to succeed in the corresponding mission objective. At the first stages of this thesis, the coverage problem of an area of interest by a group of identical nodes is examined from a numerical point of view. The mobile nodes are considered to be governed by simple discrete–time kinodynamic motion, while their sensing performance is assumed radial, range–limited, uniform around the node. As a first approach, the optimum direction at each time step for optimum deployment achievement is determined based on proper distance–based space partitioning techniques. The developed concept allows for gradual increase in the covered area among consecutive steps, although suffers from allowing motion of one node at a time. In the sequel, the aforementioned concept is extended to the case of heterogeneous networks, where heterogeneity lays mainly in the unequal limited–range of the sensing performance of the nodes. In addition, extension to continuous–time allows for simultaneous motion of the nodes, increasing drastically the convergence time towards the optimal state, especially for large–scale networks. An alternate partitioning of the space is developed that is mainly based on the nodes’ footprints, rather than their spatial positions only. The resulting assigned cells form the main core for the coordination algorithm proposed, in order to distributedly organize the mobile swarm to achieve optimum sensing performance. Motivated by the high–degree anisotropy that governs the sensing domains of certain types of sensors, i.e. directional microphones for sound sensing mainly for security applications, or even the radiation patterns of directional antennas in communication–coverage scenarios, our research is extended beyond the standard disc model of sensing. Based on certain properties for planar convex curves, a distributed strategy is developed for networks characterized by convex sensing domains of same orientation. Although convexity of the sensing sets may seem to impose a high level restriction to the overall setup, in fact can be assigned as the maximal convex inscribed set in any (originally) anisotropic pattern. The control scheme is further extended, in the sequel, for the case of adding an extra degree of freedom to the node’s mobility abilities, incorporating different and time–varying orientations among the nodes patterns. The resulting scheme is proven to lead anisotropic networks in optimum configurations, considering their sensing footprints, by properly controlling both the nodes’ positions and orientations, via an innovative pattern–based partitioning scheme of the sensed space. The thesis ends by examining the case where radio–range constraints are imposed on inter–agents communication. In the majority of the related works, this issues is usually overcome by allowing RF range as double the sensing one, guaranteeing that way distributed nature of the control schemes. The proposed scheme allows for uncorrelated RF and sensing ranges in the network, while guarantees convergence of the network towards the optimal state, via simultaneous preservation of a–priori imposed radio–range constraints. Concluding remarks along with comparative discussion are presented in the last chapter, where future research plans and ways to improve the already developed schemes are proposed.

Abstract Wireless sensor networks (WSNs) consisting of battery-powered sensors are increasingly deployed for a myriad of Internet of Things applications, e.g., environmental, industrial, and healthcare monitoring. Since wireless access is typically the main contributor to battery usage, minimizing communications is crucial to prolong network lifetime and improve user experience. The objective of this thesis is to develop and analyze energy-efficient distributed compressed data acquisition techniques for WSNs. The thesis proposes four approaches to conserve sensors' energy by minimizing the amount of information each sensor has to transmit to meet given application requirements. The first part addresses a cross-layer design to minimize the sensors’ sum transmit power via joint optimization of resource allocation and multi-path routing. A distributed consensus optimization based algorithm is proposed to solve the problem. The algorithm is shown to have superior convergence compared to several baselines. The remaining parts deal with compressed sensing (CS) of sparse/compressible sources. The second part focuses on the distributed CS acquisition of spatially and temporally correlated sensor data streams. A CS algorithm based on sliding window and recursive decoding is developed. The method is shown to achieve higher reconstruction accuracy with fewer transmissions and less decoding delay and complexity compared to several baselines, and to progressively refine past estimates. The last two approaches incorporate the quantization of CS measurements and focus on lossy source coding. The third part addresses the distributed quantized CS (QCS) acquisition of correlated sparse sources. A distortion-rate optimized variable-rate QCS method is proposed. The method is shown to achieve higher distortion-rate performance than the baselines and to enable a trade-off between compression performance and encoding complexity via the pre-quantization of measurements. The fourth part investigates information-theoretic rate-distortion (RD) performance limits of single-sensor QCS. A lower bound to the best achievable compression — defined by the remote RD function (RDF) — is derived. A method to numerically approximate the remote RDF is proposed. The results compare practical QCS methods to the derived limits, and show a novel QCS method to approach the remote RDF. / Tiivistelmä Patterikäyttöisistä antureista koostuvat langattomat anturiverkot yleistyvät esineiden internetin myötä esim. ympäristö-, teollisuus-, ja terveydenhoitosovelluksissa. Koska langaton tiedonsiirto kuluttaa merkittävästi energiaa, kommunikoinnin minimointi on elintärkeää pidentämään verkon elinikää ja parantamaan käyttäjäkokemusta. Väitöskirjan tavoitteena on kehittää ja analysoida energiatehokkaita hajautettuja pakattuja datankeruumenetelmiä langattomiin anturiverkkoihin. Työssä ehdotetaan neljä lähestymistapaa, jotka säästävät anturien energiaa minimoimalla se tiedonsiirron määrä, mikä vaaditaan täyttämään sovelluksen asettamat kriteerit. Väitöskirjan ensimmäinen osa tarkastelee protokollakerrosten yhteissuunnittelua, jossa minimoidaan anturien yhteislähetysteho optimoimalla resurssiallokaatio ja monitiereititys. Ratkaisuksi ehdotetaan konsensukseen perustuva hajautettu algoritmi. Tulokset osoittavat algoritmin suppenemisominaisuuksien olevan verrokkejaan paremmat. Loppuosat keskittyvät harvojen lähteiden pakattuun havaintaan (compressed sensing, CS). Toinen osa keskittyy tila- ja aikatasossa korreloituneen anturidatan hajautettuun keräämiseen. Työssä kehitetään liukuvaan ikkunaan ja rekursiiviseen dekoodaukseen perustuva CS-algoritmi. Tulokset osoittavat menetelmän saavuttavan verrokkejaan korkeamman rekonstruktiotarkkuuden pienemmällä tiedonsiirrolla sekä dekoodausviiveellä ja -kompleksisuudella ja kykenevän asteittain parantamaan menneitä estimaatteja. Työn viimeiset osat sisällyttävät järjestelmämalliin CS-mittausten kvantisoinnin keskittyen häviölliseen lähdekoodaukseen. Kolmas osa käsittelee hajautettua korreloitujen harvojen signaalien kvantisoitua CS-havaintaa (quantized CS, QCS). Työssä ehdotetaan särön ja muuttuvan koodinopeuden välisen suhteen optimoiva QCS-menetelmä. Menetelmällä osoitetaan olevan verrokkejaan parempi pakkaustehokkuus sekä kyky painottaa suorituskyvyn ja enkooderin kompleksisuuden välillä mittausten esikvantisointia käyttäen. Neljäs osa tutkii informaatioteoreettisia, koodisuhde-särösuhteeseen perustuvia suorituskykyrajoja yhden anturin QCS-järjestelmässä. Parhaimmalle mahdolliselle pakkaustehokkuudelle johdetaan alaraja, sekä kehitetään menetelmä sen numeeriseen arviointiin. Tulokset vertaavat käytännön QCS-menetelmiä johdettuihin rajoihin, ja osoittavat ehdotetun QCS-menetelmän saavuttavan lähes optimaalinen suorituskyky.

Abstract This thesis examines radio resource management techniques for multicell multi-input single-output (MISO) downlink networks. Specifically, the thesis focuses on developing linear transmit beamforming techniques by optimizing certain quality-of-service (QoS) features, including, spectral efficiency, fairness, and throughput. The problem of weighted sum-rate-maximization (WSRMax) has been identified as a central problem to many network optimization methods, and it is known to be NP-hard. An algorithm based on a branch and bound (BB) technique which globally solves the WSRMax problem with an optimality certificate is proposed. Novel bounding techniques via conic optimization are introduced and their efficiency is illustrated by numerical simulations. The proposed BB based algorithm is not limited to WSRMax only; it can be easily extended to maximize any system performance metric that can be expressed as a Lipschitz continuous and increasing function of the signal-to-interference-plus-noise (SINR) ratio. Beamforming techniques can provide higher spectral efficiency, only when the channel state information (CSI) of users is accurately known. However, in practice the CSI is not perfect. By using an ellipsoidal uncertainty model for CSI errors, both optimal and suboptimal robust beamforming techniques for the worst-case WSRMax problem are proposed. The optimal method is based on a BB technique. The suboptimal algorithm is derived using alternating optimization and sequential convex programming. Through a numerical example it is also shown how the proposed algorithms can be applied to a scenario with statistical channel errors. Next two decentralized algorithms for multicell MISO networks are proposed. The optimization problems considered are: P1) minimization of the total transmission power subject to minimum SINR constraints of each user, and P2) SINR balancing subject to the total transmit power constraint of the base stations. Problem P1 is of great interest for obtaining a transmission strategy with minimal transmission power that can guarantee QoS for users. In a system where the power constraint is a strict system restriction, problem P2 is useful in providing fairness among the users. Decentralized algorithms for both problems are derived by using a consensus based alternating direction method of multipliers. Finally, the problem of spectrum sharing between two wireless operators in a dynamic MISO network environment is investigated. The notion of a two-person bargaining problem is used to model the spectrum sharing problem, and it is cast as a stochastic optimization. For this problem, both centralized and distributed dynamic resource allocation algorithms are proposed. The proposed distributed algorithm is more suitable for sharing the spectrum between the operators, as it requires a lower signaling overhead, compared with centralized one. Numerical results show that the proposed distributed algorithm achieves almost the same performance as the centralized one. / Tiivistelmä Tässä väitöskirjassa tarkastellaan monisoluisten laskevan siirtotien moniantennilähetystä käyttävien verkkojen radioresurssien hallintatekniikoita. Väitöskirjassa keskitytään erityisesti kehittämään lineaarisia siirron keilanmuodostustekniikoita optimoimalla tiettyjä palvelun laadun ominaisuuksia, kuten spektritehokkuutta, tasapuolisuutta ja välityskykyä. Painotetun summadatanopeuden maksimoinnin (WSRMax) ongelma on tunnistettu keskeiseksi monissa verkon optimointitavoissa ja sen tiedetään olevan NP-kova. Tässä työssä esitetään yleinen branch and bound (BB) -tekniikkaan perustuva algoritmi, joka ratkaisee WSRMax-ongelman globaalisti ja tuottaa todistuksen ratkaisun optimaalisuudesta. Samalla esitellään uusia conic-optimointia hyödyntäviä suorituskykyrajojen laskentatekniikoita, joiden tehokkuutta havainnollistetaan numeerisilla simuloinneilla. Ehdotettu BB-perusteinen algoritmi ei rajoitu pelkästään WSRMax-ongelmaan, vaan se voidaan helposti laajentaa maksimoimaan mikä tahansa järjestelmän suorituskykyarvo, joka voidaan ilmaista Lipschitz-jatkuvana ja signaali-(häiriö+kohina) -suhteen (SINR) kasvavana funktiona. Keilanmuodostustekniikat voivat tuottaa suuremman spektritehokkuuden vain, jos käyttäjien kanavien tilatiedot tiedetään tarkasti. Käytännössä kanavan tilatieto ei kuitenkaan ole täydellinen. Tässä väitöskirjassa ehdotetaan WSRMax-ongelman ääritapauksiin sekä optimaalinen että alioptimaalinen keilanmuodostustekniikka soveltaen tilatietovirheisiin ellipsoidista epävarmuusmallia. Optimaalinen tapa perustuu BB-tekniikkaan. Alioptimaalinen algoritmi johdetaan peräkkäistä konveksiohjelmointia käyttäen. Numeerisen esimerkin avulla näytetään, miten ehdotettuja algoritmeja voidaan soveltaa skenaarioon, jossa on tilastollisia kanavavirheitä. Seuraavaksi ehdotetaan kahta hajautettua algoritmia monisoluisiin moniantennilähetyksellä toimiviin verkkoihin. Tarkastelun kohteena olevat optimointiongelmat ovat: P1) lähetyksen kokonaistehon minimointi käyttäjäkohtaisten minimi-SINR-rajoitteiden mukaan ja P2) SINR:n tasapainottaminen tukiasemien kokonaislähetystehorajoitusten mukaisesti. Ongelma P1 on erittäin kiinnostava, kun pyritään kehittämään mahdollisimman pienen lähetystehon vaativa lähetysstrategia, joka pystyy takaamaan käyttäjien palvelun laadun. Ongelma P2 on hyödyllinen tiukasti tehorajoitetussa järjestelmässä, koska se tarjoaa tasapuolisuutta käyttäjien välillä. Molempien ongelmien hajautetut algoritmit johdetaan konsensusperusteisen vuorottelevan kertoimien suuntaustavan avulla. Lopuksi tarkastellaan kahden langattoman operaattorin välisen spektrinjaon ongelmaa dynaamisessa moniantennilähetystä käyttävässä verkkoympäristössä. Spektrinjako-ongelmaa mallinnetaan käyttämällä kahden osapuolen välistä neuvottelua stokastisen optimoinnin näkökulmasta. Tähän ongelmaan ehdotetaan ratkaisuksi sekä keskitettyä että hajautettua resurssien allokoinnin algoritmia. Hajautettu algoritmi sopii paremmin spektrin jakamiseen operaattorien välillä, koska se vaatii vähemmän kontrollisignalointia. Numeeriset tulokset osoittavat, että ehdotetulla hajautetulla algoritmilla saavutetaan lähes sama suorituskyky kuin keskitetyllä algoritmillakin.

<p>In the US, seven regional transmission organizations (RTOs) operate wholesale electricity markets within three largely independent transmission systems, the largest of which includes five RTO regions and many vertically integrated utilities.</p> <p>RTOs operate a day-ahead and a real-time market. In the day-ahead market, generation and demand-side resources are optimally scheduled based on bids and offers for the next day. Those schedules are adjusted according to actual operating conditions in the real-time market. Both markets involve a unit commitment calculation, a mixed integer program that determines which generators will be online, and an economic dispatch calculation, an optimization determines the output of each online generator for every interval and calculates locational marginal prices (LMPs).</p> <p>The use of LMPs for the management of congestion in RTO transmission systems has brought efficiency and transparency to the operation of electric power systems and provides price signals that highlight the need for investment in transmission and generation. Through this work, we aim to extend these efficiency and transparency gains to the coordination across RTOs. Existing market-based inter-regional coordination schemes are limited to incremental changes in real-time markets. </p> <p>We propose a multi-regional unit-commitment that enables coordination in the day-ahead timeframe by applying a distributed approach to approximate a system-wide optimal commitment and dispatch while allowing each region to largely maintain their own rules, model only internal transmission up to the boundary, and keep sensitive financial information confidential. A heuristic algorithm based on an extension of the alternating directions method of multipliers (ADMM) for the mixed integer program is applied to the unit commitment. </p> The proposed coordinated solution was simulated and compared to the ideal single-market scenario and to a representation of the current uncoordinated solution, achieving at least 58% of the maximum potential savings, which, in terms of the annual cost of electric generation in the US, could add up to nearly $7 billion per year. In addition to the coordinated day-ahead solution, we develop a distributed solution for financial transmission rights (FTR) auctions with minimal information sharing across RTOs that constitutes the first known work to provide a viable option for market participants to seamlessly hedge price variability exposure on cross-border transactions.

<p>Recent decades have witnessed the rise of data deluge generated by heterogeneous sources, e.g., social networks, streaming, marketing services etc., which has naturally created a surge of interests in theory and applications of large-scale convex and non-convex optimization. For example, real-world instances of statistical learning problems such as deep learning, recommendation systems, etc. can generate sheer volumes of spatially/temporally diverse data (up to Petabytes of data in commercial applications) with millions of decision variables to be optimized. Such problems are often referred to as Big-data problems. Solving these problems by standard optimization methods demands intractable amount of centralized storage and computational resources which is infeasible and is the foremost purpose of parallel and decentralized algorithms developed in this thesis.</p><p><br></p><p>This thesis consists of two parts: (I) Distributed Nonconvex Optimization and (II) Distributed Convex Optimization.</p><p><br></p><p>In Part (I), we start by studying a winning paradigm in big-data optimization, Block Coordinate Descent (BCD) algorithm, which cease to be effective when problem dimensions grow overwhelmingly. In particular, we considered a general family of constrained non-convex composite large-scale problems defined on multicore computing machines equipped with shared memory. We design a hybrid deterministic/random parallel algorithm to efficiently solve such problems combining synergically Successive Convex Approximation (SCA) with greedy/random dimensionality reduction techniques. We provide theoretical and empirical results showing efficacy of the proposed scheme in face of huge-scale problems. The next step is to broaden the network setting to general mesh networks modeled as directed graphs, and propose a class of gradient-tracking based algorithms with global convergence guarantees to critical points of the problem. We further explore the geometry of the landscape of the non-convex problems to establish second-order guarantees and strengthen our convergence to local optimal solutions results to global optimal solutions for a wide range of Machine Learning problems.</p><p><br></p><p>In Part (II), we focus on a family of distributed convex optimization problems defined over meshed networks. Relevant state-of-the-art algorithms often consider limited problem settings with pessimistic communication complexities with respect to the complexity of their centralized variants, which raises an important question: can one achieve the rate of centralized first-order methods over networks, and moreover, can one improve upon their communication costs by using higher-order local solvers? To answer these questions, we proposed an algorithm that utilizes surrogate objective functions in local solvers (hence going beyond first-order realms, such as proximal-gradient) coupled with a perturbed (push-sum) consensus mechanism that aims to track locally the gradient of the central objective function. The algorithm is proved to match the convergence rate of its centralized counterparts, up to multiplying network factors. When considering in particular, Empirical Risk Minimization (ERM) problems with statistically homogeneous data across the agents, our algorithm employing high-order surrogates provably achieves faster rates than what is achievable by first-order methods. Such improvements are made without exchanging any Hessian matrices over the network. </p><p><br></p><p>Finally, we focus on the ill-conditioning issue impacting the efficiency of decentralized first-order methods over networks which rendered them impractical both in terms of computation and communication cost. A natural solution is to develop distributed second-order methods, but their requisite for Hessian information incurs substantial communication overheads on the network. To work around such exorbitant communication costs, we propose a “statistically informed” preconditioned cubic regularized Newton method which provably improves upon the rates of first-order methods. The proposed scheme does not require communication of Hessian information in the network, and yet, achieves the iteration complexity of centralized second-order methods up to the statistical precision. In addition, (second-order) approximate nature of the utilized surrogate functions, improves upon the per-iteration computational cost of our earlier proposed scheme in this setting.</p>

No description available.

<p>The advent of advanced communication technologies has given rise to large-scale networks comprised of numerous interconnected agents, which need to cooperate to accomplish various tasks, such as distributed message routing, formation control, robust statistical inference, and spectrum access coordination. These tasks can be formulated as distributed optimization problems, which require agents to agree on a parameter minimizing the average of their local cost functions by communicating only with their neighbors. However, distributed optimization algorithms are typically susceptible to malicious (or "Byzantine") agents that do not follow the algorithm. This thesis offers analysis and algorithms for such scenarios. As the malicious agent's function can be modeled as an unknown function with some fundamental properties, we begin in the first two parts by analyzing the region containing the potential minimizers of a sum of functions. Specifically, we explicitly characterize the boundary of this region for the sum of two unknown functions with certain properties. In the third part, we develop resilient algorithms that allow correctly functioning agents to converge to a region containing the true minimizer under the assumption of convex functions of each regular agent. Finally, we present a general algorithmic framework that includes most state-of-the-art resilient algorithms. Under the strongly convex assumption, we derive a geometric rate of convergence of all regular agents to a ball around the optimal solution (whose size we characterize) for some algorithms within the framework.</p>

In dieser Arbeit beschäftigen wir uns mit dem Wasserstein Baryzentrumproblem diskreter Wahrscheinlichkeitsmaße sowie mit dem population Wasserstein Baryzentrumproblem gegeben von a Fréchet Mittelwerts von der rechnerischen und statistischen Seiten. Der statistische Fokus liegt auf der Schätzung der Stichprobengröße von Maßen zur Berechnung einer Annäherung des Fréchet Mittelwerts (Baryzentrum) der Wahrscheinlichkeitsmaße mit einer bestimmten Genauigkeit. Für empirische Risikominimierung (ERM) wird auch die Frage der Regularisierung untersucht zusammen mit dem Vorschlag einer neuen Regularisierung, die zu den besseren Komplexitätsgrenzen im Vergleich zur quadratischen Regularisierung beiträgt. Der Rechenfokus liegt auf der Entwicklung von dezentralen Algorithmen zurBerechnung von Wasserstein Baryzentrum: duale Algorithmen und Sattelpunktalgorithmen. Die Motivation für duale Optimierungsmethoden ist geschlossene Formen für die duale Formulierung von entropie-regulierten Wasserstein Distanz und ihren Derivaten, während, die primale Formulierung nur in einigen Fällen einen Ausdruck in geschlossener Form hat, z.B. für Gaußsches Maß. Außerdem kann das duale Orakel, das den Gradienten der dualen Darstellung für die entropie-regulierte Wasserstein Distanz zurückgibt, zu einem günstigeren Preis berechnet werden als das primale Orakel, das den Gradienten der (entropie-regulierten) Wasserstein Distanz zurückgibt. Die Anzahl der dualen Orakel rufe ist in diesem Fall ebenfalls weniger, nämlich die Quadratwurzel der Anzahl der primalen Orakelrufe. Im Gegensatz zum primalen Zielfunktion, hat das duale Zielfunktion Lipschitz-stetig Gradient aufgrund der starken Konvexität regulierter Wasserstein Distanz. Außerdem untersuchen wir die Sattelpunktformulierung des (nicht regulierten) Wasserstein Baryzentrum, die zum Bilinearsattelpunktproblem führt. Dieser Ansatz ermöglicht es uns auch, optimale Komplexitätsgrenzen zu erhalten, und kann einfach in einer dezentralen Weise präsentiert werden. / In this thesis, we consider the Wasserstein barycenter problem of discrete probability measures as well as the population Wasserstein barycenter problem given by a Fréchet mean from computational and statistical sides. The statistical focus is estimating the sample size of measures needed to calculate an approximation of a Fréchet mean (barycenter) of probability distributions with a given precision. For empirical risk minimization approaches, the question of the regularization is also studied along with proposing a new regularization which contributes to the better complexity bounds in comparison with the quadratic regularization. The computational focus is developing decentralized algorithms for calculating Wasserstein barycenters: dual algorithms and saddle point algorithms. The motivation for dual approaches is closed-forms for the dual formulation of entropy-regularized Wasserstein distances and their derivatives, whereas the primal formulation has a closed-form expression only in some cases, e.g., for Gaussian measures.Moreover, the dual oracle returning the gradient of the dual representation forentropy-regularized Wasserstein distance can be computed for a cheaper price in comparison with the primal oracle returning the gradient of the (entropy-regularized) Wasserstein distance. The number of dual oracle calls in this case will be also less, i.e., the square root of the number of primal oracle calls. Furthermore, in contrast to the primal objective, the dual objective has Lipschitz continuous gradient due to the strong convexity of regularized Wasserstein distances. Moreover, we study saddle-point formulation of the non-regularized Wasserstein barycenter problem which leads to the bilinear saddle-point problem. This approach also allows us to get optimal complexity bounds and it can be easily presented in a decentralized setup.