No description available.

Machine learning (ML) has demonstrated great success in numerous complicated tasks. Fueled by these advances, many real-world systems like autonomous vehicles and aircraft are adopting ML techniques by adding learning-enabled components. Unfortunately, ML models widely used today, like neural networks, lack the necessary mathematical framework to provide formal guarantees on safety, causing growing concerns over these learning-enabled systems in safety-critical settings. In this dissertation, we tackle this problem by combining formal methods and machine learning to bring provable safety and robustness to learning-enabled systems. We first study the robustness verification problem of neural networks on classification tasks. We focus on providing provable safety guarantees on the absence of failures under arbitrarily strong adversaries. We propose an efficient neural network verifier LayR to compute a guaranteed and overapproximated range for the output of a neural network given an input set which contains all possible adversarially perturbed inputs. LayR relaxes nonlinear units in neural networks using linear bounds and refines such relaxations with mixed integer linear programming (MILP) to iteratively improve the approximation precision, which achieves tighter output range estimations compared to prior neural network verifiers. However, the neural network verifier focuses more on analyzing a trained neural network but less on learning provably safe neural networks. To tackle this problem, we study verifiable training that incorporates verification into training procedures to train provably safe neural networks and scale to larger models and datasets. We propose a novel framework, AdvIBP, for combining adversarial training and provable robustness verification. We show that the proposed framework can learn provable robust neural networks at a sublinear convergence rate. In the second part of the dissertation, we study the verification of system-level properties in neural-network controlled systems (NNCS). We focus on proving bounded-time safety properties by computing reachable sets. We first introduce two efficient NNCS verifiers ReachNN* and POLAR that construct polynomial-based overapproximations of neural-network controllers. We transfer NNCSs to tractable closed-loop systems with approximated polynomial controllers for computing reachable sets using existing reachability analysis tools of dynamical systems. The combination of polynomial overapproximations and reachability analysis tools opens promising directions for NNCS verification. We also include a survey and experimental study of existing NNCS verification methods, including combining state-of-the-art neural network verifiers with reachability analysis tools, to discuss what overapproximation is suitable for NNCS reachability analysis. While these verifiers enable proving safety properties of NNCS, the nonlinearity of neural-network controllers is the main bottleneck that limits their efficiency and scalability. We propose a novel framework of knowledge distillation to control “the degree of nonlinearity” of NN controllers to ease NNCS verification which improves provable safety of NNCSs especially when they are safe but cannot be verified due to their complexity. For the verification community, this opens up the possibility of reducing verification complexity by influencing how a system is trained. Though NNCS verification can prove safety when system models are known, modern deep learning, e.g., deep reinforcement learning (DRL), often targets tasks with unknown system models, also known as the model-free setting. To tackle this issue, we first focus on safe exploration of DRL and propose a novel Lyapunov-inspired method. Our method uses Gaussian Process models to provide probabilistic guarantees on the policies, and guide the exploration of the unknown environment in a safe fashion. Then, we study learning robust visual control policies in DRL to enhance the robustness against visual changes that were not seen during training. We propose a method DRIBO, which can learn robust state representations for RL via a novel contrastive version of the Multi-View Information Bottleneck (MIB). This approach enables us to train high-performance visual policies that are robust to visual distractions, and can generalize well to unseen environments.

The security of multi-robot systems is critical in various applications such as patrol, transportation, and search and rescue operations, where they face threats from adversaries attempting to gain control of the robots. These compromised robots are significant threats as they allow attackers to steer robots towards forbidden areas without being detected, potentially causing harm or compromising the mission. To address this problem, we propose a resilient planning, task assignment, and control framework. The proposed framework builds a multi-robot plan where robots are designed to get close enough to other robots according to a co-observation schedule, in order to mutually check for abnormal behaviors. For the first part of the thesis, we propose an optimal trajectory solver based on the alternating direction method of multipliers (ADMM) to generate multi-agent trajectories that satisfy spatio-temporal requirements introduced by the co-observation schedules. As part of the formulation, we provide a new reachability constraint to guarantee that, despite adversarial movement by the attacker, a compromised robot cannot reach forbidden areas between co-observations without being detected. In the second part of the thesis, to further enhance the system's performance, reliability, and robustness, we propose to deploy multiple robots on each route to form sub-teams. A new cross-trajectory co-observation scheme between sub-teams is introduced that preserves the optimal unsecured trajectories. The new planner ensures that at least one robot in each sub-team sticks to the planned trajectories, while sub-teams can constantly exchange robots during the task introducing additional co-observations that can secure originally unsecured routes. We show that the planning of cross-trajectory co-observations can be transformed into a network flow problem and solved using traditional linear program technique. In the final part of the thesis, we show that the introduction of sub-teams also improves the multi-robot system's robustness to unplanned situations, allowing servicing unplanned online events without breaking the security requirements. This is achieved by a distributed task assignment algorithm based on consensus ADMM which can handle tasks with different priorities. The assignment result and security requirements are formulated as spatio-temporal schedules and guaranteed through control barrier function (CBF) based controls.

This doctoral thesis considers the automatic verification of parameterized systems, i.e. systems with an arbitrary number of communicating components, such as mutual exclusion protocols, cache coherence protocols or heap manipulating programs. The components may be organized in various topologies such as words, multisets, rings, or trees. The task is to show correctness regardless of the size of the system and we consider two methods to prove safety:(i) a backward reachability analysis, using the well-quasi ordered framework and monotonic abstraction, and (ii) a forward analysis which only needs to inspect a small number of components in order to show correctness of the whole system. The latter relies on an abstraction function that views the system from the perspective of a fixed number of components. The abstraction is used during the verification procedure in order to dynamically detect cut-off points beyond which the search of the state-space need not continue. Our experimentation on a variety of benchmarks demonstrate that the method is highly efficient and that it works well even for classes of systems with undecidable property. It has been, for example, successfully applied to verify a fine-grained model of Szymanski's mutual exclusion protocol. Finally, we applied the methods to solve the complex problem of verifying highly concurrent data-structures, in a challenging setting: We do not a priori bound the number of threads, the size of the data-structure, the domain of the data to store nor do we require the presence of a garbage collector. We successfully verified the concurrent Treiber's stack and Michael & Scott's queue, in the aforementioned setting. To the best of our knowledge, these verification problems have been considered challenging in the parameterized verification community and could not be carried out automatically by other existing methods.

Este trabalho é dedicado ao estudo do erro de estimação em filtragem linear para sistemas lineares com parâmentros sujeitos a saltos markovianos a tempo discreto. Indroduzimos o conceito de alcançabilidade média para uma classe de sistemas. Construímos um conjunto de matrizes de alcançabilidade e mostramos que o conceito usual de alcan- çabilidade definido através da positividade do gramiano é caracterizado pela definição por posto completo destas matrizes. A alcançabilidade média funciona como condição necessária e suficiente para positividade do segundo momento do estado do sistema, resultado esse que auxilia na caracterização da positividade uniforme da matriz de covariância do erro de estimação. Abordamos a estabilidade de estimadores com a interpretação de que a covariância do erro permanece limitada na presença de erro de qualquer magnitude no modelo do ruído, que é uma característica relevante para aplicações. Apresentamos uma prova de que filtros markovianos são estáveis sempre que o segundo momento condicionado é positivo. Exemplos numéricos encontram-se inclusos. / This work studies linear filtering for discrete-time systems with Markov jump parameters. We introduce a notion of average reachability for these systems and present a set of matrices playing the role of reachability matrices, in the sense that their rank is full if and only if the system is average reachable. Reachability is also a sufficient condition for the second moment of the system to be positive. Uniform positiveness of the error covariance matrix is studied for general (possibly non-markovian) linear estimators, relying on the state second moment positiveness. Satbility of linear markovian estimators is also addressed, allowing to show that markovian estimators are stable whenever the system is reachable, with the interpretation that the error covariance remains bounded in the presence of error of any magnitude in the model of the noise, which is a relevant feature for applications. Numerical examples are included.

Neste trabalho estudamos as noções de alcançabilidade e controlabilidade para sistemas lineares a tempo contínuo com perturbações aditivas e saltos nos parâmetros sujeitos a uma cadeia de Markov geral. Definimos conceitos de alcançabilidade e controlabilidade médios de maneira natural exigindo que os valores esperados dos gramianos correspondentes sejam definidos positivos. Visando obter uma condição testável para ambos os conceitos, introduzimos conjuntos de matrizes de alcançabilidade e de controlabilidade para esta classe de sistemas e usamos certas propriedades de invariância para mostrar que: o sistema é alcançável em média, e, analogamente, controlável em média, se e somente se as matrizes respectivas, de alcançabilidade e de controlabilidade, têm posto completo. Usamos alcançabilidade média de sistemas para mostrar que a matriz de segundo momento do estado é definida positiva com uma margem uniforme. Uma consequência deste resultado no problema de estimação linear do estado é que a matriz de covariância do erro de estimação é positiva definida em média, no sentido que existe um nível mínimo de ruído nas estimativas. Na sequência, para estimadores lineares markovianos, estudamos a limitação do valor esperado da matriz de covariância do erro para mostrar que o filtro é estável num certo sentido, sendo esta uma propriedade desejável em aplicações reais. Quanto às aplicações da controlabilidade média, usamos este conceito para estabelecer condições necessárias e suficientes que garantem a existência de um processo de controle que leva a componente contínua do estado do sistema para a origem em tempo finito e com probabilidade positiva. / In this work we study the reachability and controllability notions for continuous-time linear systems with exogenous inputs and jump parameters driven by a quite general Markov chain. We define a rather natural average reachability and controllability concepts by requiring that the associated gramians are average positive definite, respectively. Aiming at testable conditions for each concept, we introduce certain sets of matrices linked with the gramians, and employ some invariance properties to find rank-based conditions. We show for average reachable systems that the state second moment is positive definite. One consequence of this result in the context of linear estimation for reachable systems is that the expectation of the error covariance matrix is positive definite. Moreover, for linear markovian filters we study the average boundedness of the error covariance matrix to show that the filter is stable in an appropriate sense, which consists in a property that is desirable in real applications. Regarding the average controllability concept, we show that it is a necessary and sufficient condition for the feasibility of the following control problem: find a control process that drives the continuous component of the state to zero in finite time with positive probability.

Para satisfazer o explosivo aumento na demanda de tráfego de voz e dados, as redes ópticas baseadas em WDM e GMPLS estão sendo desenvolvidas. A suíte de protocolos GMPLS é atualmente considerada como um plano de controle para as redes ópticas e é composta por protocolos de sinalização e de roteamento, como também do protocolo de gerenciamento de enlace (LMP). O LMP é um importante protocolo que interfere na atribuição de rótulos (comprimentos de onda) e é necessário fazer sua análise lógica para verificar se o mesmo está livre de erros de progresso. Para esta finalidade, o método denominado alcançabilidade justa foi utilizado. Verificada a corretude do LMP, o estudo foca o subproblema de atribuição de comprimento de onda do RWA nas redes GMPLS-WDM por ser um dos principais problemas que causam o baixo desempenho destas redes. O cenário estudado é das redes GMPLS-WDM que operam em um ambiente RWA dinâmico com restrição de continuidade de comprimento de onda. O problema RWA é examinado bem como as várias heurísticas de atribuição de comprimento de onda apresentadas na literatura. Com o objetivo de melhorar o desempenho das redes GMPLS-WDM com restrição de continuidade de comprimento de onda, propõe-se um algoritmo de atribuição de rótulos que utiliza os conceitos conjunto de rótulos e SRLG já implementados pelo GMPLS. O algoritmo proposto melhora a eficiência no uso de recursos nas redes em questão. O desempenho é verificado através da métricas de probabilidade de bloqueio de conexão, desempenho este próximo do ótimo e demonstrado através de simulações. / To satisfy the explosive increasing demands of voice and data traffic, optical networks based on WDM and GMPLS are being developed. The GMPLS´ suite of protocols is currently being considered as the control plane for optical networks and it is compounded of signaling and routing protocols, and also the link management protocol (LMP). The LMP is an important protocol that interferes with label (wavelength) assignment and it is necessary to logically analyse this protocol in order to verify if it is free from progress errors. For this purpose, the method called fair reachability has been used. Verified the LMP is correctable, the study focuses on the RWA wavelength assignment problem in GMPLS-WDM networks because it is one of the main problems which causes the low performance of these networks. The studied scene is GMPLS-WDM networks operating under a dynamic RWA environment with wavelength continuity constraint. The RWA problem is examined and also the various wavelength-assignment heuristics proposed in the literature. With the goal to improve the performance of the GMPLS-WDM networks with wavelength continuity constraint, it is proposed a label assignment algorithm, which uses the concepts of label set and SRLG, already implemented by GMPLS. The proposed algorithm provides an improvement in efficiency of resource use. The performance is verified by using the blocking probability metric, and it is very close to the optimum and demonstrated through simulations.

Graphen sind in vielen Bereichen des Lebens zu finden, wobei wir speziell an Graphen in der Biologie interessiert sind. Knoten in solchen Graphen sind chemische Komponenten, Enzyme, Reaktionen oder Interaktionen, die durch Kanten miteinander verbunden sind. Eine effiziente Ausführung von Graphanfragen ist eine Herausforderung. In dieser Arbeit präsentieren wir GRIcano, ein System, das die effiziente Ausführung von Graphanfragen erlaubt. Wir nehmen an, dass Graphen in relationalen Datenbankmanagementsystemen (RDBMS) gespeichert sind. Als Graphanfragesprache schlagen wir eine erweiterte Version der Pathway Query Language (PQL) vor. Der Hauptbestandteil von GRIcano ist ein kostenbasierter Anfrageoptimierer. Diese Arbeit enthält Beiträge zu allen drei benötigten Komponenten des Optimierers, der relationalen Algebra, Implementierungen und Kostenmodellen. Die Operatoren der relationalen Algebra sind nicht ausreichend, um Graphanfragen auszudrücken. Daher stellen wir zuerst neue Operatoren vor. Wir schlagen den Erreichbarkeits-, Distanz-, Pfadlängen- und Pfadoperator vor. Zusätzlich geben wir Regeln für die Umformung von Ausdrücken an. Des Weiteren präsentieren wir Implementierungen für jeden vorgeschlagenen Operator. Der Hauptbeitrag ist GRIPP, eine Indexstruktur, die die effiziente Ausführung von Erreichbarkeitsanfragen auf sehr großen Graphen erlaubt. Wir zeigen, wie GRIPP und die rekursive Anfragestrategie genutzt werden können, um Implementierungen für alle Operatoren bereitzustellen. Die dritte Komponente von GRIcano ist das Kostenmodell, das Kardinalitätsabschätzungen der Operatoren und Kostenfunktionen für die Implementierungen benötigt. Basierend auf umfangreichen Experimenten schlagen wir in dieser Arbeit Funktionen dafür vor. Der neue Ansatz unserer Kostenmodelle ist, dass die Funktionen nur Kennzahlen der Graphen verwenden. Abschließend zeigen wir die Wirkungsweise von GRIcano durch Beispielanfragen auf echten biologischen Graphen. / Graphs occur in many areas of life. We are interested in graphs in biology, where nodes are chemical compounds, enzymes, reactions, or interactions that are connected by edges. Efficiently querying these graphs is a challenging task. In this thesis we present GRIcano, a system that efficiently executes graph queries. For GRIcano we assume that graphs are stored and queried using relational database management systems (RDBMS). We propose an extended version of the Pathway Query Language PQL to express graph queries. The core of GRIcano is a cost-based query optimizer. This thesis makes contributions to all three required components of the optimizer, the relational algebra, implementations, and cost model. Relational algebra operators alone are not sufficient to express graph queries. Thus, we first present new operators to rewrite PQL queries to algebra expressions. We propose the reachability, distance, path length, and path operator. In addition, we provide rewrite rules for the newly proposed operators in combination with standard relational algebra operators. Secondly, we present implementations for each proposed operator. The main contribution is GRIPP, an index structure that allows us to answer reachability queries on very large graphs. GRIPP has advantages over other existing index structures, which we review in this work. In addition, we show how to employ GRIPP and the recursive query strategy as implementation for all four proposed operators. The third component of GRIcano is the cost model, which requires cardinality estimates for operators and cost functions for implementations. Based on extensive experimental evaluation of our proposed algorithms we present functions to estimate the cardinality of operators and the cost of executing a query. The novelty of our approach is that these functions only use key figures of the graph. We finally present the effectiveness of GRIcano using exemplary graph queries on real biological networks.

No description available.

No description available.