With the integration of renewable and distributed energy resources (DER) and advances in metering infrastructure, power systems are undergoing rapid modernization that brings forward new challenges and possibilities, which call for more advanced learning, analysis, and planning tools. While there are numerous problems present in the modern power grid, in this work, this work has addressed four of the most prominent challenges and has shown that how the new advances in generation and metering can be leveraged to address the challenges that arose by them. With regards to learning in power systems, we first have tackled power distribution system topology identification, since knowing the topology of the power grid is a crucial piece in any meaningful optimization and control task. The topology identification presented in this work is based on the idea of emph{prob-to-learn}, which is perturbing the power grid with small power injections and using the metered response to learn the topology. By using maximum-likelihood estimation, we were able to formulate the topology identification problem as a mixed-integer linear program. We next have tackled the prominent challenge of finding optimal flexibility of aggregators in distribution systems, which is a crucial step in utilizing the capacity of distributed energy resources as well as flexible loads of the distribution systems and to aid transmission systems to be more efficient and reliable. We have shown that the aggregate flexibility of a group of devices with uncertainties and non-convex models can be captured with a quadratic classifier and using that classifier we can design a virtual battery model that best describes the aggregate flexibility. For power system analysis and planning, we have addressed fast probabilistic hosting capacity analysis (PHCA), which is studying how DERs and the intermittency that they bring to the power system can impact the power grid operation in the long term. We have shown that interconnection studies can be sped up by a factor of 20 without losing any accuracy. By formulating a penalized optimal power flow (OPF), we were able to pose PHCA as an instance of multiparametric programming (MPP), and then leveraged the nice properties of MPP to efficiently solve a large number of OPFs. Regarding planning in power systems, we have tackled the problem of strategic investment in energy markets, in which we have utilized the powerful toolbox of multiparametric programming to develop two algorithms for strategic investment. Our MPP-aided grid search algorithm is useful when the investor is only considering a few locations and our MPP-aided gradient descent algorithm is useful for investing in a large number of locations. We next have presented a data-driven approach in finding the flexibility of aggregators in power systems. Finding aggregate flexibility is an important step in utilizing the full potential of smart and controllable loads in the power grid and it's challenging since an aggregator controls a large group of time-coupled devices that operate with non-convex models and are subject to random externalities. We have shown that the aggregate flexibility can be accurately captured with an ellipsoid and then used Farkas' lemma to fit a maximal volume polytope inside the aforementioned ellipsoid. The numerical test showcases that we can capture 10 times the volume that conventional virtual generator models can capture. / Doctor of Philosophy / With the integration of renewable and distributed energy resources (DER) and advances in metering infrastructure, power systems are undergoing rapid modernization that brings forward new challenges and possibilities, which call for more advanced learning, analysis, and planning tools. While there are numerous problems present in the modern power grid, in this work, this work has addressed four of the most prominent challenges and has shown that how the new advances in generation and metering can be leveraged to address the challenges that arose by them. With regards to learning in power systems, we first have tackled power distribution system topology identification, since knowing the topology of the power grid is a crucial piece in any meaningful optimization and control task. We next have tackled the prominent challenge of finding optimal flexibility of aggregators in distribution systems, which is a crucial step in utilizing the capacity of distributed energy resources as well as flexible loads of the distribution systems and to aid transmission systems to be more efficient and reliable. For power system analysis and planning, we have addressed fast probabilistic hosting capacity analysis (PHCA), which is studying how DERs and the intermittency that they bring to the power system can impact the power grid operation in the long term. We have shown that interconnection studies can be sped up by a factor of 20 without losing any accuracy. Regarding planning in power systems, we have tackled the problem of strategic investment in energy markets, in which we have utilized the powerful toolbox of multiparametric programming to develop two algorithms for strategic investment. We next have presented a data-driven approach in finding the flexibility of aggregators in power systems. Finding aggregate flexibility is an important step in utilizing the full potential of smart and controllable loads in the power grid and it's challenging since an aggregator controls a large group of time-coupled devices that operate with non-convex models and are subject to random externalities.

The measurability of Mobile ad hoc network (MANET) is the precondition of itsmanagement, performance optimization and network resources re-allocations. However, MANET is an infrastructure-free, multi-hop, andself-organized temporary network, comprised of a group of mobile nodes with wirelesscommunication devices. Not only does its topology structure vary with time going by, butalso the communication protocol used in its network layer or data link layer is diverse andnon-standard.In order to solve the problem of interior links performance (such as packet loss rate anddelay) measurement in MANET, this thesis has adopted an external measurement basedon network tomography (NT). To the best of our knowledge, NT technique is adaptable for Ad Hoc networkmeasurement.This thesis has deeply studied MANET measurement technique based on NT. The maincontributions are:(1) An analysis technique on MANET topology dynamic characteristic based onmobility model was proposed. At first, an Ad Hoc network mobility model formalizationis described. Then a MANET topology snapshots capturing method was proposed to findand verify that MANET topology varies in steady and non-steady state in turnperiodically. At the same time, it was proved that it was practicable in theory to introduceNT technique into Ad Hoc network measurement. The fitness hypothesis verification wasadopted to obtain the rule of Ad Hoc network topology dynamic characteristic parameters,and the Markov stochastic process was adopted to analyze MANET topology dynamiccharacteristic. The simulation results show that the method above not only is valid andgenerable to be used for all mobility models in NS-2 Tool, but also could obtain thetopology state keeping experimental formula and topology state varying probabilityformula.IV(2) An analysis technique for MANET topology dynamic characteristic based onmeasurement sample was proposed. When the scenario file of mobile models could notbe obtained beforehand, End-to-End measurement was used in MANET to obtain thepath delay time. Then topology steady period of MANET is inferred by judging whetherpath delay dithering is close to zero. At the same time, the MANET topology wasidentified by using hierarchical clustering method based on measurement sample of pathperformance during topology steady period in order to support the link performanceinference. The simulation result verified that the method above could not only detect themeasurement window time of MANET effectively, but also identify the MANETtopology architecture during measurement window time correctly.(3) A MANET link performance inference algorithm based on linear analysis modelwas proposed. The relation of inequality between link and path performance, such as lossrate of MANET, was deduced according to a linear model. The phenomena thatcommunication characteristic of packets, such as delay and loss rate, is more similarwhen the sub-paths has longer shared links was proved in the document. When the rankof the routing matrix is equal to that of its augmentation matrix, the linear model wasused to describe the Ad Hoc network link performance inference method. The simulationresults show that the algorithm not only is effective, but also has short computing time.(4) A Link performance inference algorithm based on multi-objectives optimizationwas proposed. When the rank of the routing matrix is not equal to that of its augmentationmatrix, the link performance inference was changed into multi-objectives optimizationand genetic algorithm is used to infer link performance. The probability distribution oflink performance in certain time t was obtained by performing more measurements andstatistically analyzing the hypo-solutions. Through the simulation, it can be safelyconcluded that the internal link performance, such as, link loss ratio and link delay, can beinferred correctly when the rank of the routing matrix is not equal to that of itsaugmentation matrix.

Performance analysis of Digital Subscriber Line (DSL) technologies, which are implemented on existing telephone subscriber loops, is of vital importance to DSL service providers. This type of analysis requires accurate prediction of the local loop structure and precise identification of the cable parameters. These cables are the main components of the loop and are typically comprised of multi-conductor twisted pair type currently being used on existing telephone subscriber loops. This system identification problem was investigated through the application of single port measurements, with preference being placed on measurements taken from the service provider's end of the loop under investigation. Once the cabling system has been identified, the performance analysis of the loop was obtained through simulation. Accurate modeling is an important aspect of any system identification solution; therefore, the modeling of the twisted pair cables was thoroughly investigated in this research. Early modeling attempts of twisted pair cabling systems for use with (DSL) technology has not been vigorously investigated due to the difficulty in obtaining wideband physical data necessary for the task as well as the limitations of simulators to accurately model the skin effects of the conductors. Models are developed in this research that produce a wideband representation of the twisted pair cables through the use of the data measured in high frequency spectra. The twisted-pair cable models were then applied to the system identification problem through a de-embedding type approach. The identification process accurately characterizes the sections of the subscriber loop closest to the measurements node, and these identified sections were then modeled and de-embedded from the system measurement in a layer removing, or "peeling", type process. After each identified section was de-embedded from the system measurement, the process was repeated until the entire system was identified. Upon completion of the system identification process, the resulting system model was simulated between the central office (CO) and resulting identified customer nodes for the evaluation of performance analysis. The performance analysis allows the providers to identify points where the DSL technology is feasible, and where so, the rates of the data transfer to the nodes that can be expected. / Ph. D.

Les réseaux de capteurs sans fils et mobiles constituent un champ de recherche dans lequel un grand nombre de capteurs de faible coût sont déployés dans un environnement pour observer un ou plusieurs phénomènes. Ces capteurs sont autonomes, communicant et disposent d'une réserve d'énergie limitée. Les problèmes issus de ce type de système sont nombreux : gestion de l'énergie, couverture, fusion de donnée, ...L'approche proposée dans cette thèse repose sur l'hypothèse que les réseaux de capteurs doivent exhiber des propriétés d'auto-organisation et d'autonomie. Chaque capteur est en soit autonome et peut interagir avec d'autres capteurs ce qui forme une organisation complexe. Ces capteurs ont un but à accomplir et le système possède les caractéristiques suivantes : i. le but du réseau ne peut généralement pas être résolu par un capteur uniqueii. Les capteurs doivent collaborer pour accomplir le but ou contribuer à une partie de ce but.iii. Chaque capteur réagit à son environnement en fonction de ses perceptions qui sont forcément locales et limitées.Après une introduction qui décrit le domaine et pose la problématique un état de l'art du domaine est présenté au chapitre 2. Deux contributions sont abordées dans cette thèse. D'une part, l'analyse des propriétés dynamiques de topologie des réseaux de capteurs sans fil et d'autre part la performance des liens de ce type de réseaux. Pour la topologie deux approches sont proposées : au chapitre 3 une première approche basée sur le modèle de mobilité et au chapitre 4 une approche basée sur des techniques de mesures. Pour la performance des liens, deux approches sont également proposées. La première, décrite dans le chapitre 5, est basée sur un modèle d'analyse linéaire. La deuxième, décrite au chapitre 6, repose sur une technique d'optimisation multi-objectif. / The measurability of Mobile ad hoc network (MANET) is the precondition of itsmanagement, performance optimization and network resources re-allocations. However, MANET is an infrastructure-free, multi-hop, andself-organized temporary network, comprised of a group of mobile nodes with wirelesscommunication devices. Not only does its topology structure vary with time going by, butalso the communication protocol used in its network layer or data link layer is diverse andnon-standard.In order to solve the problem of interior links performance (such as packet loss rate anddelay) measurement in MANET, this thesis has adopted an external measurement basedon network tomography (NT). To the best of our knowledge, NT technique is adaptable for Ad Hoc networkmeasurement.This thesis has deeply studied MANET measurement technique based on NT. The maincontributions are:(1) An analysis technique on MANET topology dynamic characteristic based onmobility model was proposed. At first, an Ad Hoc network mobility model formalizationis described. Then a MANET topology snapshots capturing method was proposed to findand verify that MANET topology varies in steady and non-steady state in turnperiodically. At the same time, it was proved that it was practicable in theory to introduceNT technique into Ad Hoc network measurement. The fitness hypothesis verification wasadopted to obtain the rule of Ad Hoc network topology dynamic characteristic parameters,and the Markov stochastic process was adopted to analyze MANET topology dynamiccharacteristic. The simulation results show that the method above not only is valid andgenerable to be used for all mobility models in NS-2 Tool, but also could obtain thetopology state keeping experimental formula and topology state varying probabilityformula.IV(2) An analysis technique for MANET topology dynamic characteristic based onmeasurement sample was proposed. When the scenario file of mobile models could notbe obtained beforehand, End-to-End measurement was used in MANET to obtain thepath delay time. Then topology steady period of MANET is inferred by judging whetherpath delay dithering is close to zero. At the same time, the MANET topology wasidentified by using hierarchical clustering method based on measurement sample of pathperformance during topology steady period in order to support the link performanceinference. The simulation result verified that the method above could not only detect themeasurement window time of MANET effectively, but also identify the MANETtopology architecture during measurement window time correctly.(3) A MANET link performance inference algorithm based on linear analysis modelwas proposed. The relation of inequality between link and path performance, such as lossrate of MANET, was deduced according to a linear model. The phenomena thatcommunication characteristic of packets, such as delay and loss rate, is more similarwhen the sub-paths has longer shared links was proved in the document. When the rankof the routing matrix is equal to that of its augmentation matrix, the linear model wasused to describe the Ad Hoc network link performance inference method. The simulationresults show that the algorithm not only is effective, but also has short computing time.(4) A Link performance inference algorithm based on multi-objectives optimizationwas proposed. When the rank of the routing matrix is not equal to that of its augmentationmatrix, the link performance inference was changed into multi-objectives optimizationand genetic algorithm is used to infer link performance. The probability distribution oflink performance in certain time t was obtained by performing more measurements andstatistically analyzing the hypo-solutions. Through the simulation, it can be safelyconcluded that the internal link performance, such as, link loss ratio and link delay, can beinferred correctly when the rank of the routing matrix is not equal to that of itsaugmentation matrix.