In smart energy systems, load curve refers to the time series reported by smart meters, which indicate the energy consumption of customers over a certain period of time. The widespread use of load curve (data) in demand side management and demand response programs makes it one of the most important resources. To capture the load behavior or energy consumption patterns, load curve modeling is widely applied to help the utilities and residents make better plans and decisions. In this dissertation, with the help of load curve modeling, we focus on data-driven solutions to three load monitoring problems in different scenarios of smart energy systems, including residential power systems and datacenter power systems and covering the research fields of i) data cleansing, ii) energy disaggregation, and iii) fine-grained power monitoring. First, to improve the data quality for load curve modeling on the supply side, we challenge the regression-based approaches as an efficient way to load curve data cleansing and propose a new approach to analyzing and organizing load curve data. Our approach adopts a new view, termed portrait, on the load curve data by analyzing the inherent periodic patterns and re-organizing the data for ease of analysis. Furthermore, we introduce strategies to build virtual portrait datasets and demonstrate how this technique can be used for outlier detection in load curve. To identify the corrupted load curve data, we propose an appliance-driven approach that particularly takes advantage of information available on the demand side. It identifies corrupted data from the smart meter readings by solving a carefully-designed optimization problem. To solve the problem efficiently, we further develop a sequential local optimization algorithm that tackles the original NP-hard problem by solving an approximate problem in polynomial time. Second, to separate the aggregated energy consumption of a residential house into that of individual appliances, we propose a practical and universal energy disaggregation solution, only referring to the readily available information of appliances. Based on the sparsity of appliances' switching events, we first build a sparse switching event recovering (SSER) model. Then, by making use of the active epochs of switching events, we develop an efficient parallel local optimization algorithm to solve our model and obtain individual appliances' energy consumption. To explore the benefit of introducing low-cost energy meters for energy disaggregation, we propose a semi-intrusive appliance load monitoring (SIALM) approach for large-scale appliances situation. Instead of using only one meter, multiple meters are distributed in the power network to collect the aggregated load data from sub-groups of appliances. The proposed SSER model and parallel optimization algorithm are used for energy disaggregation within each sub-group of appliances. We further provide the sufficient conditions for unambiguous state recovery of multiple appliances, under which a minimum number of meters is obtained via a greedy clique-covering algorithm. Third, to achieve fine-grained power monitoring at server level in legacy datacenters, we present a zero-cost, purely software-based solution. With our solution, no power monitoring hardware is needed any more, leading to much reduced operating cost and hardware complexity. In detail, we establish power mapping functions (PMFs) between the states of servers and their power consumption, and infer the power consumption of each server with the aggregated power of the entire datacenter. We implement and evaluate our solution over a real-world datacenter with 326 servers. The results show that our solution can provide high precision power estimation at both the rack level and the server level. In specific, with PMFs including only two nonlinear terms, our power estimation i) at the rack level has mean relative error of 2.18%, and ii) at the server level has mean relative errors of 9.61% and 7.53% corresponding to the idle and peak power, respectively. / Graduate / 0984 / 0791 / 0800 / tangguo1999@gmail.com

The average global temperature is rising due to climate change. This leads to an increase in cooling demand along with higher usage of electricity to operate cooling processes. One way to decrease the electricity usage is to introduce absorption cooling which uses heat instead of electricity as its main source of power. To further increase resource efficiency in urban areas centralized district cooling can substitute independent cooling units. In a district cooling network, a mixture of absorption and compressor cooling units, as well as free cooling, can be included. This enables the ability to coordinate which cooling technology is to be used based on profitability at the current time. By introducing an optimization-based plan, the operation of a district cooling network in a smart energy system can incorporate important factors for the interaction between different sectors, such as electricity and district heating prices. The usage of optimization-based tools to plan the operation of energy systems has previously shown promising results. However, further studies are needed to investigate how they perform in different scenarios. There is also a need to develope more reliable forecasts which motivated this study; a case study on the district cooling network "City" in Linköping.  The study aimed to develope a method for forecasting the cooling demand in a district cooling network, investigating how the coordination of absorption and compressor cooling units, as well as free cooling, can be improved. This has been done from a system perspective that encompasses the district heating and electricity network by developing an optimization-based operational plan. In this study an explorative method has been used to develope a forecasting tool based on an algorithm and a Mixed Integer Linear Programming (MILP) model with appertaining constraints and coefficients which can solve an Unit Commitment problem for a district cooling network. The forecasting tool and MILP model resulted in an optimization-based operational plan that enabled the ability to coordinate the usage between absorption and compressor cooling units as well as free cooling. The method can be divided into five distinct iterative steps; (1) data collection for the parameters that affect the cooling demand, (2) forecasting of the cooling demand based on the identified parameters, (3) pressure simulations of Linköping's district cooling network in the software NetSim, (4) operational optimization via MILP modeling, and (5) evaluation of the optimization-based operational plan from the perspective of operational cost, electricity and heat usage, as well as greenhouse gas emissions. Six different algorithms were developed to forecast the cooling demand. All of the algorithms were based on the retrospective operation the previous day through linear regressions. The algorithm that best followed a historical operational period on the district cooling network City had a margin of error of 14\%. The algorithm was based on the time of the day and either solar irradiation or outside temperature based on the difference between the forecasted outdoor temperature and the measured temperature the previous day. The MILP model that was developed had an objective function that minimized the operational cost which included the cost of electricity and heat usage, distribution, maintenance, and start-up and shut-down costs. The constraints that was constructed in the MILP model to define a district cooling network included balancing the cooling demand, specifications for the operation of cooling units and distribution flows. Furthermore, the coefficients that defined the City network were estimated dynamically. These included power limitations, operational costs, and start-up costs for each cooling unit, as well as distribution costs for each cooling plant.  During this case-study, it was observed that by using optimization-based operational planning produced from a MILP model solving an UC problem, the operational costs, electricity and heat usage can decrease by 27\%, 22\%, and 2\% respectively for this case-study of the City network in Linköping during a seven-month period. In addition, a decrease in greenhouse gases by 16\% was observed when applying the perspective "avoided global emissions". For the calculations an emission factor of 702 $gram \, CO_2-eq/kWh_{el}$ and 130, 72, or 3 $gram \, CO_2-eq/kWh_{heat}$ depending on if waste, bio-oil, or recycled waste wood were used as fuel for the marginal production of district heating. When there was excess heat in the district heating network the emission factor for heat usage was instead assumed to be 0 $gram \, CO_2-eq/kWh_{heat}$. Lastly, this case-study emphasizes the importance of solid operational planning as a foundational pillar in satisfying the increase of future cooling demand in a resource-efficient way for local energy systems in sustainable societies.

<p dir="ltr">The horizon for inclusion of data-driven algorithms in cyber-physical systems is rapidly expanding due to evermore availability of high-performance computing tools and the inception of novel mathematical models in the fields of deep learning and reinforcement learning. In this regard, energy systems are a suitable candidate for data-driven algorithms utilization due to rapid expansion of smart measuring tools and infrastructure. Accordingly, I decided to explore the capabilities of deep reinforcement learning in control, security, and restoration of smart energy systems to tackle well-known problems such as ensuring stability, adversarial attack avoidance, and the black start restoration. To achieve this goal, I employed various reinforcement learning techniques in different capacities to develop transfer learning modules based on a rule-based approach for online control of the power system, utilized reinforcement learning for procedural noise generation in adversarial attacks against contingency detection in a power system and exploited multiple reinforcement learning algorithms to fully restore an energy system in an optimal manner. Per the results of these endeavors, I managed to develop a rule-based transfer learning logic to control the power system under various disturbance types and intensities. Furthermore, I developed an optimal adversarial attack module using a reinforcement-learning-based procedural noise generation to avoid detection by conventional deep-learning-based detection. Finally for the system restoration, the proposed intelligent restoration module managed to provide sustainable results for the black start restoration in energy system.</p>