In the process industries, cooling capacity is an important enabler for the facility to manufacture on specification product. The cooling water network is an important part of the over-all cooling system of the facility. In this research a cooling water circuit consisting of 3 cooling towers in parallel, 2 cooling water pumps in parallel, and 11 heat exchangers in parallel, is modelled. The model developed is based on first principles and captures the dynamic, non-linear, interactive nature of the plant. The modelled plant is further complicated by continuous, as well as discrete process variables, giving the model a hybrid nature. Energy consumption is included in the model as it is a very important parameter for plant operation. The model is fitted to real industry data by using a particle swarm optimisation approach. The model is suitable to be used for optimisation and control purposes. Cooling water networks are often not instrumented and actuated, nor controlled or optimised. Significant process benefits can be achieved by better process end-user temperature control, and direct monetary benefits can be obtained from electric power minimisation. A Hybrid Non-Linear Model Predictive Control strategy is developed for these control objectives, and simulated on the developed first principles dynamic model. Continuous and hybrid control cases are developed, and tested on process scenarios that reflect conditions seen in a real plant. Various alternative techniques are evaluated in order to solve the Hybrid Non-Linear Control problem. Gradient descent with momentum is chosen and configured to be used to solve the continuous control problem. For the discrete control problem a graph traversal algorithm is developed and joined to the continuous control algorithm to form a Hybrid Non-Linear Model Predictive controller. The potential monetary benefits that can be obtained by the plant owner through implementing the designed control strategy, are estimated. A powerful computation platform is designed for the plant model and controller simulations. / Thesis (PhD)--University of Pretoria, 2019. / Electrical, Electronic and Computer Engineering / PhD / Unrestricted

The necessity for clean and sustainable energy has shifted the energy sector’s interest in renewable energy sources. Photovoltaics (PV) is the most popular renewable energy source because the sun is ubiquitous. However, several discrepancies exist in a PV system when implemented for real-world applications. Among several other existing problems related to Photovoltaics, in this work, we deal with maximum power point tracking (MPPT) under Partial Shading (PS) conditions. MPPT is a mechanism formulated as an optimization problem adjusting the PV to deliver the maximum power to the load. Under full insolation conditions, varying solar panel temperatures, and different loads MPPT problem is a convex optimization problem. However, when the PV’s surface is partially shaded, multiple power peaks are created in the power versus voltage (P-V) curve making MPPT non-convex.

The ever-expanding volume of data generated by network devices such as smartphones, personal computers, and sensors has significantly contributed to the remarkable advancements in artificial intelligence (AI) and machine learning (ML) algorithms. However, effectively processing and learning from this extensive data usually requires substantial computational capabilities centralized in a server. Moreover, concerns regarding data privacy arise when collecting training data from distributed network devices. To address these challenges, collaborative ML with decentralized data has emerged as a promising solution for large-scale machine learning across distributed devices, driven by the parallel computing and learning trends. Collaborative and distributed ML can be broadly classified into two types: server-based and fully decentralized, based on whether the model aggregation is coordinated by a parameter server or performed in a decentralized manner through peer-to-peer communication. In cases where communication between devices occurs over wireless links, which are inherently imperfect, unreliable, and resource-constrained, how can we design communication protocols to achieve the best learning performance? This thesis investigates decentralized learning using decentralized stochastic gradient descent, an established algorithm for decentralized ML, in a novel setting with imperfect and constrained communication. "Imperfect" implies that communication can fail and "constrained" implies that communication resources are limited. The communication across a link between two devices is modeled as a binary event with either success or failure, depending on if multiple neighbouring devices are transmitting information. To compensate for communication failures, every communication round can have multiple communication slots, which are limited and must be carefully allocated over the learning process. The quality of communication is quantified by introducing normalized throughput, describing the ratio of successful links in a communication round. To decide when devices should broadcast, both random and deterministic medium access policies have been developed with the goal of maximizing throughput, which has shown very efficient learning performance. Finally, two schemes for allocating communication slots over communication rounds have been defined and simulated: Delayed-Allocation and the Periodic-Allocation schemes, showing that it is better to allocate slots late rather than early, and neither too frequently nor infrequently which can depend on several factors and requires further study

No description available.

No description available.

Functionally graded materials (FGMs) are inhomogeneous materials in which the material properties vary with respect to space. Research has been done by scientific community in developing techniques like photothermal radiometry (PTR) to measure the thermal conductivity and volumetric heat capacity of FGMs. One of the problems involved in the technique is to solve the inverse problem, i.e., estimating the thermal properties after the frequency scan has been obtained. The present work involves finding the unknown thermal conductivity and volumetric heat capacity of the FGMs by using finite volume method. By taking the flux entering the sample as periodic and solving the discretized 1-D thermal wave field equation at a frequency domain, one can obtain the complex temperatures at the surface of the sample for each frequency. These complex temperatures when solved for a range of frequencies gives the phase vs frequency scan which can then be compared to original frequency scan obtained from the PTR experiment by using a residual function. Brute force and gradient descent optimization methods have been implemented to estimate the unknown thermal conductivity and volumetric specific heat of the FGMs through minimization of the residual function. In general, the spatial composition profile of the FGMs can be approximated by using a smooth curve. Three functional forms namely Arctangent curve, Hermite curve, and Bezier curve are used in approximating the thermal conductivity and volumetric heat capacity distributions in the FGMs. The use of Hermite and Bezier curves gives the flexibility to control the slope of the curve i.e. the thermal property distribution along the thickness of the sample. Two-layered samples with constant thermal properties and three layered samples in which one of the layer has varying thermal properties with respect to thickness are considered. The program is written in Fortran and several test runs are performed. Results obtained are close to the original thermal property values with some deviation based on the stopping criteria used in the gradient descent algorithm. Calculating the gradients at each iteration takes considerable amount of time and if these gradient values are already available, the problem can be solved at a faster rate. One of the methods is extending automatic differentiation to complex numbers and calculating the gradient values ahead; this is left for future work.

Diese Dissertation ist ein Beitrag zum Forschungsfeld Early stopping im Kontext iterativer Schätzverfahren. Wir betrachten Early stopping dabei sowohl aus der Perspektive impliziter Regularisierungsverfahren als auch aus der Perspektive adaptiver Methoden Analog zu expliziter Regularisierung reduziert das Stoppen eines Schätzverfahrens den stochastischen Fehler/die Varianz des endgültigen Schätzers auf Kosten eines zusätzlichen Approximationsfehlers/Bias. In diesem Forschungsbereich präsentieren wir eine neue Analyse des Gradientenabstiegsverfahrens für konvexe Lernprobleme in einem abstrakten Hilbert-Raum. Aus der Perspektive adaptiver Methoden müssen iterative Schätzerverfahren immer mit einer datengetriebenen letzten Iteration m kombiniert werden, die sowohl under- als auch over-fitting verhindert. In diesem Forschungsbereichpräsentieren wir zwei Beiträge: In einem statistischen inversen Problem, das durch iteratives Trunkieren der Singulärwertzerlegung regularisiert wird, untersuchen wir, unter welchen Umständen optimale Adaptiertheit erreicht werden kann, wenn wir an der ersten Iteration m stoppen, an der die geglätteten Residuen kleiner sind als ein kritischer Wert. Für L2-Boosting mittels Orthogonal Matching Pursuit (OMP) in hochdimensionalen linearen Modellen beweisen wir, dass sequenzielle Stoppverfahren statistische Optimalität garantieren können. Die Beweise beinhalten eine subtile punktweise Analyse einer stochastischen Bias-Varianz-Zerlegung, die durch den Greedy-Algorithmus, der OMP unterliegt, induziert wird. Simulationsstudien zeigen, dass sequentielle Methoden zu deutlich reduzierten Rechenkosten die Leistung von Standardalgorithmen wie dem kreuzvalidierten Lasso oder der nicht-sequentiellen Modellwahl über ein hochdimensionales Akaike- Kriterium erbringen können. / This dissertation contributes to the growing literature on early stopping in modern statistics and machine learning. We consider early stopping from the perspective of both implicit regularization and adaptive estimation. From the former, analogous to an explicit regularization method, halting an iterative estimation procedure reduces the stochastic error/variance of the final estimator at the cost of some bias. In this area, we present a novel analysis of gradient descent learning for convex loss functions in an abstract Hilbert space setting, which combines techniques from inexact optimization and concentration of measure. From the perspective of adaptive estimation, iterative estimation procedures have to be combined with a data-driven choice m of the effectively selected iteration in order to avoid under- as well as over-fitting. In this area, we present two contributions: For truncated SVD estimation in statistical inverse problems, we examine under what circumstances optimal adaptation can be achieved by early stopping at the first iteration at which the smoothed residuals are smaller than a critical value. For L2-boosting via orthogonal matching pursuit (OMP) in high dimensional linear models, we prove that sequential early stopping rules can preserve statistical optimality in terms of a general oracle inequality for the empirical risk and recently established optimal convergence rates for the population risk.

Concorder un modèle à certaines observations, voilà qui résume assez bien ce que l’apprentissage machine cherche à accomplir. Ce concept est maintenant omniprésent dans nos vies, entre autre grâce aux percées récentes en apprentissage profond. La stratégie d’optimisation prédominante pour ces deux domaines est la minimisation d’un objectif donné. Et pour cela, la méthode du gradient, méthode de premier-ordre qui modifie les paramètres du modèle à chaque itération, est l’approche dominante. À l’opposé, les méthodes dites de second ordre n’ont jamais réussi à s’imposer en apprentissage profond. Pourtant, elles offrent des avantages reconnus qui soulèvent encore un grand intérêt. D’où l’importance de la méthode du col, qui unifie les méthodes de premier et second ordre sous un même paradigme. Dans ce mémoire, nous établissons un parralèle direct entre la méthode du col et le domaine du contrôle optimal ; domaine qui cherche à optimiser mathématiquement une séquence de décisions. Et certains des problèmes les mieux compris et étudiés en contrôle optimal sont les commandes linéaires quadratiques. Problèmes pour lesquels on connaît très bien la solution optimale. Plus spécifiquement, nous démontrerons l’équivalence entre une itération de la méthode du col et la résolution d’une Commande Linéaire Quadratique (CLQ). Cet éclairage nouveau implique une approche unifiée quand vient le temps de déployer nombre d’algorithmes issus de la méthode du col, tel que la méthode du gradient et celle des gradients naturels, sans être limitée à ceux-ci. Approche que nous étendons ensuite aux problèmes à horizon infini, tel que les modèles à équilibre profond. Ce faisant, nous démontrons pour ces problèmes que calculer les gradients via la différentiation implicite revient à employer l’équation de Riccati pour solutionner la CLQ associée à la méthode du gradient. Finalement, notons que l’incorporation d’information sur la courbure du problème revient généralement à rencontrer une inversion matricielle dans la méthode du col. Nous montrons que l’équivalence avec les CLQ permet de contourner cette inversion en utilisant une approximation issue des séries de Neumann. Surprenamment, certaines observations empiriques suggèrent que cette approximation aide aussi à stabiliser le processus d’optimisation quand des méthodes de second-ordre sont impliquées ; en agissant comme un régularisateur adaptif implicite. / Machine learning entails training a model to fit some given observations, and recent advances in the field, particularly in deep learning, have made it omnipresent in our lives. Fitting a model usually requires the minimization of a given objective. When it comes to deep learning, first-order methods like gradient descent have become a default tool for optimization in deep learning. On the other hand, second-order methods did not see widespread use in deep learning. Yet, they hold many promises and are still a very active field of research. An important perspective into both methods is steepest descent, which allows you to encompass first and second-order approaches into the same framework. In this thesis, we establish an explicit connection between steepest descent and optimal control, a field that tries to optimize sequential decision-making processes. Core to it is the family of problems known as Linear Quadratic Regulation; problems that have been well studied and for which we know optimal solutions. More specifically, we show that performing one iteration of steepest descent is equivalent to solving a Linear Quadratic Regulator (LQR). This perspective gives us a convenient and unified framework for deploying a wide range of steepest descent algorithms, such as gradient descent and natural gradient descent, but certainly not limited to. This framework can also be extended to problems with an infinite horizon, such as deep equilibrium models. Doing so reveals that retrieving the gradient via implicit differentiation is equivalent to recovering it via Riccati’s solution to the LQR associated with gradient descent. Finally, incorporating curvature information into steepest descent usually takes the form of a matrix inversion. However, casting a steepest descent step as a LQR also hints toward a trick that allows to sidestep this inversion, by leveraging Neumann’s series approximation. Empirical observations provide evidence that this approximation actually helps to stabilize the training process, by acting as an adaptive damping parameter.

Artificiella neurala nätverk (ANN) har ett brett tillämpningsområde och blir allt relevantare på flera håll, inte minst för självkörande fordon. För att träna nätverken användsmeta-algoritmer. Nätverken kan styra fordonen med hjälp av olika typer av indata. I detta projekt har vi undersökt två meta-algoritmer: genetisk algoritm (GA) och gradient descent tillsammans med bakåtpropagering (GD & BP). Vi har även undersökt två typer av indata: avståndssensorer och linjedetektering. Vi redogör för teorin bakom de metoder vi har försökt implementera. Vi lyckades inte använda GD & BP för att träna nätverk att köra fordon, men vi redogör för hur vi försökte. I resultatdelen redovisar vi hur det med GA gick att träna ANN som använder avståndssensorer och linjedetektering som indata. Sammanfattningsvis lyckades vi implementera självkörande fordon med två olika typer av indata. / Artificial Neural Networks (ANN) have a broad area of application and are growing increasingly relevant, not least in the field of autonomous vehicles. Meta algorithms are used to train networks, which can control a vehicle using several kinds of input data. In this project we have looked at two meta algorithms: genetic algorithm (GA), and gradient descent with backpropagation (GD & BP). We have looked at two types of input to the ANN: distance sensors and line detection. We explain the theory behind the methods we have tried to implement. We did not succeed in using GD & BP to train ANNs to control vehicles, but we describe our attemps. We did however succeeded in using GA to train ANNs using a combination of distance sensors and line detection as input. In summary we managed to train ANNs to control vehicles using two methods of input, and we encountered interesting problems along the way.

Nous présentons deux contributions majeures : 1) une combinaison de plusieurs descripteurs d’images pour la classification à grande échelle, 2) des algorithmes parallèles de SVM pour la classification d’images à grande échelle. Nous proposons aussi un algorithme incrémental et parallèle de classification lorsque les données ne peuvent plus tenir en mémoire vive. / We have proposed a novel method of combination multiple of different features for image classification. For large scale learning classifiers, we have developed the parallel versions of both state-of-the-art linear and nonlinear SVMs. We have also proposed a novel algorithm to extend stochastic gradient descent SVM for large scale learning. A class of large scale incremental SVM classifiers has been developed in order to perform classification tasks on large datasets with very large number of classes and training data can not fit into memory.