The necessity for clean and sustainable energy has shifted the energy sector’s interest in renewable energy sources. Photovoltaics (PV) is the most popular renewable energy source because the sun is ubiquitous. However, several discrepancies exist in a PV system when implemented for real-world applications. Among several other existing problems related to Photovoltaics, in this work, we deal with maximum power point tracking (MPPT) under Partial Shading (PS) conditions. MPPT is a mechanism formulated as an optimization problem adjusting the PV to deliver the maximum power to the load. Under full insolation conditions, varying solar panel temperatures, and different loads MPPT problem is a convex optimization problem. However, when the PV’s surface is partially shaded, multiple power peaks are created in the power versus voltage (P-V) curve making MPPT non-convex.

The ever-expanding volume of data generated by network devices such as smartphones, personal computers, and sensors has significantly contributed to the remarkable advancements in artificial intelligence (AI) and machine learning (ML) algorithms. However, effectively processing and learning from this extensive data usually requires substantial computational capabilities centralized in a server. Moreover, concerns regarding data privacy arise when collecting training data from distributed network devices. To address these challenges, collaborative ML with decentralized data has emerged as a promising solution for large-scale machine learning across distributed devices, driven by the parallel computing and learning trends. Collaborative and distributed ML can be broadly classified into two types: server-based and fully decentralized, based on whether the model aggregation is coordinated by a parameter server or performed in a decentralized manner through peer-to-peer communication. In cases where communication between devices occurs over wireless links, which are inherently imperfect, unreliable, and resource-constrained, how can we design communication protocols to achieve the best learning performance? This thesis investigates decentralized learning using decentralized stochastic gradient descent, an established algorithm for decentralized ML, in a novel setting with imperfect and constrained communication. "Imperfect" implies that communication can fail and "constrained" implies that communication resources are limited. The communication across a link between two devices is modeled as a binary event with either success or failure, depending on if multiple neighbouring devices are transmitting information. To compensate for communication failures, every communication round can have multiple communication slots, which are limited and must be carefully allocated over the learning process. The quality of communication is quantified by introducing normalized throughput, describing the ratio of successful links in a communication round. To decide when devices should broadcast, both random and deterministic medium access policies have been developed with the goal of maximizing throughput, which has shown very efficient learning performance. Finally, two schemes for allocating communication slots over communication rounds have been defined and simulated: Delayed-Allocation and the Periodic-Allocation schemes, showing that it is better to allocate slots late rather than early, and neither too frequently nor infrequently which can depend on several factors and requires further study

Functionally graded materials (FGMs) are inhomogeneous materials in which the material properties vary with respect to space. Research has been done by scientific community in developing techniques like photothermal radiometry (PTR) to measure the thermal conductivity and volumetric heat capacity of FGMs. One of the problems involved in the technique is to solve the inverse problem, i.e., estimating the thermal properties after the frequency scan has been obtained. The present work involves finding the unknown thermal conductivity and volumetric heat capacity of the FGMs by using finite volume method. By taking the flux entering the sample as periodic and solving the discretized 1-D thermal wave field equation at a frequency domain, one can obtain the complex temperatures at the surface of the sample for each frequency. These complex temperatures when solved for a range of frequencies gives the phase vs frequency scan which can then be compared to original frequency scan obtained from the PTR experiment by using a residual function. Brute force and gradient descent optimization methods have been implemented to estimate the unknown thermal conductivity and volumetric specific heat of the FGMs through minimization of the residual function. In general, the spatial composition profile of the FGMs can be approximated by using a smooth curve. Three functional forms namely Arctangent curve, Hermite curve, and Bezier curve are used in approximating the thermal conductivity and volumetric heat capacity distributions in the FGMs. The use of Hermite and Bezier curves gives the flexibility to control the slope of the curve i.e. the thermal property distribution along the thickness of the sample. Two-layered samples with constant thermal properties and three layered samples in which one of the layer has varying thermal properties with respect to thickness are considered. The program is written in Fortran and several test runs are performed. Results obtained are close to the original thermal property values with some deviation based on the stopping criteria used in the gradient descent algorithm. Calculating the gradients at each iteration takes considerable amount of time and if these gradient values are already available, the problem can be solved at a faster rate. One of the methods is extending automatic differentiation to complex numbers and calculating the gradient values ahead; this is left for future work.

No description available.

No description available.

Concorder un modèle à certaines observations, voilà qui résume assez bien ce que l’apprentissage machine cherche à accomplir. Ce concept est maintenant omniprésent dans nos vies, entre autre grâce aux percées récentes en apprentissage profond. La stratégie d’optimisation prédominante pour ces deux domaines est la minimisation d’un objectif donné. Et pour cela, la méthode du gradient, méthode de premier-ordre qui modifie les paramètres du modèle à chaque itération, est l’approche dominante. À l’opposé, les méthodes dites de second ordre n’ont jamais réussi à s’imposer en apprentissage profond. Pourtant, elles offrent des avantages reconnus qui soulèvent encore un grand intérêt. D’où l’importance de la méthode du col, qui unifie les méthodes de premier et second ordre sous un même paradigme. Dans ce mémoire, nous établissons un parralèle direct entre la méthode du col et le domaine du contrôle optimal ; domaine qui cherche à optimiser mathématiquement une séquence de décisions. Et certains des problèmes les mieux compris et étudiés en contrôle optimal sont les commandes linéaires quadratiques. Problèmes pour lesquels on connaît très bien la solution optimale. Plus spécifiquement, nous démontrerons l’équivalence entre une itération de la méthode du col et la résolution d’une Commande Linéaire Quadratique (CLQ). Cet éclairage nouveau implique une approche unifiée quand vient le temps de déployer nombre d’algorithmes issus de la méthode du col, tel que la méthode du gradient et celle des gradients naturels, sans être limitée à ceux-ci. Approche que nous étendons ensuite aux problèmes à horizon infini, tel que les modèles à équilibre profond. Ce faisant, nous démontrons pour ces problèmes que calculer les gradients via la différentiation implicite revient à employer l’équation de Riccati pour solutionner la CLQ associée à la méthode du gradient. Finalement, notons que l’incorporation d’information sur la courbure du problème revient généralement à rencontrer une inversion matricielle dans la méthode du col. Nous montrons que l’équivalence avec les CLQ permet de contourner cette inversion en utilisant une approximation issue des séries de Neumann. Surprenamment, certaines observations empiriques suggèrent que cette approximation aide aussi à stabiliser le processus d’optimisation quand des méthodes de second-ordre sont impliquées ; en agissant comme un régularisateur adaptif implicite. / Machine learning entails training a model to fit some given observations, and recent advances in the field, particularly in deep learning, have made it omnipresent in our lives. Fitting a model usually requires the minimization of a given objective. When it comes to deep learning, first-order methods like gradient descent have become a default tool for optimization in deep learning. On the other hand, second-order methods did not see widespread use in deep learning. Yet, they hold many promises and are still a very active field of research. An important perspective into both methods is steepest descent, which allows you to encompass first and second-order approaches into the same framework. In this thesis, we establish an explicit connection between steepest descent and optimal control, a field that tries to optimize sequential decision-making processes. Core to it is the family of problems known as Linear Quadratic Regulation; problems that have been well studied and for which we know optimal solutions. More specifically, we show that performing one iteration of steepest descent is equivalent to solving a Linear Quadratic Regulator (LQR). This perspective gives us a convenient and unified framework for deploying a wide range of steepest descent algorithms, such as gradient descent and natural gradient descent, but certainly not limited to. This framework can also be extended to problems with an infinite horizon, such as deep equilibrium models. Doing so reveals that retrieving the gradient via implicit differentiation is equivalent to recovering it via Riccati’s solution to the LQR associated with gradient descent. Finally, incorporating curvature information into steepest descent usually takes the form of a matrix inversion. However, casting a steepest descent step as a LQR also hints toward a trick that allows to sidestep this inversion, by leveraging Neumann’s series approximation. Empirical observations provide evidence that this approximation actually helps to stabilize the training process, by acting as an adaptive damping parameter.

Artificiella neurala nätverk (ANN) har ett brett tillämpningsområde och blir allt relevantare på flera håll, inte minst för självkörande fordon. För att träna nätverken användsmeta-algoritmer. Nätverken kan styra fordonen med hjälp av olika typer av indata. I detta projekt har vi undersökt två meta-algoritmer: genetisk algoritm (GA) och gradient descent tillsammans med bakåtpropagering (GD & BP). Vi har även undersökt två typer av indata: avståndssensorer och linjedetektering. Vi redogör för teorin bakom de metoder vi har försökt implementera. Vi lyckades inte använda GD & BP för att träna nätverk att köra fordon, men vi redogör för hur vi försökte. I resultatdelen redovisar vi hur det med GA gick att träna ANN som använder avståndssensorer och linjedetektering som indata. Sammanfattningsvis lyckades vi implementera självkörande fordon med två olika typer av indata. / Artificial Neural Networks (ANN) have a broad area of application and are growing increasingly relevant, not least in the field of autonomous vehicles. Meta algorithms are used to train networks, which can control a vehicle using several kinds of input data. In this project we have looked at two meta algorithms: genetic algorithm (GA), and gradient descent with backpropagation (GD & BP). We have looked at two types of input to the ANN: distance sensors and line detection. We explain the theory behind the methods we have tried to implement. We did not succeed in using GD & BP to train ANNs to control vehicles, but we describe our attemps. We did however succeeded in using GA to train ANNs using a combination of distance sensors and line detection as input. In summary we managed to train ANNs to control vehicles using two methods of input, and we encountered interesting problems along the way.

Nous présentons deux contributions majeures : 1) une combinaison de plusieurs descripteurs d’images pour la classification à grande échelle, 2) des algorithmes parallèles de SVM pour la classification d’images à grande échelle. Nous proposons aussi un algorithme incrémental et parallèle de classification lorsque les données ne peuvent plus tenir en mémoire vive. / We have proposed a novel method of combination multiple of different features for image classification. For large scale learning classifiers, we have developed the parallel versions of both state-of-the-art linear and nonlinear SVMs. We have also proposed a novel algorithm to extend stochastic gradient descent SVM for large scale learning. A class of large scale incremental SVM classifiers has been developed in order to perform classification tasks on large datasets with very large number of classes and training data can not fit into memory.

現今許多網路商店或電子商務將產品銷售給消費者的過程中，皆使用推薦系統的幫助來提高銷售量。如亞馬遜公司(Amazon)、Netflix，深入了解顧客的使用習慣，建構專屬的推薦系統並進行個性化的推薦商品給每一位顧客。 推薦系統應用的技術分為協同過濾和內容過濾兩大類，本研究旨在探討協同過濾推薦系統中潛在因子模型方法，利用矩陣分解法找出評分矩陣。在Koren等人(2009)中，將矩陣分解法的演算法大致分為兩種，隨機梯度下降法(Stochastic gradient descent)與交替最小平方法(Alternating least squares)。本研究主要研究目的有三項，一為比較交替最小平方法與隨機梯度下降法的預測能力，二為兩種矩陣分解演算法在加入偏誤項後的表現，三為先完成交替最小平方法與隨機梯度下降法，以其預測值對原始資料之遺失值進行資料插補，再利用奇異值分解法對完整資料做矩陣分解，觀察其前後方法的差異。 研究結果顯示，隨機梯度下降法所需的運算時間比交替最小平方法所需的運算時間少。另外，完成兩種矩陣分解演算法後，將預測值插補遺失值，進行奇異值分解的結果也顯示預測能力有提升。 / Recommender system has been largely used by Internet companies such Amazon and Netflix to make recommendations for Internet users. Techniques for recommender systems can be divided into content filtering approach and collaborative filtering approach. Matrix factorization is a popular method for collaborative filtering approach. It minimizes the object function through stochastic gradient descent and alternating least squares. This thesis has three goals. First, we compare the alternating least squares method and stochastic gradient descent method. Secondly, we compare the performance of matrix factorization method with and without the bias term. Thirdly, we combine singular value decomposition and matrix factorization. As expected, we found the stochastic gradient descent takes less time than the alternating least squares method, and the the matrix factorization method with bias term gives more accurate prediction. We also found that combining singular value decomposition with matrix factorization can improve the predictive accuracy.

Given the overall increase in the availability of computational resources, and the importance of forecasting the future, it should come as no surprise that prediction is considered to be one of the most compelling and challenging problems for both academia and industry in the world of data analytics. But how is prediction done, what factors make it easier or harder to do, how accurate can we expect the results to be, and can we harness the available computational resources in meaningful ways? With efforts ranging from those designed to save lives in the moments before a near field tsunami to others attempting to predict the performance of Major League Baseball players, future generations need to have realistic expectations about prediction methods and analytics. This thesis takes a broad look at the problem, including motivation, methodology, accuracy, and infrastructure. In particular, a careful study involving experiments in regression, the prediction of continuous, numerical values, and classification, the assignment of a class to each sample, is provided. The results and conclusions of these experiments cover only the included data sets and the applied algorithms as implemented by the Python library. The evaluation includes accuracy and running time of different algorithms across several data sets to establish tradeoffs between the approaches, and determine the impact of variations in the size of the data sets involved. As scalability is a key characteristic required to meet the needs of future prediction problems, a discussion of some of the challenges associated with parallelization is included. / Graduate / 0984 / erickson@uvic.ca