Thesis--Auburn University, 2009. / Abstract. Vita. Includes bibliographic references (p.50-52).

No description available.

A new neural network approach for performing matrix computations is presented. The idea of this approach is to construct a feed-forward neural network (FNN) and then train it by matching a desired set of patterns. The solution of the problem is the converged weight of the FNN. Accordingly, unlike the conventional FNN research that concentrates on external properties (mappings) of the networks, this study concentrates on the internal properties (weights) of the network. The present network is linear and its weights are usually strongly constrained; hence, complicated overlapped network needs to be construct. It should be noticed, however, that the present approach depends highly on the training algorithm of the FNN. Unfortunately, the available training methods; such as, the original Back-propagation (BP) algorithm, encounter many deficiencies when applied to matrix algebra problems; e. g., slow convergence due to improper choice of learning rates (LR). Thus, this study will focus on the development of new efficient and accurate FNN training methods. One improvement suggested to alleviate the problem of LR choice is the use of a line search with steepest descent method; namely, bracketing with golden section method. This provides an optimal LR as training progresses. Another improvement proposed in this study is the use of conjugate gradient (CG) methods to speed up the training process of the neural network. The computational feasibility of these methods is assessed on two matrix problems; namely, the LU-decomposition of both band and square ill-conditioned unsymmetric matrices and the inversion of square ill-conditioned unsymmetric matrices. In this study, two performance indexes have been considered; namely, learning speed and convergence accuracy. Extensive computer simulations have been carried out using the following training methods: steepest descent with line search (SDLS) method, conventional back propagation (BP) algorithm, and conjugate gradient (CG) methods; specifically, Fletcher Reeves conjugate gradient (CGFR) method and Polak Ribiere conjugate gradient (CGPR) method. The performance comparisons between these minimization methods have demonstrated that the CG training methods give better convergence accuracy and are by far the superior with respect to learning time; they offer speed-ups of anything between 3 and 4 over SDLS depending on the severity of the error goal chosen and the size of the problem. Furthermore, when using Powell's restart criteria with the CG methods, the problem of wrong convergence directions usually encountered in pure CG learning methods is alleviated. In general, CG methods with restarts have shown the best performance among all other methods in training the FNN for LU-decomposition and matrix inversion. Consequently, it is concluded that CG methods are good candidates for training FNN of matrix computations, in particular, Polak-Ribidre conjugate gradient method with Powell's restart criteria.

by Tat-Ming Tso. / Thesis (M.Phil.)--Chinese University of Hong Kong, 1997. / Includes bibliographical references (p. 43-45). / Chapter Chapter 1 --- INTRODUCTION --- p.1 / Chapter §1.1 --- Introduction --- p.1 / Chapter §1.2 --- Preconditioned Conjugate Gradient Method --- p.3 / Chapter §1.3 --- Outline of Thesis --- p.3 / Chapter Chapter 2 --- CIRCULANT AND NON-CIRCULANT PRECONDITIONERS --- p.5 / Chapter §2.1 --- Circulant Matrix --- p.5 / Chapter §2.2 --- Circulant Preconditioners --- p.6 / Chapter §2.3 --- Circulant Preconditioners from Kernel Function --- p.8 / Chapter §2.4 --- Non-circulant Band-Toeplitz Preconditioners --- p.9 / Chapter Chapter 3 --- B-SPLINES --- p.11 / Chapter §3.1 --- Introduction --- p.11 / Chapter §3.2 --- New Version of B-splines --- p.15 / Chapter Chapter 4 --- CIRCULANT PRECONDITIONERS CONSTRUCTED FROM B-SPLINES --- p.24 / Chapter Chapter 5 --- NUMERICAL RESULTS AND CONCLUDING REMARKS --- p.28 / Chapter Chapter 6 --- APPLICATIONS TO SIGNAL PROCESSING --- p.37 / Chapter §6.1 --- Introduction --- p.37 / Chapter §6.2 --- Preconditioned regularized least squares --- p.39 / Chapter §6.3 --- Numerical Example --- p.40 / REFERENCES --- p.43

A relatively new preconditioning technique called support graph preconditioning has many merits over the traditional incomplete factorization based methods. A major limitation of this technique is that it is applicable to symmetric diagonally dominant matrices only. This work presents a technique that can be used to transform the symmetric positive definite matrices arising from elliptic finite element problems into symmetric diagonally dominant M-matrices. The basic idea is to approximate the element gradient matrix by taking the gradients along chosen edges, whose unit vectors form a new coordinate system. For Lagrangian elements, the rows of the element gradient matrix in this new coordinate system are scaled edge vectors, thus a diagonally dominant symmetric semidefinite M-matrix can be generated to approximate the element stiffness matrix. Depending on the element type, one or more such coordinate systems are required to obtain a global nonsingular M-matrix. Since such approximation takes place at the element level, the degradation in the quality of the preconditioner is only a small constant factor independent of the size of the problem. This technique of element coordinate transformations applies to a variety of first order Lagrangian elements. Combination of this technique and other techniques enables us to construct an M-matrix preconditioner for a wide range of second order elliptic problems even with higher order elements. Another contribution of this work is the proposal of a new variant of Vaidya’s support graph preconditioning technique called modified domain partitioned support graph preconditioners. Numerical experiments are conducted for various second order elliptic finite element problems, along with performance comparison to the incomplete factorization based preconditioners. Results show that these support graph preconditioners are superior when solving ill-conditioned problems. In addition, the domain partition feature provides inherent parallelism, and initial experiments show a good potential of parallelization and scalability of these preconditioners.

The purpose of the inverse-transport problems that we address is to reconstruct the material distribution inside an unknown object undergoing a nondestructive evaluation. We assume that the object is subjected to incident beams of photons or particles and that the exiting radiation is measured with detectors around the periphery of the object. In the present work we focus on problems in which radiation can undergo significant scattering within the optically thick object. We develop a set of reconstruction strategies to infer the material distribution inside such objects. When we apply these strategies to a set of neutron-tomography test problems we find that the results are substantially superior to those obtained by previous methods. We first demonstrate that traditional analytic methods such as filtered back projection (FBP) methods do not work for very thick, highly scattering problems. Then we explore deterministic optimization processes, using the nonlinear conjugate gradient iterative updating scheme to minimize an objective functional that characterizes the misfits between forward predicted measurements and actual detector readings. We find that while these methods provide more information than the analytic methods such as FBP, they do not provide sufficiently accurate solutions of problems in which the radiation undergoes significant scattering. We proceed to present some advances in inverse transport methods. Our strategies offer several advantages over previous reconstruction methods. First, our optimization procedure involves the systematic use of both deterministic and stochastic methods, using the strengths of each to mitigate the weaknesses of the other. Another key feature is that we treat the material (a discrete quantity) as the unknown, as opposed to individual cross sections (continuous variables). This changes the mathematical nature of the problem and greatly reduces the dimension of the search space. In our hierarchical approach we begin by learning some characteristics of the object from relatively inexpensive calculations, and then use knowledge from such calculations to guide more sophisticated calculations. A key feature of our strategy is dimension-reduction schemes that we have designed to take advantage of known and postulated constraints. We illustrate our approach using some neutron-tomography model problems that are several mean-free paths thick and contain highly scattering materials. In these problems we impose reasonable constraints, similar to those that in practice would come from prior information or engineering judgment. Our results, which identify exactly the correct materials and provide very accurate estimates of their locations and masses, are substantially better than those of deterministic minimization methods and dramatically more efficient than those of typical stochastic methods.

Tomography refers to the cross-sectional imaging of an object from either transmission or reflection data collected by illuminating the object from many different directions. Classical tomography fails to reconstruct the optical properties of thick scattering objects because it does not adequately account for the scattering component of the neutron beam intensity exiting the sample. We proposed a new method of computed tomography which employs an inverse problem analysis of both the transmitted and scattered images generated from a beam passing through an optically thick object. This inverse problem makes use of a computationally efficient, two-dimensional forward problem based on neutron transport theory that effectively calculates the detector readings around the edges of an object. The forward problem solution uses a Step-Characteristic (SC) code with known uncollided source per cell, zero boundary flux condition and Sn discretization for the angular dependence. The calculation of the uncollided sources is performed by using an accurate discretization scheme given properties and position of the incoming beam and beam collimator. The detector predictions are obtained considering both the collided and uncollided components of the incoming radiation. The inverse problem is referred as an optimization problem. The function to be minimized, called an objective function, is calculated as the normalized-squared error between predicted and measured data. The predicted data are calculated by assuming a uniform distribution for the optical properties of the object. The objective function depends directly on the optical properties of the object; therefore, by minimizing it, the correct property distribution can be found. The minimization of this multidimensional function is performed with the Polack Ribiere conjugate-gradient technique that makes use of the gradient of the function with respect to the cross sections of the internal cells of the domain. The forward and inverse models have been successfully tested against numerical results obtained with MCNP (Monte Carlo Neutral Particles) showing excellent agreements. The reconstructions of several objects were successful. In the case of a single intrusion, TNTs (Tomography Neutron Transport using Scattering) was always able to detect the intrusion. In the case of the double body object, TNTs was able to reconstruct partially the optical distribution. The most important defect, in terms of gradient, was correctly located and reconstructed. Difficulties were discovered in the location and reconstruction of the second defect. Nevertheless, the results are exceptional considering they were obtained by lightening the object from only one side. The use of multiple beams around the object will significantly improve the capability of TNTs since it increases the number of constraints for the minimization problem.

The progress of high performance computing platforms is dramatic, and most of the simulations carried out on these platforms result in improvements on one level, yet expose shortcomings of current CFD packages. Therefore, hardware-aware design and optimizations are crucial towards exploiting modern computing resources. This thesis proposes optimizations aimed at accelerating numerical simulations, which are illus- trated in OpenFOAM solvers. A hybrid MPI and GPGPU parallel conjugate gradient linear solver has been designed and implemented to solve the sparse linear algebraic kernel that derives from two CFD solver: icoFoam, which is an incompressible flow solver, and laplacianFoam, which solves the Poisson equation, for e.g., thermal dif- fusion. A load-balancing step is applied using heterogeneous decomposition, which decomposes the computations taking into account the performance of each comput- ing device and seeking to minimize communication. In addition, we implemented the recently developed pipeline conjugate gradient as an algorithmic improvement, and parallelized it using MPI, GPGPU, and a hybrid technique. While many questions of ultimately attainable per node performance and multi-node scaling remain, the ex- perimental results show that the hybrid implementation of both solvers significantly outperforms state-of-the-art implementations of a widely used open source package.

A blackbody optical fiber thermometer consists of an optical fiber whose sensing tip is given a metallic coating. The sensing tip of the fiber forms an isothermal cavity, and the emission from this cavity is approximately equal to the emission from a blackbody. Standard two-color optical fiber thermometry involves measuring the spectral intensity at the end of the fiber at two wavelengths. The temperature at the sensing tip of the fiber can then be inferred using Planck's law and the ratio of the spectral intensities. If, however, the length of the optical fiber is exposed to elevated temperatures, erroneous temperature measurements will occur due to emission by the fiber. This thesis presents a method to account for emission by the fiber and accurately infer the temperature at the tip of the optical fiber. Additionally, an estimate of the temperature profile along the fiber may be obtained. A mathematical relation for radiation transfer down the optical fiber is developed. The radiation exiting the fiber and the temperature profile along the fiber are related to the detector signal by a signal measurement equation. Since the temperature profile cannot be solved for directly using the signal measurement equation, two inverse minimization techniques are developed to find the temperature profile. Simulated temperature profile reconstructions show the techniques produce valid and unique results. Tip temperatures are reconstructed to within 1.0%. Experimental results are also presented. Due to the limitations of the detection system and the optical fiber probe, the uncertainty in the signal measurement equation is high. Also, due to the limitations of the laboratory furnace and the optical detector, the measurement uncertainty is also high. This leads to reconstructions that are not always accurate. Even though the temperature profiles are not completely accurate, the tip-temperatures are reconstructed to within 1%—a significant improvement over the standard two-color technique under the same conditions. Improvements are recommended that will lead to decreased measurement and signal measurement equation uncertainty. This decreased uncertainty will lead to the development of a reliable and accurate temperature measurement device.

A scaled gradient descent method for competition of applications of conjugate gradient with priori estimations of the minimum value / This research proposes a novel method of improving the Gradient Descent method in an effort to be competitive with applications of the conjugate gradient method while reducing computation per iteration. Iterative methods for unconstrained optimization have found widespread application in digital signal processing applications for large inverse problems, such as the use of conjugate gradient for parallel image reconstruction in MR Imaging. In these problems, very good estimates of the minimum value at the objective function can be obtained by estimating the noise variance in the signal, or using additional measurements. The method proposed uses an estimation of the minimum to develop a scaling for Gradient Descent at each iteration, thus avoiding the necessity of a computationally extensive line search. A sufficient condition for convergence and proof are provided for the method, as well as an analysis of convergence rates for varying conditioned problems. The method is compared against the gradient descent and conjugate gradient methods. A method with a computationally inexpensive scaling factor is achieved that converges linearly for well-conditioned problems. The method is tested with tricky non-linear problems against gradient descent, but proves unsuccessful without augmenting with a line search. However with line search augmentation the method still outperforms gradient descent in iterations. The method is also benchmarked against conjugate gradient for linear problems, where it achieves similar convergence for well-conditioned problems even without augmenting with a line search. / Thesis / Master of Science (MSc) / This research proposes a novel method of improving the Gradient Descent method in an effort to be competitive with applications of the conjugate gradient method while reducing computation per iteration. Iterative methods for unconstrained optimization have found widespread application in digital signal processing applications for large inverse problems, such as the use of conjugate gradient for parallel image reconstruction in MR Imaging. In these problems, very good estimates of the minimum value at the objective function can be obtained by estimating the noise variance in the signal, or using additional measurements. The method proposed uses an estimation of the minimum to develop a scaling for Gradient Descent at each iteration, thus avoiding the necessity of a computationally extensive line search. A sufficient condition for convergence and proof are provided for the method, as well as an analysis of convergence rates for varying conditioned problems. The method is compared against the gradient descent and conjugate gradient methods. A method with a computationally inexpensive scaling factor is achieved that converges linearly for well-conditioned problems. The method is tested with tricky non-linear problems against gradient descent, but proves unsuccessful without augmenting with a line search. However with line search augmentation the method still outperforms gradient descent in iterations. The method is also benchmarked against conjugate gradient for linear problems, where it achieves similar convergence for well-conditioned problems even without augmenting with a line search.