In visually guided control of a robot, a large residual problem occurs when the robot configuration is not in the neighborhood of the target acquisition configuration. Most existing uncalibrated visual servoing algorithms use quasi-Gauss-Newton methods which are effective for small residual problems. The solution used in this study switches between a full quasi-Newton method for large residual case and the quasi-Gauss-Newton methods for the small case. Visual servoing to handle large residual problems for tracking a moving target has not previously appeared in the literature. For large residual problems various Hessian approximations are introduced including an approximation of the entire Hessian matrix, the dynamic BFGS (DBFGS) algorithm, and two distinct approximations of the residual term, the modified BFGS (MBFGS) algorithm and the dynamic full Newton method with BFGS (DFN-BFGS) algorithm. Due to the fact that the quasi-Gauss-Newton method has the advantage of fast convergence, the quasi-Gauss-Newton step is used as the iteration is sufficiently near the desired solution. A switching algorithm combines a full quasi-Newton method and a quasi-Gauss-Newton method. Switching occurs if the image error norm is less than the switching criterion, which is heuristically selected. An adaptive forgetting factor called the dynamic adaptive forgetting factor (DAFF) is presented. The DAFF method is a heuristic scheme to determine the forgetting factor value based on the image error norm. Compared to other existing adaptive forgetting factor schemes, the DAFF method yields the best performance for both convergence time and the RMS error. Simulation results verify validity of the proposed switching algorithms with the DAFF method for large residual problems. The switching MBFGS algorithm with the DAFF method significantly improves tracking performance in the presence of noise. This work is the first successfully developed model independent, vision-guided control for large residual with capability to stably track a moving target with a robot.

In many fields of science, engineering, and economics large amounts of data are stored and there is a need to analyze these data in order to extract information for various purposes. Data mining is a general concept involving different tools for performing this kind of analysis. The development of mathematical models and efficient algorithms is of key importance. In this thesis we discuss algorithms for the reduced rank regression problem and algorithms for the computation of the best multilinear rank approximation of tensors. The first two papers deal with the reduced rank regression problem, which is encountered in the field of state-space subspace system identification. More specifically the problem is \[ \min_{\rank(X) = k} \det (B - X A)(B - X A)\tp, \] where $A$ and $B$ are given matrices and we want to find $X$ under a certain rank condition that minimizes the determinant. This problem is not properly stated since it involves implicit assumptions on $A$ and $B$ so that $(B - X A)(B - X A)\tp$ is never singular. This deficiency of the determinant criterion is fixed by generalizing the minimization criterion to rank reduction and volume minimization of the objective matrix. The volume of a matrix is defined as the product of its nonzero singular values. We give an algorithm that solves the generalized problem and identify properties of the input and output signals causing a singular objective matrix. Classification problems occur in many applications. The task is to determine the label or class of an unknown object. The third paper concerns with classification of handwritten digits in the context of tensors or multidimensional data arrays. Tensor and multilinear algebra is an area that attracts more and more attention because of the multidimensional structure of the collected data in various applications. Two classification algorithms are given based on the higher order singular value decomposition (HOSVD). The main algorithm makes a data reduction using HOSVD of 98--99 \% prior the construction of the class models. The models are computed as a set of orthonormal bases spanning the dominant subspaces for the different classes. An unknown digit is expressed as a linear combination of the basis vectors. The resulting algorithm achieves 5\% in classification error with fairly low amount of computations. The remaining two papers discuss computational methods for the best multilinear rank approximation problem \[ \min_{\cB} \| \cA - \cB\| \] where $\cA$ is a given tensor and we seek the best low multilinear rank approximation tensor $\cB$. This is a generalization of the best low rank matrix approximation problem. It is well known that for matrices the solution is given by truncating the singular values in the singular value decomposition (SVD) of the matrix. But for tensors in general the truncated HOSVD does not give an optimal approximation. For example, a third order tensor $\cB \in \RR^{I \x J \x K}$ with rank$(\cB) = (r_1,r_2,r_3)$ can be written as the product \[ \cB = \tml{X,Y,Z}{\cC}, \qquad b_{ijk}=\sum_{\lambda,\mu,\nu} x_{i\lambda} y_{j\mu} z_{k\nu} c_{\lambda\mu\nu}, \] where $\cC \in \RR^{r_1 \x r_2 \x r_3}$ and $X \in \RR^{I \times r_1}$, $Y \in \RR^{J \times r_2}$, and $Z \in \RR^{K \times r_3}$ are matrices of full column rank. Since it is no restriction to assume that $X$, $Y$, and $Z$ have orthonormal columns and due to these constraints, the approximation problem can be considered as a nonlinear optimization problem defined on a product of Grassmann manifolds. We introduce novel techniques for multilinear algebraic manipulations enabling means for theoretical analysis and algorithmic implementation. These techniques are used to solve the approximation problem using Newton and Quasi-Newton methods specifically adapted to operate on products of Grassmann manifolds. The presented algorithms are suited for small, large and sparse problems and, when applied on difficult problems, they clearly outperform alternating least squares methods, which are standard in the field.

Στην παρούσα διατριβή μελετάμε το πρόβλημα της βελτιστοποίησης μη γραμμικών συναρτήσεων πολλών μεταβλητών, όπου η αντικειμενική συνάρτηση είναι συνεχώς διαφορίσιμη σε ένα ανοιχτό υποσύνολο του Rn. Αναπτύσσουμε μαθηματικές μεθόδους βελτιστοποίησης αποσκοπώντας στην επίλυση προβλημάτων μεγάλης κλίμακας, δηλαδή προβλημάτων των οποίων οι μεταβλητές είναι πολλές χιλιάδες, ακόμα και εκατομμύρια. Η βασική ιδέα των μεθόδων που αναπτύσσουμε έγκειται στη θεωρητική μελέτη των χαρακτηριστικών μεγεθών των Quasi-Newton ενημερώσεων ελάχιστης και μικρής μνήμης. Διατυπώνουμε θεωρήματα αναφορικά με το χαρακτηριστικό πολυώνυμο, τον αριθμό των διακριτών ιδιοτιμών και των αντίστοιχων ιδιοδιανυσμάτων. Εξάγουμε κλειστούς τύπους για τον υπολογισμό των ανωτέρω ποσοτήτων, αποφεύγοντας τόσο την αποθήκευση όσο και την παραγοντοποίηση πινάκων. Τα νέα θεωρητικά απoτελέσματα εφαρμόζονται αφενός μεν στην επίλυση μεγάλης κλίμακας υποπροβλημάτων περιοχής εμπιστοσύνης, χρησιμοποιώντας τη μέθοδο της σχεδόν ακριβούς λύσης, αφετέρου δε, στην καμπυλόγραμμη αναζήτηση, η οποία χρησιμοποιεί ένα ζεύγος κατευθύνσεων μείωσης, την Quasi-Newton κατεύθυνση και την κατεύθυνση αρνητικής καμπυλότητας. Η νέα μέθοδος μειώνει δραστικά τη χωρική πολυπλοκότητα των γνωστών αλγορίθμων του μη γραμμικού προγραμματισμού, διατηρώντας παράλληλα τις καλές ιδιότητες σύγκλισής τους. Ως αποτέλεσμα, οι προκύπτοντες νέοι αλγόριθμοι έχουν χωρική πολυπλοκότητα Θ(n). Τα αριθμητικά αποτελέσματα δείχνουν ότι οι νέοι αλγόριθμοι είναι αποδοτικοί, γρήγοροι και πολύ αποτελεσματικοί όταν χρησιμοποιούνται στην επίλυση προβλημάτων με πολλές μεταβλητές. / In this thesis we study the problem of minimizing nonlinear functions of several variables, where the objective function is continuously differentiable on an open subset of Rn. We develop mathematical optimization methods for solving large scale problems, i.e., problems whose variables are many thousands, even millions. The proposed method is based on the theoretical study of the properties of minimal and low memory Quasi-Newton updates. We establish theorems concerning the characteristic polynomial, the number of distinct eigenvalues and corresponding eigenvectors. We derive closed formulas for calculating these quantities, avoiding both the storage and factorization of matrices. The new theoretical results are applied in the large scale trust region subproblem for calculating nearly exact solutions as well as in a curvilinear search that uses a Quasi-Newton and a negative curvature direction. The new method is drastically reducing the spatial complexity of known algorithms of nonlinear programming. As a result, the new algorithms have spatial complexity Θ(n), while they are maintaining good convergence properties. The numerical results show that the proposed algorithms are efficient, fast and very effective when used in solving large scale problems.

<p>The focus of this thesis is on numerical algorithms for efficient solution of QTL analysis problem in genetics.</p><p>Firstly, we consider QTL mapping problems where a standard least-squares model is used for computing the model fit. We develop optimization methods for the local problems in a hybrid global-local optimization scheme for determining the optimal set of QTL locations. Here, the local problems have constant bound constraints and may be non-convex and/or flat in one or more directions. We propose an enhanced quasi-Newton method and also implement several schemes for constrained optimization. The algorithms are adopted to the QTL optimization problems. We show that it is possible to use the new schemes to solve problems with up to 6 QTLs efficiently and accurately, and that the work is reduced with up to two orders magnitude compared to using only global optimization.</p><p>Secondly, we study numerical methods for QTL mapping where variance component estimation and a REML model is used. This results in a non-linear optimization problem for computing the model fit in each set of QTL locations. Here, we compare different optimization schemes and adopt them for the specifics of the problem. The results show that our version of the active set method is efficient and robust, which is not the case for methods used earlier. We also study the matrix operations performed inside the optimization loop, and develop more efficient algorithms for the REML computations. We develop a scheme for reducing the number of objective function evaluations, and we accelerate the computations of the derivatives of the log-likelihood by introducing an efficient scheme for computing the inverse of the variance-covariance matrix and other components of the derivatives of the log-likelihood.</p>

The focus of this thesis is on numerical algorithms for efficient solution of QTL analysis problem in genetics. Firstly, we consider QTL mapping problems where a standard least-squares model is used for computing the model fit. We develop optimization methods for the local problems in a hybrid global-local optimization scheme for determining the optimal set of QTL locations. Here, the local problems have constant bound constraints and may be non-convex and/or flat in one or more directions. We propose an enhanced quasi-Newton method and also implement several schemes for constrained optimization. The algorithms are adopted to the QTL optimization problems. We show that it is possible to use the new schemes to solve problems with up to 6 QTLs efficiently and accurately, and that the work is reduced with up to two orders magnitude compared to using only global optimization. Secondly, we study numerical methods for QTL mapping where variance component estimation and a REML model is used. This results in a non-linear optimization problem for computing the model fit in each set of QTL locations. Here, we compare different optimization schemes and adopt them for the specifics of the problem. The results show that our version of the active set method is efficient and robust, which is not the case for methods used earlier. We also study the matrix operations performed inside the optimization loop, and develop more efficient algorithms for the REML computations. We develop a scheme for reducing the number of objective function evaluations, and we accelerate the computations of the derivatives of the log-likelihood by introducing an efficient scheme for computing the inverse of the variance-covariance matrix and other components of the derivatives of the log-likelihood.

This thesis proposes a new class of bivariate autoregressive conditional median duration models for matched high-frequency data and develops some inferential methods for an existing univariate model as well as the bivariate models introduced here to facilitate model fitting and forecasting. During the last two decades, the autoregressive conditional mean duration (ACD) model has been playing a dominant role in analyzing irregularly spaced high-frequency financial data. Univariate ACD models have been extensively discussed in the literature. However, some major challenges remain. The existing ACD models do not provide a good distributional fit to financial durations, which are right-skewed and often exhibit unimodal hazard rates. Birnbaum-Saunders (BS) distribution is capable of modeling a wide variety of positively skewed data. Median is not only a robust measure of central tendency, but also a natural scale parameter of the BS distribution. A class of conditional median duration models, the BS-ACD and the scale-mixture BS ACD models based on the BS, BS power-exponential and Student-t BS (BSt) distributions, have been suggested in the literature to improve the quality of the model fit. The BSt-ACD model is more flexible than the BS-ACD model in terms of kurtosis and skewness. In Chapter 2, we develop the maximum likelihood estimation method for the BSt-ACD model. The estimation is performed by utilizing a hybrid of optimization algorithms. The performance of the estimates is then examined through an extensive Monte Carlo simulation study. We also carry out model discrimination using both likelihood-based method and information-based criterion. Applications to real trade durations and comparison with existing alternatives are then made. The bivariate version of the ACD model has not received attention due to non-synchronicity. Although some bivariate generalizations of the ACD model have been introduced, they do not possess enough flexibility in modeling durations since they are conditional mean-based and do not account for non-monotonic hazard rates. Recently, the bivariate BS (BVBS) distribution has been developed with many desirable properties and characteristics. It allows for unimodal shapes of marginal hazard functions. In Chapter 3, upon using this bivariate BS distribution, we propose the BVBS-ACD model as a natural bivariate extension of the BS-ACD model. It enables us to jointly analyze matched duration series, and also capture the dependence between the two series. The maximum likelihood estimation of the model parameters and associated inferential methods have been developed. A Monte Carlo simulation study is then carried out to examine the performance of the proposed inferential methods. The goodness-of-fit and predictive performance of the model are also discussed. A real bivariate duration data analysis is provided to illustrate the developed methodology. The bivariate Student-t BS (BVBSt) distribution has been introduced in the literature as a robust extension of the BVBS distribution. It provides greater flexibility in terms of the kurtosis and skewness through the inclusion of an additional shape parameter. In Chapter 4, we propose the BVBSt-ACD model as a natural extension of the BSt-ACD model to the bivariate case. We then discuss the maximum likelihood estimation of the model parameters. A simulation study is carried out to investigate the performance of these estimators. Model discrimination is then done by using information-based criterion. Methods for evaluating the goodness-of-fit and predictive ability of the model are also discussed. A simulated data example is used to illustrate the proposed model as compared to the BVBS-ACD model. Finally, in Chapter 5, some concluding comments are made and also some problems for future research are mentioned. / Thesis / Master of Science (MSc)

Humanitarian clearance of minefields is most often carried out by hand, conventionally using a a metal detector and a probe. Detection is a very slow process, as every piece of detected metal must treated as if it were a landmine and carefully probed and excavated, while many of them are not. The process can be safely sped up by use of Ground-Penetrating Radar (GPR) to image the subsurface, to verify metal detection results and safely ignore any objects which could not possibly be a landmine. In this thesis, we explore the possibility of using Full Wave Inversion (FWI) to improve GPR imaging for landmine detection. Posing the imaging task as FWI means solving the large-scale, non-linear and ill-posed optimisation problem of determining the physical parameters of the subsurface (such as electrical permittivity) which would best reproduce the data. This thesis begins by giving an overview of all the mathematical and implementational aspects of FWI, so as to provide an informative text for both mathematicians (perhaps already familiar with other inverse problems) wanting to contribute to the mine detection problem, as well as a wider engineering audience (perhaps already working on GPR or mine detection) interested in the mathematical study of inverse problems and FWI.We present the first numerical 3D FWI results for GPR, and consider only surface measurements from small-scale arrays as these are suitable for our application. The FWI problem requires an accurate forward model to simulate GPR data, for which we use a hybrid finite-element boundary-integral solver utilising first order curl-conforming N\'d\'{e}lec (edge) elements. We present a novel `line search' type algorithm which prioritises inversion of some target parameters in a region of interest (ROI), with the update outside of the area defined implicitly as a function of the target parameters. This is particularly applicable to the mine detection problem, in which we wish to know more about some detected metallic objects, but are not interested in the surrounding medium. We may need to resolve the surrounding area though, in order to account for the target being obscured and multiple scattering in a highly cluttered subsurface. We focus particularly on spatial sensitivity of the inverse problem, using both a singular value decomposition to analyse the Jacobian matrix, as well as an asymptotic expansion involving polarization tensors describing the perturbation of electric field due to small objects. The latter allows us to extend the current theory of sensitivity in for acoustic FWI, based on the Born approximation, to better understand how polarization plays a role in the 3D electromagnetic inverse problem. Based on this asymptotic approximation, we derive a novel approximation to the diagonals of the Hessian matrix which can be used to pre-condition the GPR FWI problem.

Over the last two decades, great progress has been made in the understanding of multiferroic materials, ones where multiple long-range orders simultaneously exist. However, much of the research has focused on bulk systems. If these materials are to be incorporated into devices, they would not be in bulk form, but would be miniaturized, such as in nanoparticle form. Accordingly, a better understanding of multiferroic nanoparticles is necessary. This manuscript examines the multiferroic phase diagram of multiferroic nanoparticles related to system size and surface-induced magnetic anisotropy. There is a particular focus on bismuth ferrite, the room-temperature antiferromagnetic-ferroelectric multiferroic. Theoretical results will be presented which show that at certain sizes, a bistability develops in the cycloidal wavevector. This implies bistability in the ferroelectric and magnetic moments of the nanoparticles. This novel magnetoelectric bistability may be of use in the creation of an electrically-written, magnetically-read memory element. / Graduate