This thesis deals with solutions of nonlinear regression problems using R programming language. The introductory theoretical part is devoted to familiarization with the principles of solving nonlinear regression models and of their applications in the program R. In both, theoretical and practical part, the most famous and used differentiator algorithms are presented, particularly the Gauss-Newton's and of the steepest descent method, for estimating the parameters of nonlinear regression. Further, in the practical part, there are some demo solutions of particular tasks using nonlinear regression methods. Overall, a large number of graphs processed by the author is used in this thesis for better comprehension.

<p dir="ltr">This thesis presents advances in numerical methods for partial differential equations (PDEs) and optimization problems, with a focus on improving efficiency, stability, and accuracy across various applications. We begin by addressing 3D Poisson-type equations, developing a GPU-accelerated spectral-element method that utilizes the tensor product structure to achieve extremely fast performance. This approach enables solving problems with over one billion degrees of freedom in less than one second on modern GPUs, with applications to Schrödinger and Cahn<i>–</i>Hilliard equations demonstrated. Next, we focus on parabolic PDEs, specifically the Cahn<i>–</i>Hilliard equation with dynamical boundary conditions. We propose an efficient energy-stable numerical scheme using a unified framework to handle both Allen<i>–</i>Cahn and Cahn<i>–</i>Hilliard type boundary conditions. The scheme employs a scalar auxiliary variable (SAV) approach to achieve linear, second-order, and unconditionally energy stable properties. Shifting to a machine learning perspective for PDEs, we introduce an unsupervised learning-based numerical method for solving elliptic PDEs. This approach uses deep neural networks to approximate PDE solutions and employs least-squares functionals as loss functions, with a focus on first-order system least-squares formulations. In the realm of optimization, we present an efficient and robust SAV based algorithm for discrete gradient systems. This method modifies the standard SAV approach and incorporates relaxation and adaptive strategies to achieve fast convergence for minimization problems while maintaining unconditional energy stability. Finally, we address optimization in the context of machine learning by developing a structure-guided Gauss<i>–</i>Newton method for shallow ReLU neural network optimization. This approach exploits both the least-squares and neural network structures to create an efficient iterative solver, demonstrating superior performance on challenging function approximation problems. Throughout the thesis, we provide theoretical analysis, efficient numerical implementations, and extensive computational experiments to validate the proposed methods. </p>

The work done in this thesis is to examine various methods for curve fitting. Linear least squares and non-linear least squares will be described and compared, and the Newton method, Gauss--Newton method and Levenberg--Marquardt method will be applied to example problems. / Syftet med denna uppsats är att beskriva och använda olika metoder för kurvanpassning, det vill säga att passa matematiska funktioner till data. De metoder som undersöks är Newtons metod, Gauss--Newton metoden och Levenberg--Marquardt metoden. Även skillnaden mellan linjär minsta kvadrat anpassning och olinjär minsta kvadrat anpassning. Till sist tillämpas Newton, Gauss Newton och Levenberg--Marquardt metoderna på olika exempel.

Stable and computationally efficient reconstruction methodologies are developed to solve two important medical imaging problems which use near-infrared (NIR) light as the source of interrogation, namely, diffuse optical tomography (DOT) and one of its variations, ultrasound-modulated optical tomography (UMOT). Since in both these imaging modalities the system matrices are ill-conditioned owing to insufficient and noisy data, the emphasis in this work is to develop robust stochastic filtering algorithms which can handle measurement noise and also account for inaccuracies in forward models through an appropriate assignment of a process noise. However, we start with demonstration of speeding of a Gauss-Newton (GN) algorithm for DOT so that a video-rate reconstruction from data recorded on a CCD camera is rendered feasible. Towards this, a computationally efficient linear iterative scheme is proposed to invert the normal equation of a Gauss-Newton scheme in the context of recovery of absorption coefficient distribution from DOT data, which involved the singular value decomposition (SVD) of the Jacobian matrix appearing in the update equation. This has sufficiently speeded up the inversion that a video rate recovery of time evolving absorption coefficient distribution is demonstrated from experimental data. The SVD-based algorithm has made the number of operations in image reconstruction to be rather than. 2()ONN3()ONN The rest of the algorithms are based on different forms of stochastic filtering wherein we arrive at a mean-square estimate of the parameters through computing their joint probability distributions conditioned on the measurement up to the current instant. Under this, the first algorithm developed uses a Bootstrap particle filter which also uses a quasi-Newton direction within. Since keeping track of the Newton direction necessitates repetitive computation of the Jacobian, for all particle locations and for all time steps, to make the recovery computationally feasible, we devised a faster update of the Jacobian. It is demonstrated, through analytical reasoning and numerical simulations, that the proposed scheme, not only accelerates convergence but also yields substantially reduced sample variance in the estimates vis-à-vis the conventional BS filter. Both accelerated convergence and reduced sample variance in the estimates are demonstrated in DOT optical parameter recovery using simulated and experimental data. In the next demonstration a derivative free variant of the pseudo-dynamic ensemble Kalman filter (PD-EnKF) is developed for DOT wherein the size of the unknown parameter is reduced by representing of the inhomogeneities through simple geometrical shapes. Also the optical parameter fields within the inhomogeneities are approximated via an expansion based on the circular harmonics (CH) (Fourier basis functions). The EnKF is then used to recover the coefficients in the expansion with both simulated and experimentally obtained photon fluence data on phantoms with inhomogeneous inclusions. The process and measurement equations in the Pseudo-Dynamic EnKF (PD-EnKF) presently yield a parsimonious representation of the filter variables, which consist of only the Fourier coefficients and the constant scalar parameter value within the inclusion. Using fictitious, low-intensity Wiener noise processes in suitably constructed ‘measurement’ equations, the filter variables are treated as pseudo-stochastic processes so that their recovery within a stochastic filtering framework is made possible. In our numerical simulations we have considered both elliptical inclusions (two inhomogeneities) and those with more complex shapes ( such as an annular ring and a dumbbell) in 2-D objects which are cross-sections of a cylinder with background absorption and (reduced) scattering coefficient chosen as = 0.01 mm-1 and = 1.0 mm-1respectively. We also assume=0.02 mm-1 within the inhomogeneity (for the single inhomogeneity case) and=0.02 and 0.03 mm-1 (for the two inhomogeneities case). The reconstruction results by the PD-EnKF are shown to be consistently superior to those through a deterministic and explicitly regularized Gauss-Newton algorithm. We have also estimated the unknown from experimentally gathered fluence data and verified the reconstruction by matching the experimental data with the computed one. The superiority of a modified version of the PD-EnKF, which uses an ensemble square root filter, is also demonstrated in the context of UMOT by recovering the distribution of mean-squared amplitude of vibration, related to the Young’s modulus, in the ultrasound focal volume. Since the ability of a coherent light probe to pick-up the overall optical path-length change is limited to modulo an optical wavelength, the individual displacements suffered owing to the US forcing should be very small, say within a few angstroms. The sensitivity of modulation depth to changes in these small displacements could be very small, especially when the ROI is far removed from the source and detector. The contrast recovery of the unknown distribution in such cases could be seriously impaired whilst using a quasi-Newton scheme (e.g. the GN scheme) which crucially makes use of the derivative information. The derivative-free gain-based Monte Carlo filter not only remedies this deficiency, but also provides a regularization insensitive and computationally competitive alternative to the GN scheme. The inherent ability of a stochastic filter in accommodating the model error owing to a diffusion approximation of the correlation transport may be cited as an added advantage in the context of the UMOT inverse problem. Finally to speed up forward solve of the partial differential equation (PDE) modeling photon transport in the context of UMOT for which the PDE has time as a parameter, a spectral decomposition of the PDE operator is demonstrated. This allows the computation of the time dependent forward solution in terms of the eigen functions of the PDE operator which has speeded up the forward solution, which in turn has rendered the UMOT parameter recovery computationally efficient.