The Kaczmarz method is an iterative algorithm that solves overdetermined systems of linear equalities. This paper studies a system of linear equalities and inequalities. We use the block version of the Kaczmarz method applied towards the equalities with the simple randomized Kaczmarz scheme for the inequalities. This primarily involves combining Needell and Tropp's work on the block Kaczmarz method with the application of a randomized Kaczmarz approach towards a system of equalities and inequalities performed by Leventhal and Lewis. We give an expected linear rate of convergence for this kind of system and find that using the block Kaczmarz scheme for the equalities can improve the rate compared to the simple Kaczmarz method.

This paper will demonstrate the principles and important facts of the randomized Kaczmarz algorithm as well as its extended version proposed by Zouzias and Ferris. Through the analysis made by Strohmer and Vershynin as well as Needell, it can be shown that the randomized Kaczmarz method is theoretically applicable in solving over-determined linear systems with or without noise. The extension of the randomized Kaczmarz algorithm further applies to the linear systems with non-unique solutions. In the experiment section of this paper, we compare the accuracies of the algorithms discussed in the paper in terms of making real-world macroeconomic analyses and predictions. The extended randomized Kaczmarz method outperforms both the randomized Kaczmarz method and the randomized Gauss-Seidel method on our data sets.

Numerous scientific applications have seen the rise of massive inverse problems, where there are too much data to implement an all-at-once strategy to compute a solution. Additionally, tools for regularizing ill-posed inverse problems are infeasible when the problem is too large. This thesis focuses on the development of row-action methods, which can be used to iteratively solve inverse problems when it is not possible to access the entire data-set or forward model simultaneously. We investigate these techniques for linear inverse problems and for separable, nonlinear inverse problems where the objective function is nonlinear in one set of parameters and linear in another set of parameters. For the linear problem, we perform a convergence analysis of these methods, which shows favorable asymptotic and initial convergence properties, as well as a trade-off between convergence rate and precision of iterates that is based on the step-size. These row-action methods can be interpreted as stochastic Newton and stochastic quasi-Newton approaches on a reformulation of the least squares problem, and they can be analyzed as limited memory variants of the recursive least squares algorithm. For ill-posed problems, we introduce sampled regularization parameter selection techniques, which include sampled variants of the discrepancy principle, the unbiased predictive risk estimator, and the generalized cross-validation. We demonstrate the effectiveness of these methods using examples from super-resolution imaging, tomography reconstruction, and image classification. / Doctor of Philosophy / Numerous scientific problems have seen the rise of massive data sets. An example of this is super-resolution, where many low-resolution images are used to construct a high-resolution image, or 3-D medical imaging where a 3-D image of an object of interest with hundreds of millions voxels is reconstructed from x-rays moving through that object. This work focuses on row-action methods that numerically solve these problems by repeatedly using smaller samples of the data to avoid the computational burden of using the entire data set at once. When data sets contain measurement errors, this can cause the solution to get contaminated with noise. While there are methods to handle this issue, when the data set becomes massive, these methods are no longer feasible. This dissertation develops techniques to avoid getting the solution contaminated with noise, even when the data set is immense. The methods developed in this work are applied to numerous scientific applications including super-resolution imaging, tomography, and image classification.

O método de Kaczmarz é um algoritmo iterativo que soluciona sistemas lineares do tipo Ax = b através de projeções sobre hiperplanos bastante usado em aplicações que envolvem a Tomografia Computadorizada. Recentemente voltou a ser destaque após a publicação de uma versão aleatória apresentada por Strohmer e Vershynin em 2009 a qual foi provada possuir taxa de convergência esperada exponencial. Posteriormente, Eldar e Needell em 2011 sugeriram uma versão modificada do algoritmo de Strohmer e Vershynin, na qual a cada iteração é selecionada a projeção ótima a partir de um conjunto aleatório, utilizando para isto o lema de Johnson-Lindenstrauss. Nenhum dos artigos mencionados apresenta uma técnica para a escolha do parâmetro de relaxação, entretanto, a seleção apropriada deste parâmetro pode ter uma influência substancial na velocidade do método. Neste trabalho apresentamos uma metodologia para a escolha do parâmetro de relaxação, bem como implementações paralelas do algoritmo de Kaczmarz utilizando as ideias de Eldar e Needell. Nossa metodologia para seleção do parâmetro utiliza uma nova generalização dos resultados de Strohmer e Vershynin que agora leva em consideração o parâmetro λ de relaxação e, a partir daí, obtemos uma estimativa da taxa de convergência como função de λ. Escolhemos então, para uso no algoritmo, aquele que otimiza esta estimativa. A paralelização dos métodos foi realizada através da plataforma CUDA e se mostrou muito promissora, pois conseguimos, através dela, um ganho significativo na velocidade de convergência / The Kaczmarz method is an iterative algorithm for finding the solution of a system of linear equations Ax = b by projecting onto the hyperplanes widely used in applications involving Computerized Tomography. It has been recently highlighted after the publication of a random version presented by Strohmer and Vershynin in 2009 that yields probably exponential convergence in expectation. Thereafter, Eldar and Needell in 2011 suggested a modified version of Strohmer and Vershynin algorithm, which at each iteration selects the optimal projection from a random set making use of the Johnson-Lindenstrauss lemma. None of the mentioned articles presents a technique for choosing the relaxation parameter, however, the proper selection of this parameter can achieve a substantial gain on the speed of the method. In this project we present a methodology for finding the relaxation parameter, as well as parallel implementations of Kacmarzs Algorithm using the ideas of Eldar and Needell. Our methodology for parameter selection uses a new generalization on Strohmer and Vershynins results which now regards the relaxation parameter λ. Thenceforward, we obtain an estimate of the convergence rate as a function of λ. Then we use this estimate in the algorithm the optimizer of this estimate. The parallelization of the methods has been implemented through the CUDA platform and appears to be very promising, since it delivers substantial gain in the convergence speed

O método de Kaczmarz é um algoritmo iterativo que soluciona sistemas lineares do tipo Ax = b através de projeções sobre hiperplanos bastante usado em aplicações que envolvem a Tomografia Computadorizada. Recentemente voltou a ser destaque após a publicação de uma versão aleatória apresentada por Strohmer e Vershynin em 2009 a qual foi provada possuir taxa de convergência esperada exponencial. Posteriormente, Eldar e Needell em 2011 sugeriram uma versão modificada do algoritmo de Strohmer e Vershynin, na qual a cada iteração é selecionada a projeção ótima a partir de um conjunto aleatório, utilizando para isto o lema de Johnson-Lindenstrauss. Nenhum dos artigos mencionados apresenta uma técnica para a escolha do parâmetro de relaxação, entretanto, a seleção apropriada deste parâmetro pode ter uma influência substancial na velocidade do método. Neste trabalho apresentamos uma metodologia para a escolha do parâmetro de relaxação, bem como implementações paralelas do algoritmo de Kaczmarz utilizando as ideias de Eldar e Needell. Nossa metodologia para seleção do parâmetro utiliza uma nova generalização dos resultados de Strohmer e Vershynin que agora leva em consideração o parâmetro λ de relaxação e, a partir daí, obtemos uma estimativa da taxa de convergência como função de λ. Escolhemos então, para uso no algoritmo, aquele que otimiza esta estimativa. A paralelização dos métodos foi realizada através da plataforma CUDA e se mostrou muito promissora, pois conseguimos, através dela, um ganho significativo na velocidade de convergência / The Kaczmarz method is an iterative algorithm for finding the solution of a system of linear equations Ax = b by projecting onto the hyperplanes widely used in applications involving Computerized Tomography. It has been recently highlighted after the publication of a random version presented by Strohmer and Vershynin in 2009 that yields probably exponential convergence in expectation. Thereafter, Eldar and Needell in 2011 suggested a modified version of Strohmer and Vershynin algorithm, which at each iteration selects the optimal projection from a random set making use of the Johnson-Lindenstrauss lemma. None of the mentioned articles presents a technique for choosing the relaxation parameter, however, the proper selection of this parameter can achieve a substantial gain on the speed of the method. In this project we present a methodology for finding the relaxation parameter, as well as parallel implementations of Kacmarzs Algorithm using the ideas of Eldar and Needell. Our methodology for parameter selection uses a new generalization on Strohmer and Vershynins results which now regards the relaxation parameter λ. Thenceforward, we obtain an estimate of the convergence rate as a function of λ. Then we use this estimate in the algorithm the optimizer of this estimate. The parallelization of the methods has been implemented through the CUDA platform and appears to be very promising, since it delivers substantial gain in the convergence speed

<p>A computerized tomography scan enables the visualization of an object interior without opening it up. This technique is used in many fields e.g. in medical imaging, geology, and industry. To obtain information about an object, exterior measurements by means of X-rays are performed. Then, to reconstruct an image of the object’s interior, image-reconstructions methods are applied. The problem of reconstructing images from measurements of X-ray radiation belongs to the class of inverse problems. A class of important methods for inverse problems is Algebraic Reconstruction Techniques (ART). The performance of these methods depends on the choice of a relaxation parameter.</p> <p>In this thesis, we compare numerically various ART methods, namely Kaczmarz, symmetric Kaczmarz, randomized Kaczmarz and simultaneous ART. We perform an extensive numerical investigation of the behaviour of these methods, and in particular, study how they perform with respect to this relaxation parameter. We propose a simple heuristic for finding a good relaxation parameter for each of these methods. Comparisons of the new proposed strategy with a previously proposed one shows that our strategy has a slightly better performance in terms of relative error, relative residual and image discrepancy of the reconstructed image. Both strategies showed relatively close numerical results, but interestingly enough, for different values of this parameter.</p> / Master of Computer Science (MCS)

No description available.

Submitted by Edisangela Bastos (edisangela@ufpa.br) on 2018-03-21T16:46:59Z No. of bitstreams: 2 license_rdf: 0 bytes, checksum: d41d8cd98f00b204e9800998ecf8427e (MD5) Tese_MetodosProjecaoConvergencia.pdf: 3858869 bytes, checksum: 6d1d5430b45704ff15ac44b85f148097 (MD5) / Made available in DSpace on 2018-03-21T16:46:59Z (GMT). No. of bitstreams: 2 license_rdf: 0 bytes, checksum: d41d8cd98f00b204e9800998ecf8427e (MD5) Tese_MetodosProjecaoConvergencia.pdf: 3858869 bytes, checksum: 6d1d5430b45704ff15ac44b85f148097 (MD5) Previous issue date: 1987-03-20 / CAPES - Coordenação de Aperfeiçoamento de Pessoal de Nível Superior / Neste trabalho, mostramos de forma mais geral que é possível obtermos métodos de projeção com a mesma propriedade dos métodos propostos por Bjorck e Elfving. Em particular, estabelecemos versões modificadas dos métodos de Kaczmarz, Cimmino [ 5] e Garza que apresentam a propriedade anteriormente citada. Isto é mostrado como segue. Os capítulos 1 e 2 são dedicados a resolução numérica de sistemas algébricos de equações lineares consistentes. No capítulo 1, apresentamos uma versão bloco acelerada do método de Kaczmarz e outra, também bloco acelerada, do método de Cimmino que serão úteis para o desenvolvimento dos capítulos posteriores. Nõ capítulo 2, estabelecemos de forma geral, um algoritmo do tipo projeção e demonstramos que a convergência é atingida em um número finito e conhecido de passos mostrado que as versões dos métodos de Kaczxnarz e Ciinmino, apresentadas no capítulo 1, convenientemente modificadas, são do tipo do algoritmo estabelecido. O capítulo 3 é dedicado a resolução numérica do problema de Quadrados Mínimos Lineares. De forma similar ao capítulo 2, são estabelecidas as versões aceleradas dos métodos de Garza e Cimmino para a resolução desse problema. No capítulo 4, mostramos uma aplicação desses tipos de algoritmos, através da resolução de um problema de Engenharia Hidráulica.

Ill-posed sets of linear equations typically arise when discretizing certain types of integral transforms. A well known example is image reconstruction, which can be modeled using the Radon transform. After expanding the solution into a finite series of basis functions a large, sparse and ill-conditioned linear system occurs. We consider the solution of such systems. In particular we study a new class of iteration methods named DROP (for Diagonal Relaxed Orthogonal Projections) constructed for solving both linear equations and linear inequalities. This class can also be viewed, when applied to linear equations, as a generalized Landweber iteration. The method is compared with other iteration methods using test data from a medical application and from electron microscopy. Our theoretical analysis include convergence proofs of the fully-simultaneous DROP algorithm for linear equations without consistency assumptions, and of block-iterative algorithms both for linear equations and linear inequalities, for the consistent case. When applying an iterative solver to an ill-posed set of linear equations the error usually initially decreases but after some iterations, depending on the amount of noise in the data, and the degree of ill-posedness, it starts to increase. This phenomenon is called semi-convergence. We study the semi-convergence performance of Landweber-type iteration, and propose new ways to specify the relaxation parameters. These are computed so as to control the propagated error. We also describe a class of stopping rules for Landweber-type iteration for solving linear inverse problems. The class includes the well known discrepancy principle, and the monotone error rule. We unify the error analysis of these two methods. The stopping rules depend critically on a certain parameter whose value needs to be specified. A training procedure is therefore introduced for securing robustness. The advantages of using trained rules are demonstrated on examples taken from image reconstruction from projections. Kaczmarz's method, also called ART (Algebraic Reconstruction Technique) is often used for solving the linear system which appears in image reconstruction. This is a fully sequential method. We examine and compare ART and its symmetric version. It is shown that the cycles of symmetric ART, unlike ART, converge to a weighted least squares solution if and only if the relaxation parameter lies between zero and two. Further we show that ART has faster asymptotic rate of convergence than symmetric ART. Also a stopping criterion is proposed and evaluated for symmetric ART. We further investigate a class of block-iterative methods used in image reconstruction. The cycles of the iterative sequences are characterized in terms of the original linear system. We define symmetric block-iteration and compare the behavior of symmetric and non-symmetric block-iteration. The results are illustrated using some well-known methods. A stopping criterion is offered and assessed for symmetric block-iteration.

Since the inverse problem in Diffusive Optical Tomography (DOT) is nonlinear and severely ill-posed, only low resolution reconstructions are feasible when noise is added to the data nowadays. The purpose of this thesis is to improve image reconstruction in DOT of the main optical properties of tissues with some novel mathematical methods. We have used the Landweber (L) method, the Landweber-Kaczmarz (LK) method and its improved Loping-Landweber-Kaczmarz (L-LK) method combined with sparsity or with total variation regularizations for single and simultaneous image reconstructions of the absorption and scattering coefficients. The sparsity method assumes the existence of a sparse solution which has a simple description and is superposed onto a known background. The sparsity method is solved using a smooth gradient and a soft thresholding operator. Moreover, we have proposed an improved sparsity method. For the total variation reconstruction imaging, we have used the split Bregman method and the lagged diffusivity method. For the total variation method, we also have implemented a memory-efficient method to minimise the storage of large Hessian matrices. In addition, an individual and simultaneous contrast value reconstructions are presented using the level set (LS) method. Besides, the shape derivative of DOT based on the RTE is derived using shape sensitivity analysis, and some reconstructions for the absorption coefficient are presented using this shape derivative via the LS method.\\Whereas most of the approaches for solving the nonlinear problem of DOT make use of the diffusion approximation (DA) to the radiative transfer equation (RTE) to model the propagation of the light in tissue, the accuracy of the DA is not satisfactory in situations where the medium is not scattering dominant, in particular close to the light sources and to the boundary, as well as inside low-scattering or non-scattering regions. Therefore, we have solved the inverse problem in DOT by the more accurate time-dependant RTE in two dimensions.