Title: Krylov Subspace Methods - Analysis and Application Author: Tomáš Gergelits Department: Department of Numerical Mathematics Supervisor: prof. Ing. Zdeněk Strakoš, DrSc., Department of Numerical Mathematics Abstract: Convergence behavior of Krylov subspace methods is often studied for linear algebraic systems with symmetric positive definite matrices in terms of the condition number of the system matrix. As recalled in the first part of this thesis, their actual convergence behavior (that can be in practice also substantially affected by rounding errors) is however determined by the whole spectrum of the system matrix, and by the projections of the initial residual to the associated invariant subspaces. The core part of this thesis investigates the spectra of infinite dimensional operators −∇ · (k(x)∇) and −∇ · (K(x)∇), where k(x) is a scalar coefficient function and K(x) is a symmetric tensor function, preconditioned by the Laplace operator. Subsequently, the focus is on the eigenvalues of the matrices that arise from the discretization using conforming finite elements. Assuming continuity of K(x), it is proved that the spectrum of the preconditi- oned infinite dimensional operator is equal to the convex hull of the ranges of the diagonal function entries of Λ(x) from the spectral decomposition K(x) =...

Closed-loop system identification and fault detection and isolation are the two fundamental building blocks of process monitoring. Efficient and accurate process monitoring increases plant availability and utilisation. This dissertation investigates a subspace system identification and fault detection methodology for the Benfield process, used by Sasol, Synfuels in Secunda, South Africa, to remove CO2 from CO2-rich tail gas. Subspace identification methods originated between system theory, geometry and numerical linear algebra which makes it a computationally efficient tool to estimate system parameters. Subspace identification methods are classified as Black-Box identification techniques, where it does not rely on a-priori process information and estimates the process model structure and order automatically. Typical subspace identification algorithms use non-parsimonious model formulation, with extra terms in the model that appear to be non-causal (stochastic noise components). These extra terms are included to conveniently perform subspace projection, but are the cause for inflated variance in the estimates, and partially responsible for the loss of closed-loop identifiably. The subspace identification methodology proposed in this dissertation incorporates two successive LQ decompositions to remove stochastic components and obtain state-space models of the plant respectively. The stability of the identified plant is further guaranteed by using the shift invariant property of the extended observability matrix by appending the shifted extended observability matrix by a block of zeros. It is shown that the spectral radius of the identified system matrices all lies within a unit boundary, when the system matrices are derived from the newly appended extended observability matrix. The proposed subspace identification methodology is validated and verified by re-identifying the Benfield process operating in closed-loop, with an RMPCT controller, using measured closed-loop process data. Models that have been identified from data measured from the Benfield process operating in closed-loop with an RMPCT controller produced validation data fits of 65% and higher. From residual analysis results, it was concluded that the proposed subspace identification method produce models that are accurate in predicting future outputs and represent a wide variety of process inputs. A parametric fault detection methodology is proposed that monitors the estimated system parameters as identified from the subspace identification methodology. The fault detection methodology is based on the monitoring of parameter discrepancies, where sporadic parameter deviations will be detected as faults. Extended Kalman filter theory is implemented to estimate system parameters, instead of system states, as new process data becomes readily available. The extended Kalman filter needs accurate initial parameter estimates and is thus periodically updated by the subspace identification methodology, as a new set of more accurate parameters have been identified. The proposed fault detection methodology is validated and verified by monitoring process behaviour of the Benfield process. Faults that were monitored for, and detected include foaming, flooding and sensor faults. Initial process parameters as identified from the subspace method can be tracked efficiently by using an extended Kalman filter. This enables the fault detection methodology to identify process parameter deviations, with a process parameter deviation sensitivity of 2% or higher. This means that a 2% parameter deviation will be detected which greatly enhances the fault detection efficiency and sensitivity. / Dissertation (MEng)--University of Pretoria, 2008. / Electrical, Electronic and Computer Engineering / unrestricted

Tato práce pojednává o modelování v podprostoru parametrů směsí gaussovských rozložení pro rozpoznávání mluvčího. Práce se skládá ze tří částí. První část je věnována skórovacím metodám při použití sdružené faktorové analýzy k modelování mluvčího. Studované metody se liší převážně v tom, jak se vypořádávají s variabilitou kanálu testovacích nahrávek. Metody jsou prezentovány v souvislosti s obecnou formou funkce pravděpodobnosti pro sdruženou faktorovou analýzu a porovnány jak z hlediska přesnosti, tak i z hlediska rychlosti.  Je zde prokázáno, že použití lineární aproximace pravděpodobnostní funkce dává výsledky srovnatelné se standardním vyhodnocením pravděpodobnosti při dramatickém zjednodušení matematického zápisu a tím i zvýšení rychlosti vyhodnocování. Druhá část pojednává o extrakci tzv. i-vektorů, tedy nízkodimenzionálních reprezentací nahrávek. Práce prezentuje dva přístupy ke zjednodušení extrakce. Motivací pro tuto část bylo jednak urychlení extrakce i-vektorů, jednak nasazení této úspěšné techniky na jednoduchá zařízení typu mobilní telefon, a také matematické zjednodušení umožněňující využití numerických optimalizačních metod pro diskriminativní trénování.  Výsledky ukazují, že na dlouhých nahrávkách je zrychlení vykoupeno poklesem úspěšnosti rozpoznávání, avšak na krátkých nahrávkách, kde je úspěšnost rozpoznávání nízká, se rozdíly úspěšnosti stírají. Třetí část se zabývá diskriminativním trénováním v oblasti rozpoznávání mluvčího. Jsou zde shrnuty poznatky z předchozích prací zabývajících se touto problematikou. Kapitola navazuje na poznatky z předchozích dvou částí a pojednává o diskriminativním trénování parametrů extraktoru i-vektorů.  Výsledky ukazují, že při klasickém trénování extraktoru a následném diskriminatviním přetrénování tyto metody zvyšují úspěšnost.

Děti mají již od útlého věku vrozenou schopnost vyvozovat jazykové znalosti z mluvené řeči - dlouho předtím, než se naučí číst a psát. Moderní systémy pro rozpoznávání řeči oproti tomu potřebují k dosažení nízké chybovosti značná množství přepsaných řečových dat. Teprve nedávno založená vědecká oblast "učení řeči bez supervize" se věnuje přenosu popsaných lidských schopností do strojového učení. V rámci této oblasti se naše práce zaměřuje na problém určení sady akustických jednotek z jazyka, kde jsou k disposici pouze nepřepsané zvukové nahrávky. Pro řešení tohoto problému zkoumáme zejména potenciál bayesovské inference. V práci nejprve pro úlohu určování akustických jednotek revidujeme využití state-of-the-art neparametrického bayesovského modelu, pro který jsme odvodili rychlý a efektivní algoritmus variační bayesovské inference. Náš přístup se opírá o konstrukci Dirichletova procesu pomocí "lámání hůlky" (stick breaking) umožňující vyjádření modelu jako fonémové smyčky založené na skrytém Markovově modelu. S tímto modelem a vhodnou středopolní (mean-field) aproximací variační posteriorní pravděpodobnosti je inference realizována pomocí efektivního iteračního algoritmu, podobného známému schématu Expectation-Maximization (EM). Experimenty ukazují, že tento přístup zajišťuje lepší shlukování než původní model, přičemž je řádově rychlejší. Druhým přínosem práce je řešení problému definice smysluplného apriorního rozdělení na potenciální akustické jednotky. Za tímto účelem představujeme zobecněný podprostorový model (Generalized Subspace Model) - teoretický rámec umožňující definovat pravděpodobnostní rozdělení v nízkodimenzionálních nadplochách (manifoldech) ve vysokorozměrném prostoru parametrů. Pomocí tohoto nástroje učíme fonetický podprostor - kontinuum vektorových reprezentací (embeddingů) fonémů - z několika jazyků s přepsanými nahrávkami. Pak je tento fonetický podprostor použit k omezení našeho systému tak, aby určené akustické jednotky byly podobné fonémům z ostatních jazyků. Experimentální výsledky ukazují,že tento přístup významně zlepšuje kvalitu shlukování i přesnost segmentace systému pro určování akustických jednotek.

No description available.

[pt] Hoje em dia, grande parte do conhecimento acumulado está armazenada em forma de dados. Dentre as ferramentas capazes de atuar como modelos representativos de sistemas reais, os Sistemas de Inferência Fuzzy têm se destacado pela capacidade de fornecer modelos precisos e, ao mesmo tempo, interpretáveis. A interpretabilidade é obtida a partir de regras linguísticas, que podem ser extraídas de bases de dados bases históricas e que permitem ao usuário compreender a relação entre as variáveis do problema. Entretanto, tais sistemas sofrem com a maldição da dimensionalidade ao lidar com problemas complexos, isto é, com um grande número de variáveis de entrada ou padrões, gerando problemas de escalabilidade. Esta dissertação apresenta um novo algoritmo de geração automática de regras, denominado RandomFIS, especificamente para problemas de classificação, capaz de lidar com grandes bases de dados tanto em termos de número de variáveis de entrada (atributos) quanto em termos de padrões (instâncias). O modelo RandomFIS utiliza os conceitos de seleção de variáveis (Random Subspace) e Bag of Little Bootstrap (BLB), que é uma versão escalável do Bootstrapping, criando uma estrutura de comitê de classificadores. O RandomFIS é avaliado em várias bases benchmark, demostrando ser um modelo robusto que mantém a interpretabilidade e apresenta boa acurácia mesmo em problemas envolvendo grandes bases de dados. / [en] Nowadays, much of the accumulated knowledge is stored as data. Among the tools capable of acting as representative models of real systems, Fuzzy Inference Systems are recognized by their ability to provide accurate and at the same time interpretable models. Interpretability is obtained from linguistic rules, which can be extracted from historical databases. These rules allow the end user to understand the relationship between variables in a specific problem. However, such systems experience the curse of dimensionality when handling complex problems, i.e. with a large number of input variables or patterns in the dataset, giving origin to scalability issues. This dissertation presents a new algorithm for automatic generation of fuzzy rules, called RandomFIS, specifically for classification problems, which is able to handle large databases both in terms of number of input variables (attributes) and in terms of patterns (instances). The RandomFIS model makes use of feature selection concepts (Random Subspace) and Bag of Little Bootstrap (BLB), which is a scalable version of Bootstrapping, creating a classifier committee structure. RandomFIS is tested in several benchmark datasets and shows to be a robust model that maintains interpretability and good accuracy even in problems involving large databases.

Direction Finding (DF) algorithms are used by the Swedish Defence Research Agency (FOI) in the context of electronic warfare against radio. Parallelizing these algorithms using a Graphics Processing Unit (GPU) might improve performance, and thereby increase military support capabilities. This thesis selects the DF algorithms Correlative Interferometer (CORR), Multiple Signal Classification (MUSIC) and Weighted Subspace Fitting (WSF), and examines to what extent GPU implementation of these algorithms is suitable, by analysing, implementing and evaluating. Firstly, six general criteria for GPU suitability are formulated. Then the three algorithms are analyzed with regard to these criteria, giving that MUSIC and WSF are both 58% suitable, closely followed by CORR on 50% suitability. MUSIC is selected for implementation, and an open source implementation is extended to three versions: a multicore CPU version, a GPU version (with Eigenvalue Decomposition (EVD) and pseudo spectrum calculation performed on the GPU), and a MIXED version (with only pseudo spectrum calculation on the GPU). These versions are then evaluated for angle resolutions between 1° and 0.025°, and CUDA block sizes between 8 and 1024. It is found that the GPU version is faster than the CPU version for angle resolutions above 0.1°, and the largest measured speedup is 1.4 times. The block size has no large impact on the total runtime. In conclusion, the overall results indicate that it is not entirely suitable, yet somewhat beneficial for large angle resolutions, to implement MUSIC using GPU computing.

This thesis focuses on four main research themes namely shot boundary detection, fast frame alignment, activity-driven video summarisation, and highlights based video annotation and retrieval. A number of novel algorithms have been proposed to address these issues, which can be highlighted as follows. Firstly, accurate and robust shot boundary detection is achieved through modelling of cuts into sub-categories and appearance based modelling of several gradual transitions, along with some novel features extracted from compressed video. Secondly, fast and robust frame alignment is achieved via the proposed subspace phase correlation (SPC) and an improved sub-pixel strategy. The SPC is proved to be insensitive to zero-mean-noise, and its gradient-based extension is even robust to non-zero-mean noise and can be used to deal with non-overlapped regions for robust image registration. Thirdly, hierarchical modelling of rush videos using formal language techniques is proposed, which can guide the modelling and removal of several kinds of junk frames as well as adaptive clustering of retakes. With an extracted activity level measurement, shot and sub-shot are detected for content-adaptive video summarisation. Fourthly, highlights based video annotation and retrieval is achieved, in which statistical modelling of skin pixel colours, knowledge-based shot detection, and improved determination of camera motion patterns are employed. Within these proposed techniques, one important principle is to integrate various kinds of feature evidence and to incorporate prior knowledge in modelling the given problems. High-level hierarchical representation is extracted from the original linear structure for effective management and content-based retrieval of video data. As most of the work is implemented in the compressed domain, one additional benefit is the achieved high efficiency, which will be useful for many online applications. / EU IST FP6 Project

In this dissertation, some new algorithms are developed for hyperspectral imaging analysis enhancement. Tensor data format is applied in hyperspectral dataset sparse and low-rank decomposition, which could enhance the classification and detection performance. And multi-view learning technique is applied in hyperspectral imaging clustering. Furthermore, kernel version of multi-view learning technique has been proposed, which could improve clustering performance. Most of low-rank and sparse decomposition algorithms are based on matrix data format for HSI analysis. As HSI contains high spectral dimensions, tensor based extended low-rank and sparse decomposition (TELRSD) is proposed in this dissertation for better performance of HSI classification with low-rank tensor part, and HSI detection with sparse tensor part. With this tensor based method, HSI is processed in 3D data format, and information between spectral bands and pixels maintain integrated during decomposition process. This proposed algorithm is compared with other state-of-art methods. And the experiment results show that TELRSD has the best performance among all those comparison algorithms. HSI clustering is an unsupervised task, which aims to group pixels into different groups without labeled information. Low-rank sparse subspace clustering (LRSSC) is the most popular algorithms for this clustering task. The spatial-spectral based multi-view low-rank sparse subspace clustering (SSMLC) algorithms is proposed in this dissertation, which extended LRSSC with multi-view learning technique. In this algorithm, spectral and spatial views are created to generate multi-view dataset of HSI, where spectral partition, morphological component analysis (MCA) and principle component analysis (PCA) are applied to create others views. Furthermore, kernel version of SSMLC (k-SSMLC) also has been investigated. The performance of SSMLC and k-SSMLC are compared with sparse subspace clustering (SSC), low-rank sparse subspace clustering (LRSSC), and spectral-spatial sparse subspace clustering (S4C). It has shown that SSMLC could improve the performance of LRSSC, and k-SSMLC has the best performance. The spectral clustering has been proved that it equivalent to non-negative matrix factorization (NMF) problem. In this case, NMF could be applied to the clustering problem. In order to include local and nonlinear features in data source, orthogonal NMF (ONMF), graph-regularized NMF (GNMF) and kernel NMF (k-NMF) has been proposed for better clustering performance. The non-linear orthogonal graph NMF combine both kernel, orthogonal and graph constraints in NMF (k-OGNMF), which push up the clustering performance further. In the HSI domain, kernel multi-view based orthogonal graph NMF (k-MOGNMF) is applied for subspace clustering, where k-OGNMF is extended with multi-view algorithm, and it has better performance and computation efficiency.

In this thesis I propose methods and strategies for the design of advanced model predictive control designs. The contributions are in the areas of data-driven model based MPC, model monitoring and explicit incorporation of closed-loop response considerations in the MPC, while handling issues such as plant-model mismatch, constraints and uncertainty. In the initial phase of this research, I address the problem of handling plant-model mismatch by designing a subspace identification based MPC framework that includes model monitoring and closed-loop identification components. In contrast to performance monitoring based approaches, the validity of the underlying model is monitored by proposing two indexes that compare model predictions with measured past output. In the event that the model monitoring threshold is breached, a new model is identified using an adapted closed-loop subspace identification method. To retain the knowledge of the nominal system dynamics, the proposed approach uses the past training data and current input, output and set-point as the training data for re-identification. A model validity mechanism then checks if the new model predictions are better than the existing model, and if they are, then the new model is utilized within the MPC. Next, the proposed MPC with re-identification method is extended to batch processes. To this end, I first utilize a subspace-based model identification approach for batch processes to be used in model predictive control. A model performance index is developed for batch process, then in the case of poor prediction, re-identification is triggered to identify a new model. In order to emphasize on the recent batch data, the identification is developed in order to increase the contribution of the current data. In another direction, the stability of data driven predictive control is addressed. To this end, first, a data-driven Lyapunov-based MPC is designed, and shown to be capable of stabilizing a system at an unstable equilibrium point. The data driven Lyapunov-based MPC utilizes a linear time invariant (LTI) model cognizant of the fact that the training data, owing to the unstable nature of the equilibrium point, has to be obtained from closed-loop operation or experiments. Simulation results are first presented demonstrating closed-loop stability under the proposed data-driven Lyapunov-based MPC. The underlying data-driven model is then utilized as the basis to design an economic MPC. Finally, I address the problem of control of nonlinear systems to deliver a prescribed closed-loop behavior. In particular, the framework allows for the practitioner to first specify the nature and specifics of the desired closed-loop behavior (e.g., first order with smallest time constant, second order with no more than a certain percentage overshoot, etc.). An optimization based formulation then computes the control action to deliver the best attainable closed loop behavior. To decouple the problems of determining the best attainable behavior and tracking it as closely as possible, the optimization problem is posed and solved in two tiers. In the first tier, the focus is on determining the best closed-loop behavior attainable, subject to stability and tracking constraints. In the second tier, the inputs are tweaked to possibly improve the tracking of the optimal output trajectories given by the first tier. The effectiveness of all of the proposed methods are illustrated through simulations on nonlinear systems. / Dissertation / Doctor of Philosophy (PhD)