Multiple Sequenzalignments (MSAs) von homologen Proteinen sind nützliche Werkzeuge, um kompensatorische Mutationen zwischen nicht-konservierten Residuen zu charakterisieren. Die Identifizierung dieser Residuen in MSAs ist eine wichtige Aufgabe um die strukturellen Grundlagen und molekularen Mechanismen von Proteinfunktionen besser zu verstehen. Trotz der vielen Anzahl an Literatur über kompensatorische Mutationen sowie über die Sequenzkonservierungsanalyse für die Erkennung von wichtigen Residuen, haben vorherige Methoden meistens die biochemischen Eigenschaften von Aminosäuren nicht mit in Betracht gezogen, welche allerdings entscheidend für die Erkennung von kompensatorischen Mutationssignalen sein können. Jedoch werden kompensatorische Mutationssignale in MSAs oft durch das Rauschen verfälscht. Aus diesem Grund besteht ein weiteres Problem der Bioinformatik in der Trennung signifikanter Signale vom phylogenetischen Rauschen und beziehungslosen Paarsignalen. Das Ziel dieser Arbeit besteht darin Methoden zu entwickeln, welche biochemische Eigenschaften wie Ähnlichkeiten und Unähnlichkeiten von Aminosäuren in der Identifizierung von kompensatorischen Mutationen integriert und sich mit dem Rauschen auseinandersetzt. Deshalb entwickeln wir unterschiedliche Methoden basierend auf klassischer- und quantum Informationstheorie sowie multiple Testverfahren. Unsere erste Methode basiert auf der klassischen Informationstheorie. Diese Methode betrachtet hauptsächlich BLOSUM62-unähnliche Paare von Aminosäuren als ein Modell von kompensatorischen Mutationen und integriert sie in die Identifizierung von wichtigen Residuen. Um diese Methode zu ergänzen, entwickeln wir unsere zweite Methode unter Verwendung der Grundlagen von quantum Informationstheorie. Diese neue Methode unterscheidet sich von der ersten Methode durch gleichzeitige Modellierung ähnlicher und unähnlicher Signale in der kompensatorischen Mutationsanalyse. Des Weiteren, um signifikante Signale vom Rauschen zu trennen, entwickeln wir ein MSA-spezifisch statistisches Modell in Bezug auf multiple Testverfahren. Wir wenden unsere Methode für zwei menschliche Proteine an, nämlich epidermal growth factor receptor (EGFR) und glucokinase (GCK). Die Ergebnisse zeigen, dass das MSA-spezifisch statistische Modell die signifikanten Signale vom phylogenetischen Rauschen und von beziehungslosen Paarsignalen trennen kann. Nur unter Berücksichtigung BLOSUM62-unähnlicher Paare von Aminosäuren identifiziert die erste Methode erfolgreich die krankheits-assoziierten wichtigen Residuen der beiden Proteine. Im Gegensatz dazu, durch die gleichzeitige Modellierung ähnlicher und unähnlicher Signale von Aminosäurepaare ist die zweite Methode sensibler für die Identifizierung von katalytischen und allosterischen Residuen.

Image registration is a fundamental problem that can be found in a diverse range of fields within the research community. It is used in areas such as engineering, science, medicine, robotics, computer vision and image processing, which often require the process of developing a spatial mapping between sets of data. Registration plays a crucial role in the medical imaging field where continual advances in imaging modalities, including MRI, CT and PET, allow the generation of 3D images that explicitly outline detailed in vivo information of not only human anatomy, but also human function. Mutual Information (MI) is a popular entropy-based similarity measure which has found use in a large number of image registration applications. Stemming from information theory, this measure generally outperforms most other intensity-based measures in multimodal applications as it does not assume the existence of any specific relationship between image intensities. It only assumes a statistical dependence. The basic concept behind any approach using MI is to find a transformation, which when applied to an image, will maximise the MI between two images. This thesis presents research using MI in three major topics encompassed by the computer vision and medical imaging field: rigid image registration, stereo vision, and non-rigid image registration. In the rigid domain, a novel gradient-based registration algorithm (MIGH) is proposed that uses Parzen windows to estimate image density functions and Gauss-Hermite quadrature to estimate the image entropies. The use of this quadrature technique provides an effective and efficient way of estimating entropy while bypassing the need to draw a second sample of image intensities (a procedure required in previous Parzen-based MI registration approaches). It is possible to achieve identical results with the MIGH algorithm when compared to current state of the art MI-based techniques. These results are achieved using half the previously required sample sizes, thus doubling the statistical power of the registration algorithm. Furthermore, the MIGH technique improves algorithm complexity by up to an order of N, where N represents the number of samples extracted from the images. In stereo vision, a popular passive method of depth perception, new extensions have been pro- posed in order to increase the robustness of MI-based stereo matching algorithms. Firstly, prior probabilities are incorporated into the MI measure to considerably increase the statistical power of the matching windows. The statistical power, directly related to the number of samples, can become too low when small matching windows are utilised. These priors, which are calculated from the global joint histogram, are tuned to a two level hierarchical approach. A 2D match surface, in which the match score is computed for every possible combination of template and matching windows, is also utilised to enforce left-right consistency and uniqueness constraints. These additions to MI-based stereo matching significantly enhance the algorithms ability to detect correct matches while decreasing computation time and improving the accuracy, particularly when matching across multi-spectra stereo pairs. MI has also recently found use in the non-rigid domain due to a need to compute multimodal non-rigid transformations. The viscous fluid algorithm is perhaps the best method for re- covering large local mis-registrations between two images. However, this model can only be used on images from the same modality as it assumes similar intensity values between images. Consequently, a hybrid MI-Fluid algorithm is proposed to compute a multimodal non-rigid registration technique. MI is incorporated via the use of a block matching procedure to generate a sparse deformation field which drives the viscous fluid algorithm, This algorithm is also compared to two other popular local registration techniques, namely Gaussian convolution and the thin-plate spline warp, and is shown to produce comparable results. An improved block matching procedure is also proposed whereby a Reversible Jump Markov Chain Monte Carlo (RJMCMC) sampler is used to optimally locate grid points of interest. These grid points have a larger concentration in regions of high information and a lower concentration in regions of small information. Previous methods utilise only a uniform distribution of grid points throughout the image.

Abstract In this thesis, we consider autonomous environmental modelling, where robotic sensing platforms are utilized in environmental surveying. In order to allow a wide range of different environments, our models must be flexible to the data with some a prior assumptions. Respectively, in order to guide action planning, we need to have a unified sensing quality metric that depends on the prediction quality of our models. Finally, in order to be able to adapt to the observed information, at each iteration of the action planning algorithm, we must be able to provide solutions that aim at minimum travelling time needed to reach a certain level of sensing quality. These are the main topics in this thesis. At the center of our approaches are stationary and non-stationary Gaussian processes based on the assumption that the observed phenomenon is due to the diffusion of white noise, where diffusion kernel anisotropy and scale may vary between locations. For these models, we propose adaptation of diffusion kernels based on a structure tensor approach. Proposed methods are demonstrated with experiments that show, assuming sensor noise is not dominating, our iterative approach is able to return diffusion kernel values close to correct ones. In order to quantify how precise our models are, we propose a mutual information based sensing quality criterion, and prove that the optimal design using our sensing quality provides the best prediction quality for the model. To incorporate localization uncertainty in modelling, we also propose an approach where a posterior model is marginalized over sensing path distribution. The benefit is that this approach implicitly favors actions that result in previously visited or otherwise well-defined areas, meanwhile, maximizing the information gain. Experiments support our claims that our proposed approaches are best when considering predictive distribution quality. In action planning, our approach is to use graph-based approximation algorithms to obtain a certain level of model quality in an efficient way. In order account for spatial dependency and active localization, we propose adaptation methods that map sensing quality to vertex prices in a graph. Experiments demonstrate the benefit of our adaptation methods compared to the action planning algorithms that do not consider these specific features. / Tiivistelmä Tässä väitöskirjassa tarkastellaan autonomista ympäristön mallinnusta, missä ympäristön kartoitukseen hyödynnetään robottimittausalustoja. Erilaisia ympäristöjä varten, käytettävien mallien tulee olla joustavia datalle tietyillä a priori oletuksilla. Mittausalustojen ohjaus vaatii vastaavasti yhtenäisen, mallien ennustuslaadusta riippuvan, kartoituksen laatumetriikan. Mukautuakseen uuteen informaatioon, ohjausalgoritmin tulee lisäksi pyrkiä joka iteraatiolla minimoimaan tietyn kartoituksen laadun saavuttava kulkuaika. Nämä ovat tämän väitöskirjan pääaiheet. Tämän väitöskirjan keskiössä ovat sellaiset stationaariset ja ei-stationaariset Gaussin prosessit, jotka perustuvat oletukseen että havaittu ilmiö johtuu valkoisen kohinan diffuusiosta. Diffuusiokernelin anisotrooppisuudelle ja skaalalle sallitaan paikkariippuvaisuus. Tässä väitöskirjassa esitetään näiden mallien mukauttamiseen rakennetensoripohjaisia menetelmiä. Suoritetut kokeet osoittavat, että esitetyt iteratiiviset mukauttamismenetelmät tuottavat lähes oikeita diffuusiokernelien arvoja, olettaen, että sensorikohina ei dominoi mittauksia. Mallien ennustustarkkuuden määrittämiseen esitetään keskinäisinformaatioon perustuva kartoituksen laatumetriikka. Väitöskirjassa todistetaan, että optimaalinen ennustuslaatu saavutetaan käyttämällä esitettyä laatumetriikkaa. Väitöskirjassa esitetään lisäksi laatumetriikka, jossa posteriori malli on marginalisoitu kartoituspolkujen jakauman yli. Tämän avulla voidaan huomioida paikannusepävarmuuden vaikutukset mallinnuksessa. Tällöin etuna on se, että kyseinen laatumetriikka suosii implisiittisesti sellaisia mittausalustojen ohjauksia, jotka johtavat aeimmin kartoitetuille tai helposti ennustettaville alueille samalla maksimoiden informaatiohyödyn. Suoritetut kokeet tukevat väittämiä, että väitöskirjassa esitetyt menetelmät tuottavat parhaan ennustusjakauman laadun. Mittausalustojen ohjaus vaatii vastaavasti yhtenäisen, mallien ennustuslaadusta riippuvan, kartoituksen laatumetriikan. Väitöskirjassa esitetään mukautusmenetelmiä kartoituksen laadun kuvaukseksi graafin solmujen kustannuksiksi. Tämän avulla sallitaan sekä spatiaalinen riippuvuus että aktiivinen paikannus. Mittausalustojen ohjaus vaatii vastaavasti yhtenäisen, mallien ennustuslaadusta riippuvan, kartoituksen laatumetriikan.

Coordenação de Aperfeiçoamento de Pessoal de Nível Superior - CAPES / Model predictive control (MPC) strategies have become the standard for advanced control applications in the process industry. Significant benefits are generated from the MPC's capacity to ensure that the plant operates within its constraints more profitably. However, like any controller, after some time under operation, MPCs rarely function as when they were initially designed. A large percentage of performance degradation of MPC is associated with the deterioration of model that controller uses to predict process outputs and calculate inputs. The objective of the present work is implementation of mathematical methods that can be used to detect model-plant mismatch in linear and nonlinear MPC systems. In this work, techniques based on cross correlation, partial correlation and mutual information are implemented and tested by numerical simulation in case studies characteristic of the petrochemical industry, represented by linear and nonlinear models, operating under MPC control. The results obtained through the applying the techniques are analyzed and compared as to their efficiency is not intended to offer their potential for real industrial applications. / Estratégias de controle preditivo (MPC) têm-se tornado o padrão para aplicações de controle avançado na indústria de processos. Os benefícios significativos são gerados a partir da habilidade do controlador MPC de assegurar que a planta opere dentro das restrições de forma mais lucrativa. Porém, como todo controlador, depois de algum tempo em operação, os MPCs raramente funcionam como quando foram inicialmente projetados. Uma grande porcentagem da degradação do desempenho dos controladores MPC está associada à deterioração do modelo que o controlador usa para fazer a predição das saídas do processo e calcular as entradas. O objetivo do presente trabalho é a implementação de métodos matemáticos que possam ser utilizados para a detecção de erros planta-modelo em sistemas de controle MPC lineares e não lineares. Neste trabalho, técnicas baseadas em correlação cruzada, correlação parcial e informação mútua são implementadas e testadas por simulação numérica em estudos de caso característicos da indústria petroquímica, representados por modelos lineares e não lineares, operando sob controle MPC. Os resultados obtidos através da aplicação das técnicas são analisados e comparados quanto à sua eficiência no objetivo proposto avaliando seu potencial para aplicações industriais reais.

The estimation of the ratio of two probability densities is an important statistical tool in supervised machine learning. In this work, we introduce new methods of density ratio estimation based on the solution of a multidimensional integral equation involving cumulative distribution functions. The resulting methods use the novel V -matrix, a concept that does not appear in previous density ratio estimation methods. Experiments demonstrate the good potential of this new approach against previous methods. Mutual Information - MI - estimation is a key component in feature selection and essentially depends on density ratio estimation. Using one of the methods of density ratio estimation proposed in this work, we derive a new estimator - VMI - and compare it experimentally to previously proposed MI estimators. Experiments conducted solely on mutual information estimation show that VMI compares favorably to previous estimators. Experiments applying MI estimation to feature selection in classification tasks evidence that better MI estimation leads to better feature selection performance. Parameter selection greatly impacts the classification accuracy of the kernel-based Support Vector Machines - SVM. However, this step is often overlooked in experimental comparisons, for it is time consuming and requires familiarity with the inner workings of SVM. In this work, we propose procedures for SVM parameter selection which are economic in their running time. In addition, we propose the use of a non-linear kernel function - the min kernel - that can be applied to both low- and high-dimensional cases without adding another parameter to the selection process. The combination of the proposed parameter selection procedures and the min kernel yields a convenient way of economically extracting good classification performance from SVM. The Regularized Least Squares - RLS - regression method is another kernel method that depends on proper selection of its parameters. When training data is scarce, traditional parameter selection often leads to poor regression estimation. In order to mitigate this issue, we explore a kernel that is less susceptible to overfitting - the additive INK-splines kernel. Then, we consider alternative parameter selection methods to cross-validation that have been shown to perform well for other regression methods. Experiments conducted on real-world datasets show that the additive INK-splines kernel outperforms both the RBF and the previously proposed multiplicative INK-splines kernel. They also show that the alternative parameter selection procedures fail to consistently improve performance. Still, we find that the Finite Prediction Error method with the additive INK-splines kernel performs comparably to cross-validation. / A estimação da razão entre duas densidades de probabilidade é uma importante ferramenta no aprendizado de máquina supervisionado. Neste trabalho, novos métodos de estimação da razão de densidades são propostos baseados na solução de uma equação integral multidimensional. Os métodos resultantes usam o conceito de matriz-V , o qual não aparece em métodos anteriores de estimação da razão de densidades. Experimentos demonstram o bom potencial da nova abordagem com relação a métodos anteriores. A estimação da Informação Mútua - IM - é um componente importante em seleção de atributos e depende essencialmente da estimação da razão de densidades. Usando o método de estimação da razão de densidades proposto neste trabalho, um novo estimador - VMI - é proposto e comparado experimentalmente a estimadores de IM anteriores. Experimentos conduzidos na estimação de IM mostram que VMI atinge melhor desempenho na estimação do que métodos anteriores. Experimentos que aplicam estimação de IM em seleção de atributos para classificação evidenciam que uma melhor estimação de IM leva as melhorias na seleção de atributos. A tarefa de seleção de parâmetros impacta fortemente o classificador baseado em kernel Support Vector Machines - SVM. Contudo, esse passo é frequentemente deixado de lado em avaliações experimentais, pois costuma consumir tempo computacional e requerer familiaridade com as engrenagens de SVM. Neste trabalho, procedimentos de seleção de parâmetros para SVM são propostos de tal forma a serem econômicos em gasto de tempo computacional. Além disso, o uso de um kernel não linear - o chamado kernel min - é proposto de tal forma que possa ser aplicado a casos de baixa e alta dimensionalidade e sem adicionar um outro parâmetro a ser selecionado. A combinação dos procedimentos de seleção de parâmetros propostos com o kernel min produz uma maneira conveniente de se extrair economicamente um classificador SVM com boa performance. O método de regressão Regularized Least Squares - RLS - é um outro método baseado em kernel que depende de uma seleção de parâmetros adequada. Quando dados de treinamento são escassos, uma seleção de parâmetros tradicional em RLS frequentemente leva a uma estimação ruim da função de regressão. Para aliviar esse problema, é explorado neste trabalho um kernel menos suscetível a superajuste - o kernel INK-splines aditivo. Após, são explorados métodos de seleção de parâmetros alternativos à validação cruzada e que obtiveram bom desempenho em outros métodos de regressão. Experimentos conduzidos em conjuntos de dados reais mostram que o kernel INK-splines aditivo tem desempenho superior ao kernel RBF e ao kernel INK-splines multiplicativo previamente proposto. Os experimentos também mostram que os procedimentos alternativos de seleção de parâmetros considerados não melhoram consistentemente o desempenho. Ainda assim, o método Finite Prediction Error com o kernel INK-splines aditivo possui desempenho comparável à validação cruzada.

Les standards « Long-term evolution » (LTE) et LTE-Advanced (LTE-A) devraient influencer fortement l’avenir de la cinquième génération (5G) des réseaux mobiles. Ces normes exigent de hauts débits de données et une qualité de service de très bon niveau, ce qui permet d’assurer un faible taux d’erreur, avec une faible latence. Par ailleurs, la complexité doit être limitée. Dans le but de déterminer des solutions technologiques modernes qui satisfont ces contraintes fortes, nous étudions dans la thèse des systèmes de communication sans fil MIMO codés. D’abord, nous imposons un simple code convolutif récursif systématique (RSC) pour limiter la complexité et la latence. En considérant des récepteurs itératifs, nous optimisons alors la performance en termes de taux d’erreur de ces systèmes en définissant un précodage linéaire MIMO et des techniques de mapping appropriées. Dans la deuxième partie de la thèse, nous remplaçons le RSC par un LDPC non-binaire (NB-LDPC). Nous proposons d’utiliser les techniques de précodage MIMO afin de réduire la complexité des récepteurs des systèmes MIMO intégrant des codes NB-LDPC. Enfin, nous proposons également un nouvel algorithme de décodage itératif à faible complexité adapté aux codes NB-LDPC. / The long-term evolution (LTE) and the LTE-Advanced (LTE-A) standardizations are predicted to play essential roles in the future fifth-generation (5G) mobile networks. These standardizations require high data rate and high quality of service, which assures low error-rate and low latency. Besides, as discussed in the recent surveys, low complexity communication systems are also essential in the next 5G mobile networks. To adapt to the modern trend of technology, in this PhD thesis, we investigate the multiple-input multiple-output (MIMO) wireless communication schemes. In the first part of this thesis, low-complex forward error correction (FEC) codes are used for low complexity and latency. By considering iterative receivers at the receiver side, we exploit MIMO linear precoding and mapping methods to optimize the error-rate performance of these systems. In the second part of this thesis, non-binary low density parity check (NB-LDPC) codes are investigated. We propose to use MIMO precoders to reduce the complexity for NB-LDPC encoded MIMO systems. A novel low complexity decoding algorithm for NB-LDPC codes is also proposed at the end of this thesis.

Les travaux présentés dans cette thèse s'inscrivent dans la problématique de la connectivité cérébrale, connectivité tripartite puisqu'elle sous-tend les notions de connectivité structurelle, fonctionnelle et effective. Ces trois types de connectivité que l'on peut considérer à différentes échelles d'espace et de temps sont bien évidemment liés et leur analyse conjointe permet de mieux comprendre comment structures et fonctions cérébrales se contraignent mutuellement. Notre recherche relève plus particulièrement de la connectivité effective qui permet de définir des graphes de connectivité qui renseignent sur les liens causaux, directs ou indirects, unilatéraux ou bilatéraux via des chemins de propagation, représentés par des arcs, entre les nœuds, ces derniers correspondant aux régions cérébrales à l'échelle macroscopique. Identifier les interactions entre les aires cérébrales impliquées dans la génération et la propagation des crises épileptiques à partir d'enregistrements intracérébraux est un enjeu majeur dans la phase pré-chirurgicale et l'objectif principal de notre travail. L'exploration de la connectivité effective suit généralement deux approches, soit une approche basée sur les modèles, soit une approche conduite par les données comme nous l'envisageons dans le cadre de cette thèse où les outils développés relèvent de la théorie de l'information et plus spécifiquement de l'entropie de transfert, la question phare que nous adressons étant celle de la précision des estimateurs de cette grandeur dans le cas des méthodes développées basées sur les plus proches voisins. Les approches que nous proposons qui réduisent le biais au regard d'estimateurs issus de la littérature sont évaluées et comparées sur des signaux simulés de type bruits blancs, processus vectoriels autorégressifs linéaires et non linéaires, ainsi que sur des modèles physiologiques réalistes avant d'être appliquées sur des signaux électroencéphalographiques de profondeur enregistrés sur un patient épileptique et comparées à une approche assez classique basée sur la fonction de transfert dirigée. En simulation, dans les situations présentant des non-linéarités, les résultats obtenus permettent d'apprécier la réduction du biais d'estimation pour des variances comparables vis-à-vis des techniques connues. Si les informations recueillies sur les données réelles sont plus difficiles à analyser, elles montrent certaines cohérences entre les méthodes même si les résultats préliminaires obtenus s'avèrent davantage en accord avec les conclusions des experts cliniciens en appliquant la fonction de transfert dirigée. / The work presented in this thesis deals with brain connectivity, including structural connectivity, functional connectivity and effective connectivity. These three types of connectivities are obviously linked, and their joint analysis can give us a better understanding on how brain structures and functions constrain each other. Our research particularly focuses on effective connectivity that defines connectivity graphs with information on causal links that may be direct or indirect, unidirectional or bidirectional. The main purpose of our work is to identify interactions between different brain areas from intracerebral recordings during the generation and propagation of seizure onsets, a major issue in the pre-surgical phase of epilepsy surgery treatment. Exploring effective connectivity generally follows two kinds of approaches, model-based techniques and data-driven ones. In this work, we address the question of improving the estimation of information-theoretic quantities, mainly mutual information and transfer entropy, based on k-Nearest Neighbors techniques. The proposed approaches we developed are first evaluated and compared with existing estimators on simulated signals including white noise processes, linear and nonlinear vectorial autoregressive processes, as well as realistic physiology-based models. Some of them are then applied on intracerebral electroencephalographic signals recorded on an epileptic patient, and compared with the well-known directed transfer function. The experimental results show that the proposed techniques improve the estimation of information-theoretic quantities for simulated signals, while the analysis is more difficult in real situations. Globally, the different estimators appear coherent and in accordance with the ground truth given by the clinical experts, the directed transfer function leading to interesting performance.

This thesis focuses on possibility of chaotic (specially economic) time-series prediction. Chaotic time-series are unpredictable in long-term due to their high sensitivity on initial conditions. Nevertheless, their behavior should be more or less predictable in short-term. Goal of this thesis is to show, how much and if any prediction, is possible by non-linear prediction method, and try to reveal or to reject presence of chaotic behavior in them. Work is split into three chapters. Chapter One briefly introduces chosen important concepts and methods from this area. In addition, to describe some prediction methods, there are outlined which indicators and methods are possible to use in order to find possibilities and boundaries of this prediction. Chapter Two is focused on modifications of FracLab software, which is used for create this prediction. Last chapter is experimental. Besides the description of examined time-series and methods, it includes discussion of results.

In the human visual system, one of the most prominent functions of the extensive feedback from the higher brain areas within and outside of the visual cortex is attentional modulation. The feedback helps the brain to concentrate its resources on visual features that are relevant for recognition, i. e. it iteratively selects certain aspects of the visual scene for refined processing by the lower areas until the inference process in the higher areas converges to a single hypothesis about this scene. In order to minimize a number of required selection-refinement iterations, one has to find a short sequence of maximally informative portions of the visual input. Since the feedback is not static, the selection process is adapted to a scene that should be recognized. To find a scene-specific subset of informative features, the adaptive selection process on every iteration utilizes results of previous processing in order to reduce the remaining uncertainty about the visual scene. This phenomenon inspired us to develop a computational algorithm solving a visual classification task that would incorporate such principle, adaptive feature selection. It is especially interesting because usually feature selection methods are not adaptive as they define a unique set of informative features for a task and use them for classifying all objects. However, an adaptive algorithm selects features that are the most informative for the particular input. Thus, the selection process should be driven by statistics of the environment concerning the current task and the object to be classified. Applied to a classification task, our adaptive feature selection algorithm favors features that maximally reduce the current class uncertainty, which is iteratively updated with values of the previously selected features that are observed on the testing sample. In information-theoretical terms, the selection criterion is the mutual information of a class variable and a feature-candidate conditioned on the already selected features, which take values observed on the current testing sample. Then, the main question investigated in this thesis is whether the proposed adaptive way of selecting features is advantageous over the conventional feature selection and in which situations. Further, we studied whether the proposed adaptive information-theoretical selection scheme, which is a computationally complex algorithm, is utilized by humans while they perform a visual classification task. For this, we constructed a psychophysical experiment where people had to select image parts that as they think are relevant for classification of these images. We present the analysis of behavioral data where we investigate whether human strategies of task-dependent selective attention can be explained by a simple ranker based on the mutual information, a more complex feature selection algorithm based on the conventional static mutual information and the proposed here adaptive feature selector that mimics a mechanism of the iterative hypothesis refinement. Hereby, the main contribution of this work is the adaptive feature selection criterion based on the conditional mutual information. Also it is shown that such adaptive selection strategy is indeed used by people while performing visual classification.:1. Introduction 2. Conventional feature selection 3. Adaptive feature selection 4. Experimental investigations of ACMIFS 5. Information-theoretical strategies of selective attention 6. Discussion Appendix Bibliography

This thesis investigates the extension of the well-known canonical correlation analysis for random elements on abstract real measurable Hilbert spaces. One focus is on the application of this extension to the calculation of information-theoretical quantities on finite time intervals. Analytical approaches for the calculation of the mutual information and the information density between Gaussian distributed random elements on arbitrary real measurable Hilbert spaces are derived. With respect to mutual information, the results obtained are comparable to [4] and [1] (Baker, 1970, 1978). They can also be seen as a generalization of earlier findings in [20] (Gelfand and Yaglom, 1958). In addition, some of the derived equations for calculating the information density, its characteristic function and its n-th central moments extend results from [45] and [44] (Pinsker, 1963, 1964). Furthermore, explicit examples for the calculation of the mutual information, the characteristic function of the information density as well as the n-th central moments of the information density for the important special case of an additive Gaussian channel with Gaussian distributed input signal with rational spectral density are elaborated, on the one hand for white Gaussian noise and on the other hand for Gaussian noise with rational spectral density. These results extend the corresponding concrete examples for the calculation of the mutual information from [20] (Gelfand and Yaglom, 1958) as well as [28] and [29] (Huang and Johnson, 1963, 1962).:Kurzfassung Abstract Notations Abbreviations 1 Introduction 1.1 Software Used 2 Mathematical Background 2.1 Basic Notions of Measure and Probability Theory 2.1.1 Characteristic Functions 2.2 Stochastic Processes 2.2.1 The Consistency Theorem of Daniell and Kolmogorov 2.2.2 Second Order Random Processes 2.3 Some Properties of Fourier Transforms 2.4 Some Basic Inequalities 2.5 Some Fundamentals in Functional Analysis 2.5.1 Hilbert Spaces 2.5.2 Linear Operators on Hilbert Spaces 2.5.3 The Fréchet-Riesz Representation Theorem 2.5.4 Adjoint and Compact Operators 2.5.5 The Spectral Theorem for Compact Operators 3 Mutual Information and Information Density 3.1 Mutual Information 3.2 Information Density 4 Probability Measures on Hilbert Spaces 4.1 Measurable Hilbert Spaces 4.2 The Characteristic Functional 4.3 Mean Value and Covariance Operator 4.4 Gaussian Probability Measures on Hilbert Spaces 4.5 The Product of Two Measurable Hilbert Spaces 4.5.1 The Product Measure 4.5.2 Cross-Covariance Operator 5 Canonical Correlation Analysis on Hilbert Spaces 5.1 The Hellinger Distance and the Theorem of Kakutani 5.2 Canonical Correlation Analysis on Hilbert Spaces 5.3 The Theorem of Hájek and Feldman 6 Mutual Information and Information Density Between Gaussian Measures 6.1 A General Formula for Mutual Information and Information Density for Gaussian Random Elements 6.2 Hadamard’s Factorization Theorem 6.3 Closed Form Expressions for Mutual Information and Related Quantities 6.4 The Discrete-Time Case 6.5 The Continuous-Time Case 6.6 Approximation Error 7 Additive Gaussian Channels 7.1 Abstract Channel Model and General Definitions 7.2 Explicit Expressions for Mutual Information and Related Quantities 7.2.1 Gaussian Random Elements as Input to an Additive Gaussian Channel 8 Continuous-Time Gaussian Channels 8.1 White Gaussian Channels 8.1.1 Two Simple Examples 8.1.2 Gaussian Input with Rational Spectral Density 8.1.3 A Method of Youla, Kadota and Slepian 8.2 Noise and Input Signal with Rational Spectral Density 8.2.1 Again a Method by Slepian and Kadota Bibliography / Diese Arbeit untersucht die Erweiterung der bekannten kanonischen Korrelationsanalyse (canonical correlation analysis) für Zufallselemente auf abstrakten reellen messbaren Hilberträumen. Ein Schwerpunkt liegt dabei auf der Anwendung dieser Erweiterung zur Berechnung informationstheoretischer Größen auf endlichen Zeitintervallen. Analytische Ansätze für die Berechnung der Transinformation und der Informationsdichte zwischen gaußverteilten Zufallselementen auf beliebigen reelen messbaren Hilberträumen werden hergeleitet. Bezüglich der Transinformation sind die gewonnenen Resultate vergleichbar zu [4] und [1] (Baker, 1970, 1978). Sie können auch als Verallgemeinerung früherer Erkenntnisse aus [20] (Gelfand und Yaglom, 1958) aufgefasst werden. Zusätzlich erweitern einige der hergeleiteten Formeln zur Berechnung der Informationsdichte, ihrer charakteristischen Funktion und ihrer n-ten zentralen Momente Ergebnisse aus [45] und [44] (Pinsker, 1963, 1964). Weiterhin werden explizite Beispiele für die Berechnung der Transinformation, der charakteristischen Funktion der Informationsdichte sowie der n-ten zentralen Momente der Informationsdichte für den wichtigen Spezialfall eines additiven Gaußkanals mit gaußverteiltem Eingangssignal mit rationaler Spektraldichte erarbeitet, einerseits für gaußsches weißes Rauschen und andererseits für gaußsches Rauschen mit einer rationalen Spektraldichte. Diese Ergebnisse erweitern die entsprechenden konkreten Beispiele zur Berechnung der Transinformation aus [20] (Gelfand und Yaglom, 1958) sowie [28] und [29] (Huang und Johnson, 1963, 1962).:Kurzfassung Abstract Notations Abbreviations 1 Introduction 1.1 Software Used 2 Mathematical Background 2.1 Basic Notions of Measure and Probability Theory 2.1.1 Characteristic Functions 2.2 Stochastic Processes 2.2.1 The Consistency Theorem of Daniell and Kolmogorov 2.2.2 Second Order Random Processes 2.3 Some Properties of Fourier Transforms 2.4 Some Basic Inequalities 2.5 Some Fundamentals in Functional Analysis 2.5.1 Hilbert Spaces 2.5.2 Linear Operators on Hilbert Spaces 2.5.3 The Fréchet-Riesz Representation Theorem 2.5.4 Adjoint and Compact Operators 2.5.5 The Spectral Theorem for Compact Operators 3 Mutual Information and Information Density 3.1 Mutual Information 3.2 Information Density 4 Probability Measures on Hilbert Spaces 4.1 Measurable Hilbert Spaces 4.2 The Characteristic Functional 4.3 Mean Value and Covariance Operator 4.4 Gaussian Probability Measures on Hilbert Spaces 4.5 The Product of Two Measurable Hilbert Spaces 4.5.1 The Product Measure 4.5.2 Cross-Covariance Operator 5 Canonical Correlation Analysis on Hilbert Spaces 5.1 The Hellinger Distance and the Theorem of Kakutani 5.2 Canonical Correlation Analysis on Hilbert Spaces 5.3 The Theorem of Hájek and Feldman 6 Mutual Information and Information Density Between Gaussian Measures 6.1 A General Formula for Mutual Information and Information Density for Gaussian Random Elements 6.2 Hadamard’s Factorization Theorem 6.3 Closed Form Expressions for Mutual Information and Related Quantities 6.4 The Discrete-Time Case 6.5 The Continuous-Time Case 6.6 Approximation Error 7 Additive Gaussian Channels 7.1 Abstract Channel Model and General Definitions 7.2 Explicit Expressions for Mutual Information and Related Quantities 7.2.1 Gaussian Random Elements as Input to an Additive Gaussian Channel 8 Continuous-Time Gaussian Channels 8.1 White Gaussian Channels 8.1.1 Two Simple Examples 8.1.2 Gaussian Input with Rational Spectral Density 8.1.3 A Method of Youla, Kadota and Slepian 8.2 Noise and Input Signal with Rational Spectral Density 8.2.1 Again a Method by Slepian and Kadota Bibliography