<p>In this thesis, the theory of reinforcement learning is described and its relation to learning in biological systems is discussed. Some basic issues in reinforcement learning, the credit assignment problem and perceptual aliasing, are considered. The methods of temporal difference are described. Three important design issues are discussed: information representation and system architecture, rules for improving the behaviour and rules for the reward mechanisms. The use of local adaptive models in reinforcement learning is suggested and exemplified by some experiments. This idea is behind all the work presented in this thesis. A method for learning to predict the reward called the prediction matrix memory is presented. This structure is similar to the correlation matrix memory but differs in that it is not only able to generate responses to given stimuli but also to predict the rewards in reinforcement learning. The prediction matrix memory uses the channel representation, which is also described. A dynamic binary tree structure that uses the prediction matrix memories as local adaptive models is presented. The theory of canonical correlation is described and its relation to the generalized eigenproblem is discussed. It is argued that the directions of canonical correlations can be used as linear models in the input and output spaces respectively in order to represent input and output signals that are maximally correlated. It is also argued that this is a better representation in a response generating system than, for example, principal component analysis since the energy of the signals has nothing to do with their importance for the response generation. An iterative method for finding the canonical correlations is presented. Finally, the possibility of using the canonical correlation for response generation in a reinforcement learning system is indicated.</p>

<p>The work presented in this thesis is based on the basic idea of learning by reinforcement, within the theory of behaviorism. The reason for this choice is the generality of such an approach, especially that the reinforcement learning paradigm allows systems to be designed which can improve their behavior beyond that of their teacher. The role of the teacher is to define the reinforcement function, which acts as a description of the problem the machine is to solve.</p><p>Learning is considered to be a bootstrapping procedure. Fragmented past experience, of what to do when performing well, is used for response generation. The new response, in its turn, adds more information to the system about the environment. Gained knowledge is represented by a behavior probability density function. This density function is approximated with a number of normal distributions which are stored in the nodes of a binary tree. The tree structure is grown by applying a recursive algorithm to the stored stimuli-response combinations, called decisions. By considering both the response and the stimulus, the system is able to bring meaning to structures in the input signal. The recursive algorithm is first applied to the whole set of stored decisions. A mean decision vector and a covariance matrix are calculated and stored in the root node. The decision space is then partitioned into two halves across the direction of maximal data variation. This procedure is now repeated recursively for each of the two halves of the decision space, forming a binary tree with mean vectors and covariance matrices in its nodes.</p><p>The tree is the system's guide to response generation. Given a stimulus, the system searches for responses likely to result in highly reinforced decisions. This is accomplished by treating the sum of the normal distributions in the leaves as distribution describing the behavior of the system. The sum of normal distributions, with the current stimulus held fixed, is finally used for random generation of the response.</p><p>This procedure makes it possible for the system to have several equally plausible responses to one stimulus. Not applying maximum likelihood principles will make the system more explorative and reduce its risk of being trapped in local minima.</p><p>The performance and complexity of the learning tree is investigated and compared to some well known alternative methods. Presented are also some simple, yet principally important, experiments verifying the behavior of the proposed algorithm.</p>

<p>In this licenciate thesis, we discuss how to generate actions from percepts within an autonomous robotic system. In particular, we discuss and propose an original reactive architecture suitable for response generation, learning and self-organization.</p><p>The architecture uses incremental learning and supports self organization through distributed dynamic model generation and self-contained components. Signals to and from the architecture are represented using the channel representation, which is presented in that context.</p><p>The components of the architecture use a novel and flexible implementation of an artificial neural network. The learning rules for this implementation are derived.</p><p>A simulator is presented. It has been designed and implemented in order to test and evaluate the proposed architecture.</p><p>Results of a series of experiments on the reactive architecture are discussed and accounted for. The experiments have been performed within three different scenarios, using the developed simulator.</p><p>The problem of information representation in robotic architectures is illustrated by a problem of anchoring symbols to visual data. This is presented in the context of the WITAS project.</p>

<p>This thesis presents a number of methods to estimate 3D structures with a single translating camera. The camera is assumed to be calibrated and to have a known translation and rotation.</p><p>Applications for aircraft altitude estimation and ground structure estimation ahead of the aircraft are discussed. The idea is to mount a camera on the aircraft and use the motion estimates obtained in the inertia navigation system. One reason for this arrangement is to make the aircraft more passive, in comparison to conventional radar based altitude estimation.</p><p>Two groups of methods are considered, optical flow based and region tracking based. Both groups have advantages and drawbacks.</p><p>Two methods to estimate the optical flow are presented. The accuracy of the estimated ground structure is increased by varying the temporal distance between the frames used in the optical flow estimation algorithms.</p><p>Four region tracking algorithms are presented. Two of them use canonical correlation and the other two are based on sum of squared difference and complex correlation respectively.</p><p>The depth estimates are then temporally filtered using weighted least squares or a Kalman filter.</p><p>A simple estimation of the computational complexity and memory requirements for the algorithms is presented to aid estimation of the hardware requirements.</p><p>Tests on real flight sequences are performed, showing that the aircraft altitude can be estimated with a good accuracy.</p>

<p>It is hard to find another research field than functional magnetic resonance imaging (fMRI) that combines so many different areas of research. Without the beautiful physics of MRI we would not have any images to look at in the first place. To get images with good quality it is necessary to fully understand the concepts of the frequency domain. The analysis of fMRI data requires understanding of signal processing and statistics and also knowledge about the anatomy and function of the human brain. The resulting brain activity maps are used by physicians and neurologists in order to plan surgery and to increase their understanding of how the brain works.</p><p>This thesis presents methods for signal processing of fMRI data in real-time situations. Real-time fMRI puts higher demands on the signal processing, than conventional fMRI, since all the calculations have to be made in realtime and in more complex situations. The result from the real-time fMRI analysis can for example be used to look at the subjects brain activity in real-time, for interactive planning of surgery or understanding of brain functions. Another possibility is to use the result in order to change the stimulus that is given to the subject, such that the brain and the computer can work together to solve a given task. These kind of setups are often called brain computer interfaces (BCI).</p><p>Two BCI are presented in this thesis. In the first BCI the subject was able to balance a virtual inverted pendulum by thinking of activating the left or right hand or resting. In the second BCI the subject in the MR scanner was able to communicate with a person outside the MR scanner, through a communication interface.</p><p>Since head motion is common during fMRI experiments it is necessary to apply image registration to align the collected volumes. To do image registration in real-time can be a challenging task, therefore how to implement a volume registration algorithm on a graphics card is presented. The power of modern graphic cards can also be used to save time in the daily clinical work, an example of this is also given in the thesis.</p><p>Finally a method for calculating and incorporating a structural based certainty in the analysis of the fMRI data is proposed. The results show that the structural certainty helps to remove false activity that can occur due to head motion, especially at the edge of the brain.</p>

<p>In this thesis, novel methods for estimation of orientation and velocity are presented. The methods are designed exclusively in the spatial domain.</p><p>Two important concepts in the use of the spatial domain for signal processing is projections into subspaces, e.g. the subspace of second degree polynomials, and representations by frames, e.g. wavelets. It is shown how these concepts can be unified in a least squares framework for representation of finite dimensional vectors by bases, frames, subspace bases, and subspace frames.</p><p>This framework is used to give a new derivation of Normalized Convolution, a method for signal analysis that takes uncertainty in signal values into account and also allows for spatial localization of the analysis functions.</p><p>With the help of Normalized Convolution, a novel method for orientation estimation is developed. The method is based on projection onto second degree polynomials and the estimates are represented by orientation tensors. A new concept for orientation representation, orientation functionals, is introduced and it is shown that orientation tensors can be considered a special case of this representation. A very efficient implementation of the estimation method is presented and by evaluation on a test sequence it is demonstrated that the method performs excellently.</p><p>Considering an image sequence as a spatiotemporal volume, velocity can be estimated from the orientations present in the volume. Two novel methods for velocity estimation are presented, with the common idea to combine the orientation tensors over some region for estimation of the velocity fkield according to a motion model, e.g. affine motion. The first method involves a simultaneous segmentation and velocity estimation algorithm to obtain appropriate regions. The second method is designed for computational efficiency and uses local neighborhoods instead of trying to obtain regions with coherent motion. By evaluation on the Yosemite sequence, it is shown that both methods give substantially more accurate results than previously published methods.</p>

<p>This thesis presents a framework for estimation of motion fields both for single and multiple layers. All the methods have in common that they generate or use constraints on the local motion. Motion constraints are represented by vectors whose directions describe one component of the local motion and whose magnitude indicate confidence.</p><p>Two novel methods for estimating these motion constraints are presented. Both methods take two images as input and apply orientation sensitive quadrature filters. One method is similar to a gradient method applied on the phase from the complex filter outputs. The other method is based on novel results using canonical correlation presented in this thesis.</p><p>Parametric models, e.g. affine or FEM, are used to estimate motion from constraints on local motion. In order to estimate smooth fields for models with many parameters, cost functions on deformations are introduced.</p><p>Motions of transparent multiple layers are estimated by implicit or explicit clustering of motion constraints into groups. General issues and difficulties in analysis of multiple motions are described. An extension of the known EM algorithm is presented together with experimental results on multiple transparent layers with affine motions. Good accuracy in estimation allows reconstruction of layers using a backprojection algorithm. As an alternative to the EM algorithm, this thesis also introduces a method based on higher order tensors.</p><p>A result with potential applicatications in a number of diffeerent research fields is the extension of canonical correlation to handle complex variables. Correlation is maximized using a novel method that can handle singular covariance matrices.</p>

<p>This Master's Thesis addresses the problem of segmenting an image sequence with respect to the motion in the sequence. As a basis for the motion estimation, 3D orientation tensors are used. The goal of the segmentation is to partition the images into regions, characterized by having a coherent motion. The motion model is affine with respect to the image coordinates. A method to estimate the parameters of the motion model from the orientation tensors in a region is presented. This method can also be generalized to a large class of motion models.</p><p>Two segmentation algorithms are presented together with a postprocessing algorithm. All these algorithms are based on the competitive algorithm, a general method for distributing points between a number of regions, without relying on arbitrary threshold values. The first segmentation algorithm segments each image independently, while the second algorithm recursively takes advantage of the previous segmentation. The postprocessing algorithm stabilizes the segmentations of a whole sequence by imposing continuity constraints.</p><p>The algorithms have been implemented and the results of applying them to a test sequence are presented. Interesting properties of the algorithms are that they are robust to the aperture problem and that they do not require a dense velocity ¯eld.</p><p>It is finally discussed how the algorithms can be developed and improved. It is straightforward to extend the algorithms to base the segmentations on alternative or additional features, under not too restrictive conditions on the features.</p>

<p>To find a shape in an image, a technique called snakes or active contours can be used. An active contour is a curve that moves towards the sought-for shape in a way controlled by internal forces - such as rigidity and elasticity - and an image force. The image force should attract the contour to certain features, such as edges, in the image. This is done by creating an attractor image, which defines how strongly each point in the image should attract the contour.</p><p>In this thesis the extension to contours (surfaces) in three dimensional images is studied. Methods of representation of the contour and computation of the internal forces are treated.</p><p>Also, a new way of creating the attractor image, using the orientation tensor to detect planar structure in 3D images, is studied. The new method is not generally superior to those already existing, but still has its uses in specific applications.</p><p>During the project, it turned out that the main problem of active contours in 3D images was instability due to strong internal forces overriding the influence of the attractor image. The problem was solved satisfactory by projecting the elasticity force on the contour’s tangent plane, which was approximated efficiently using sphere-fitting.</p>

<p>Chapter 2 describes the concept of canonical correlation. This you have to know about in order to understand the continuing discussion.</p><p>Chapter 3 introduce you to the problem that was to be solved.</p><p>Chapter 4, 5 and 6 discusses three different suggestions how to approach the problem. Each chapter begins with a section of experiments as a motivation of the approach. Then follows some theory and mathematical manipulations to structure the thoughts. The last sections contains discussions and suggestions concerning the approach.</p><p>Finally chapter 7 contains a summary and a comparismental discussion of the approaches.</p><p> </p>