Autonomous Driving or self driving vehicles are concepts of vehicles knowing their environment and making driving manoeuvres without instructions from a driver. The concepts have been around for decades but has improved significantly in the last years since research in this area has made significant progress. Benefits of autonomous driving include the possibility to decrease the number of accidents in traffic and thereby saving lives. A major challenge in autonomous driving is to acquire 3D information and relations between all objects in surrounding traffic. This is referred to as \textit{spatial perception}. Stereo camera systems have become a central sensor module for advanced driver assistance systems and autonomous driving. For object detection and measurements at large distances stereo vision encounter difficulties. This includes objects being small, having low contrast and the presence of image noise. Having an accurate perception of the environment at large distances is however of high interest for many applications, especially autonomous driving. This thesis proposes a method which tries to increase the range to where generic objects are first detected using a given stereo camera setup. Objects are represented by planes in 3D space. The input image is segmented into the various objects and the 3D plane parameters are estimated jointly. The 3D plane parameters are estimated directly from the stereo image pairs. In particular, this thesis investigates methods to introduce geometric constraints to the segmentation or labeling task, i.e assigning each considered pixel in the image to a plane. The methods provided in this thesis show that despite the difficulties at large distances it is possible to exploit planar primitives in 3D space for obstacle detection at distances where other methods fail. / En autonom bil innebär att bilen har en uppfattning om sin omgivning och kan utifran det ta beslut angående hur bilen ska manövreras. Konceptet med självkörande bilar har existerat i årtionden men har utvecklats snabbt senaste åren sedan billigare datorkraft finns lättare tillgänglig. Fördelar med autonomiska bilar innebär bland annat att antalet olyckor i trafiken minskas och därmed liv räddas. En av de största utmaningarna med autonoma bilar är att få 3D information och relationer mellan objekt som finns i den omgivande trafikmiljön. Detta kallas för spatial perception och innebär att detektera alla objekt och tilldela en korrekt postition till dem. Stereo kamerasystem har fått en central roll för avancerade förarsystem och autonoma bilar. För detektion av objekt på stora avstånd träffar stereo system på svårigheter. Detta inkluderar väldigt små objekt, låg kontrast och närvaron av brus i bilden. Att ha en ackurativ perception på stora avstånd är dock vitalt för många applikationer, inte minst autonoma bilar. Den här rapporten föreslar en metod som försöker öka avståndet till där objekt först upptäcks. Objekt representeras av plan i 3D rymden. Bilder givna från stereo par segmenteras i olika object och plan parametrar estimeras samtidigt. Planens parametrar estimeras direkt från stereo bild paren. Den här rapporten utreder metoder att introducera gemoetriska begränsningar att använda vid segmenteringsuppgiften. Metoderna som presenteras i denna rapport visar att trots den höga närvaron av brus på stora avstånd är det möjligt att estimera geometriska objekt som är starka nog att möjliggöra detektion av objekt på ett avstand där andra metoder misslyckas.

Networks of cameras such as building security systems can be a source of localization information for a mobile robot assuming a map of camera locations as well as calibration information for each camera is available. This thesis describes an automated system to acquire such information. A fully automated camera calibration system uses fiducial markers and a mobile robot in order to drastically improve ease-of-use compared to standard techniques. A 6DOF EKF is used for mapping and is validated experimentally over a 50 m hallway environment. Motion planning strategies are considered both in front of a single camera to maximize calibration accuracy and globally between cameras in order to facilitate accurate measurements. For global motion planning, an adaptive exploration strategy based on heuristic search allows compromise between distance traveled and final map uncertainty which provides the system a level of autonomy which could not be obtained with previous techniques.

The application of hyperspectral imaging technique in the wavelength range of 400-1000 nm to estimate some of the maturity parameters of mangoes was investigated. Mangoes with different quality levels were grouped using principle component analysis (PCA). Feature wavelengths were identified to predict total soluble solids content, water content and firmness using simple correlation, first derivative, partial least square (PLS) regression analysis and measured values. Calibration models were developed using the selected wavelengths from correlation coefficients, first derivative, partial least square (PLS) regression analysis and corresponding maturity parameters employing artificial neural network model to predict total soluble solids content, water content and firmness of the fruit. Performance of the models was compared using the correlation coefficient (r) values. Fruit firmness was predicted with high correlation coefficient (r=0.88) followed by water content (r=0.81) and total soluble solids (r=0.78) using wavelengths selected from simple correlation of first derivative data with the parameters and ANN model. The results of the study demonstrated the scope for further research on maturity and quality evaluation of fruits using hyperspectral imaging technique.

This thesis explores the use of physics based human motion models in the context of video-based human motion estimation and scene analysis. Two abstract models of human locomotion are described and used as the basis for video-based estimation. These models demonstrate the power of physics based models to provide meaningful cues for estimation without the use of motion capture data. However promising, the abstract nature of these models limit the range of motion they can faithfully capture. A more detailed model of human motion and ground interaction is also described. This model is used to estimate the ground surface which a subject interacts with, the forces driving the motion and, finally, to smooth corrupted motions from existing trackers in a physically realistic fashion. This thesis suggests that one of the key difficulties in using physical models is the discontinuous nature of contact and collisions. Two different approaches to handling ground contacts are demonstrated,one using explicit detection and collision resolution and the other using a continuous approximation. This difficulty also distinguishes the models used here from others used in other areas which often sidestep the issue of collisions.

Large-scale earthmoving activities make up a costly and air-polluting aspect of many construction projects and mining operations, which depend entirely on the use of heavy construction equipment. The long-term jobsites and manufacturing nature of the mining sector has encouraged the application of automated controlling systems, more specifically GPS, to control the earthmoving fleet. Computer vision-based methods are another potential tool to provide real-time information at low-cost and to reduce human error in surface earthmoving sites as relatively clear views can be selected and the equipment offer recognizable targets. Vision-based methods have some advantages over positioning devices as they are not intrusive, provide detailed data about the behaviour of each piece of equipment, and offer reliable documentation for future reviews. This dissertation explains the development of a vision-based system, named server-customer interaction planner (SCIT), to recognize and estimate earth material loading cycles. The SCIT system consists of three main modules: object recognition, tracking, and action recognition. Different object recognition and tracking algorithms were evaluated and modified, and then the ideal methods were used to develop the object recognition and tracking modules. A novel hybrid tracking framework was developed for the SCIT system to track dump trucks in the challenging views found in the loading zones. The object recognition and tracking engines provide spatiotemporal data about the equipment which are then analyzed by the action recognition module to estimate loading cycles. The entire framework was evaluated using videos taken under varying conditions. The results highlight the promising performance of the SCIT system with the hybrid tracking engine, thereby validating the possibility of its practical application.

This thesis introduces principled, broadly applicable, and efficient patch-based models for data processing applications. Recently, "epitomes" were introduced as patch-based probability models that are learned by compiling together a large number of examples of patches from input images. This thesis describes how epitomes can be used to model video data and a significant computational speedup is introduced that can be incorporated into the epitome inference and learning algorithm. In the case of videos, epitomes are estimated so as to model most of the small space-time cubes from the input data. Then, the epitome can be used for various modelling and reconstruction tasks, of which we show results for video super-resolution, video interpolation, and object removal. Besides computational efficiency, an interesting advantage of the epitome as a representation is that it can be reliably estimated even from videos with large amounts of missing data. This ability is illustrated on the task of reconstructing the dropped frames in a video broadcast using only the degraded video. Further, a new patch-based model is introduced, that when applied to epitomes, accounts for the varying geometric configurations of object features. The power of this model is illustrated on tasks such as multiple object registration and detection and missing data interpolation, including a difficult task of photograph relighting.

A critical practical problem in the field of object recognition is an insufficient number of labeled training images, as manually labeling images is a time consuming task. For this reason, unsupervised learning techniques are used to take advantage of unlabeled training images to extract image representations that are useful for classification. However, unsupervised learning is in general difficult. We propose simplifying the unsupervised training problem considerably by taking the advance of motion information. The output of our method is a model that can generate a vector representation from any static image. However, the model is trained using images with additional motion information. To demonstrate the flobject analysis framework, we extend the latent Dirichlet allocation model to account for word-specific flow vectors. We show that the static image representations extracted using our model achieve higher classification rates and better generalization than standard topic models, spatial pyramid matching, and Gist descriptors.

This thesis explores the use of physics based human motion models in the context of video-based human motion estimation and scene analysis. Two abstract models of human locomotion are described and used as the basis for video-based estimation. These models demonstrate the power of physics based models to provide meaningful cues for estimation without the use of motion capture data. However promising, the abstract nature of these models limit the range of motion they can faithfully capture. A more detailed model of human motion and ground interaction is also described. This model is used to estimate the ground surface which a subject interacts with, the forces driving the motion and, finally, to smooth corrupted motions from existing trackers in a physically realistic fashion. This thesis suggests that one of the key difficulties in using physical models is the discontinuous nature of contact and collisions. Two different approaches to handling ground contacts are demonstrated,one using explicit detection and collision resolution and the other using a continuous approximation. This difficulty also distinguishes the models used here from others used in other areas which often sidestep the issue of collisions.

This thesis introduces principled, broadly applicable, and efficient patch-based models for data processing applications. Recently, "epitomes" were introduced as patch-based probability models that are learned by compiling together a large number of examples of patches from input images. This thesis describes how epitomes can be used to model video data and a significant computational speedup is introduced that can be incorporated into the epitome inference and learning algorithm. In the case of videos, epitomes are estimated so as to model most of the small space-time cubes from the input data. Then, the epitome can be used for various modelling and reconstruction tasks, of which we show results for video super-resolution, video interpolation, and object removal. Besides computational efficiency, an interesting advantage of the epitome as a representation is that it can be reliably estimated even from videos with large amounts of missing data. This ability is illustrated on the task of reconstructing the dropped frames in a video broadcast using only the degraded video. Further, a new patch-based model is introduced, that when applied to epitomes, accounts for the varying geometric configurations of object features. The power of this model is illustrated on tasks such as multiple object registration and detection and missing data interpolation, including a difficult task of photograph relighting.

Large-scale earthmoving activities make up a costly and air-polluting aspect of many construction projects and mining operations, which depend entirely on the use of heavy construction equipment. The long-term jobsites and manufacturing nature of the mining sector has encouraged the application of automated controlling systems, more specifically GPS, to control the earthmoving fleet. Computer vision-based methods are another potential tool to provide real-time information at low-cost and to reduce human error in surface earthmoving sites as relatively clear views can be selected and the equipment offer recognizable targets. Vision-based methods have some advantages over positioning devices as they are not intrusive, provide detailed data about the behaviour of each piece of equipment, and offer reliable documentation for future reviews. This dissertation explains the development of a vision-based system, named server-customer interaction planner (SCIT), to recognize and estimate earth material loading cycles. The SCIT system consists of three main modules: object recognition, tracking, and action recognition. Different object recognition and tracking algorithms were evaluated and modified, and then the ideal methods were used to develop the object recognition and tracking modules. A novel hybrid tracking framework was developed for the SCIT system to track dump trucks in the challenging views found in the loading zones. The object recognition and tracking engines provide spatiotemporal data about the equipment which are then analyzed by the action recognition module to estimate loading cycles. The entire framework was evaluated using videos taken under varying conditions. The results highlight the promising performance of the SCIT system with the hybrid tracking engine, thereby validating the possibility of its practical application.