Fault detection and diagnosis is quite important in engineering systems, and deserves further attention in view of the increasing complexity of modern machinery. Traditional single-sensor methods of fault monitoring and diagnosis may find it difficult to meet modern industrial requirements because there is usually no direct way to measure and accurately correlate a machine fault to a single sensor output. Fusion of information from multiple sensors can overcome this shortcoming. In this thesis, a neural-fuzzy approach of multi-sensor fusion is developed for a network-enabled remote fault diagnosis system. The approach is validated by applying it to an industrial machine called the Iron Butcher, which is a machine used in the fish processing industry for the removal of the head in fish prior to further processing for canning. An important characteristic of the fault diagnosis approach developed in this thesis is to make an accurate decision of the machine condition by fusing information from different sensors. First, sound, vibration and vision signals are acquired from the machine using a microphone, an accelerometer and a digital CCD camera, respectively. Second, the sound and vibration signals are transformed into the frequency domain using fast Fourier transformation (FFT). A feature vector from the FFT frequency spectra is defined and extracted from the acquired information. Also, a feature based vision tracking approach—the Scale Invariant Feature Transform (SIFT)—is applied to the vision data to track the object of interest (fish) in a robust manner. Third, Sound, vibration and vision feature vectors are provided as inputs to a neuro-fuzzy network for fault detection and diagnosis. A four-layer neural network including a fuzzy hidden layer is developed in the thesis to analyze and diagnose existing faults. By training the neural network with sample data for typical faults, faults of five crucial components in the fish cutting machine are detected with high reliability and robustness. Alarms to warn about impending faults may be generated as well during the machine operation. A network-based remote monitoring architecture is developed as well in the thesis, which will facilitate engineers to monitor the machine condition in a more flexible manner from a remote site. Developed multi-sensor approaches are validated using computer simulations and physical experimentation with the industrial machine, and compared with a single-sensor approach. / Applied Science, Faculty of / Mechanical Engineering, Department of / Graduate

A critical ability of any aircraft is to be able to detect potential collisions with other airborne objects, and maneuver to avoid these collisions. This can be done by utilizing sensors on the aircraft to monitor the sky for collision threats. However, several problems face a system which aims to use multiple sensors for target tracking. The data collected from sensors needs to be clustered, fused, and otherwise processed such that the flight control system can make accurate decisions based on it. Raw sensor data, while filled with useful information, is tainted with inaccuracies due to limitations and imperfections of the sensor. Combined use of different sensors presents further issues in how to handle disagreements between sensor data. This thesis project tackles the problem of processing data from multiple sensors (in this application, a radar and an infrared sensor) on an airborne platform in order to allow the aircraft to make flight corrections to avoid collisions.

The cocktail party problem introduced in 1953 describes the ability to focus auditory attention in a noisy environment epitomised by a cocktail party. An individual with normal hearing uses several cues to unmask talkers of interest, such cues often lacks for people with hearing loss. This thesis explores the possibility to use a pair of glasses equipped with an inertial measurement unit (IMU), monocular camera and eye tacker to estimate an auditory scene and estimate the attention of the person wearing the glasses. Three main areas of interest have been investigated: estimating head orientation of the user; track faces in the scene and determine talker of interest using gaze. Implemented on a hearing aid, this solution could be used to artificially unmask talkers in a noisy environment. The head orientation of the user has been estimated with an extended Kalman filter (\EKF) algorithm, with a constant velocity model and different sets of measurements: accelerometer; gyrosope; monocular visual odometry (MVO); gaze estimated bias (GEB). An intrinsic property of IMU sensors is a drift in yaw. A method using eye data and gyroscope measurements to estimate gyroscope bias has been investigated and is called GEB. The MVO methods investigated use either optical flow to track features in succeeding frames or a key frame approach to match features over multiple frames.Using estimated head orientation and face detection software, faces have been tracked since they can be assumed as regions of interest in a cocktail party environment. A constant position EKF with a nearest neighbour approach has been used for tracking. Further, eye data retrieved from the glasses has been analyzed to investigate the relation between gaze direction and current talker during conversations.

With the advent of the Global Position System (GPS), we now have the ability to determine absolute position anywhere on the globe. Although GPS systems work well in open environments with no overhead obstructions, they are subject to large unavoidable errors when the reception from some of the satellites is blocked. This occurs frequently in urban environments, such as downtown New York City. GPS systems require at least four satellites visible to maintain a good position 'fix' . Tall buildings and tunnels often block several, if not all, of the satellites. Additionally, due to Selective Availability (SA), where small amounts of error are intentionally introduced, GPS errors can typically range up to 100 ft or more. This thesis proposes several methods for improving the position estimation capabilities of a system by incorporating other sensor and data technologies, including Kalman filtered inertial navigation systems, rule-based and fuzzy-based sensor fusion techniques, and a unique map-matching algorithm. / Master of Science

Convolutional Neural Networks (CNNs) have been extensively studied and applied to various computer vision problems, including object detection, semantic segmentation, and autonomous driving. Convolutional Neural Networks (CNN)s extract complex features from input images or data to represent objects or patterns. Their highly complex architecture, however, and the size of their learned weights make their time and resource intensive. Measures like pruning and fusion, which aim to simplify the structure and lessen the load on the network’s resources, should be considered to resolve this problem. In this thesis, we intend to explore the effect of pruning on segmentation and object detection as well as the benefits of using sensor fusion operators in the 2d space to boost the existing networks’ performance. Specifically, we focus on structured pruning, quantization, and simple and learnable fusion operators. We also study the scalability of different algorithms in terms of the number of parameters and floating points used. First, we provide a general overview of CNNs and the history of pruning and fusion operations. Second, we explain the advantages of pruning and discuss the contrast between the unstructured and structured types. Third, we discuss the differences between simple fusion and learnable fusion. In order to evaluate our algorithms, we use several classification and object detection datasets such as Cifar-10, KITTI and Microsoft COCO. By applying our proposed methods to the studied datasets, we can assess the efficiency of the algorithms. Furthermore, this allows us to observe the improvements in task-specific losses. In conclusion, our work is focused on analyzing the effect of pruning and fusion to simplify existing networks and improve their performance in terms of scalability, task-specific losses, and resource consumption. We also discuss various algorithms, as well as datasets which serve as a basis for the evaluation of our proposed approaches.

The following thesis addresses the problem of localizing an outdoor mobile robot and mapping the environment using the state of the art of consumer grade RTK GPS. The thesis investigates limitations and possibilities for sensor fusion to increase reliability and usability. The main subject of research is a robotic lawn mower from Husqvarna, the Automower 430x, connected to existing hardware on the product with an auxiliary real time kinematic global positioning system, the Emlid Reach. The test conducted showed that the auxiliary RTK GPS module is currently unsatisfactory as sole absolute position sensor for the Automower platform, mainly due to inconsistent performance. This thesis is meant as a preliminary study for future use of GNSS sensors for outdoor mobile robots and as a suggestive study of the current performance of the increasingly popular Emlid Reach GPS module.

Connected vehicles is a growing market. There are currently several such services available, but many of them are constrained in the sense that they are bound to recently produced cars and either expensive or strongly limited in the services that they provide. In this master thesis we investigate the possibility to implement a generic platform that is of low cost and simple to install in any vehicle, but that still has the ability to provide a wide range of services. It is proposed that a crucial step in such a system is to reconstruct the vehicle’s kinematics, as this enables the possibility to developed a wide range of services by feature extraction and interpret the result from a dynamics perspective. A mathematical model that describes how the kinematics can be reconstructed is proposed, and a filter that performs such reconstruction is implemented. Based on this reconstruction, two filters that interpret the output are implemented as a proof of concept for the proposed mathematical model. The complete implemented filter solution is tested on measurement data from actual driving scenarios and it is seen that we can identify when the vehicle makes a hard turn, and find where the surrounding road conditions are poor. / Uppkopplade fordon är en växande marknad. I dagsläget finns flera sådana tjänster, men ofta är dessa begränsade i den meningen att de antingen endast finns tillgängliga för nyproducerade fordon eller bara erbjuder ett smalt utbud av tjänster. I detta examensarbete undersöker vi möjligheten att utveckla en generisk plattform för uppkopplade fordon som är billig och enkel att installera, men som också kan erbjuda ett stort urval av tjänster. Det föreslås att ett viktigt steg i en sådan lösning är att rekonstruera fordonets kinematik, då detta möjliggör utvecklandet av ett brett urval av tjänster genom att identifiera karakteristiska egenskaper i kinematiken, samt göra tolkningar utifrån dynamikbetraktelser. En matematisk modell för att beskriva hur kinematiken kan rekonstrueras från givna indata presenteras, och ett filter som utför denna rekonstruktion implementeras. Ytterligare två filter implementeras för att påvisa att den rekonstruerade kinematiken samt den föreslagna matematiska modellen kan användas till att identifiera olika scenarion ur verkligheten. Den kompletta filterlösningen testas på mätdata från faktiska körningar och vi ser att vi kan identifiera när fordonet gör skarpa svängar, samt när vägförhållandena är dåliga.

Recording a video sequence with a camera during movement often produces blurred results. This is mainly due to motion blur which is caused by rapid movement of objects in the scene or the camera during recording. By correcting for changes in the orientation of the camera, caused by e.g. uneven terrain, it is possible to minimize the motion blur and thus, produce a stabilized video. In order to do this, data gathered from a gyroscope and the camera itself can be used to measure the orientation of the camera. The raw data needs to be processed, synchronized and filtered to produce a robust estimate of the orientation. This estimate can then be used as input to some automatic control system in order to correct for changes in the orientation This thesis focuses on examining the possibility of such a stabilization. The actual stabilization is left for future work. An evaluation of the hardware as well as the implemented methods are done with emphasis on speed, which is crucial in real time computing. / En videosekvens som spelas in under rörelse blir suddig. Detta beror främst på rörelseoskärpa i bildrutorna orsakade av snabb rörelse av objekt i scenen eller av kameran själv. Genom att kompensera för ändringar i kamerans orientering, orsakade av t.ex. ojämn terräng, är det möjligt att minimera rörelseoskärpan och på så sätt stabilisera videon. För att åstadkomma detta används data från ett gyroskop och kameran i sig för att skatta kamerans orientering. Den insamlade datan behandlas, synkroniseras och filtreras för att få en robust skattning av orienteringen. Denna orientering kan sedan användas som insignal till ett reglersystem för att kompensera för ändringar i kamerans orientering. Denna avhandling undersöker möjligheten för en sådan stabilisering. Den faktiska stabiliseringen lämnas till framtida arbete. Hårdvaran och de implementerade metoderna utvärderas med fokus på beräkningshastighet, som är kritiskt inom realtidssystem.

Today’s heavy duty vehicles are equipped with safety and comfort systems, e.g. ABS and ESP, which totally or partly take over the vehicle in certain risk situations. When these systems become more and more autonomous more robust positioning is needed. In the right conditions the GPS system provides precise and robust positioning. However, in harsh environments, e.g. dense urban areas and in dense forests, the GPS signals may be affected by multipaths, which means that the signals are reflected on their way from the satellites to the receiver. This can cause large errors in the positioning and thus can give rise to devastating effects for autonomous systems. This thesis evaluate different methods to enhance a low cost GPS in harsh environments, with focus on mitigating multipaths. Mainly there are four different methods: Regular Unscented Kalman filter, probabilistic multipath mitigation, Unscented Kalman filter with vehicle sensor input and probabilistic multipath mitigation with vehicle sensor input. The algorithms will be tested and validated on real data from both dense forest areas and dense urban areas. The results show that the positioning is enhanced, in particular when integrating the vehicle sensors, compared to a low cost GPS.

Robôs móveis dependem de dados provenientes de sensores para ter uma representação do seu ambiente. Porém, os sensores geralmente fornecem informações incompletas, inconsistentes ou imprecisas. Técnicas de fusão de sensores têm sido empregadas com sucesso para aumentar a precisão de medidas obtidas com sensores. Este trabalho propõe e investiga o uso de técnicas de inteligência artificial para fusão de sensores com o objetivo de melhorar a precisão e acurácia de medidas de distância entre um robô e um objeto no seu ambiente de trabalho, obtidas com diferentes sensores. Vários algoritmos de aprendizado de máquina são investigados para fundir os dados dos sensores. O melhor modelo gerado com cada algoritmo é chamado de estimador. Neste trabalho, é mostrado que a utilização de estimadores pode melhorar significativamente a performance alcançada por cada sensor isoladamente. Mas os vários algoritmos de aprendizado de máquina empregados têm diferentes características, fazendo com que os estimadores tenham diferentes comportamentos em diferentes situações. Objetivando atingir um comportamento mais preciso e confiável, os estimadores são combinados em comitês. Os resultados obtidos sugerem que essa combinação pode melhorar a confiança e precisão das medidas de distâncias dos sensores individuais e estimadores usados para fusão de sensores. / Mobile robots rely on sensor data to have a representation of their environment. However, the sensors usually provide incomplete, inconsistent or inaccurate information. Sensor fusion has been successfully employed to enhance the accuracy of sensor measures. This work proposes and investigates the use of artificial intelligence techniques for sensor fusion. Its main goal is to improve the accuracy and reliability of a distance between a robot and an object in its work environment using measures obtained from different sensors. Several machine learning algorithms are investigated to fuse the sensors data. The best model generated with each algorithm are called estimator. It is shown that the employment of the estimators based on artificial intelligence can improve significantly the performance achieved by each sensor alone. The machine learning algorithms employed have different characteristics, causing the estimators to have different behaviour in different situations. Aiming to achieve more accurate and reliable behavior, the estimators are combined in committees. The results obtained suggest that this combination can improve the reliability and accuracy of the distance measures by the individual sensors and estimators used for sensor fusion.