Longline fishing is a method of fishing that utilizes baited hooks to catch fish in an environmentally friendly way. In order to reduce the number of catch lost while longline fishing, it is of great interest to be able to keep an even tension on the fishing line. This can be done by estimating the speed at the point of interest (POI) at which the fishing line is attached to the boat. Due to the harsh conditionson the seas, it is not recommended to put any sensors directly at that point. The aim of this thesis was to explore whether or not it is possible to estimate the vertical speed at the POI by having sensors measuring linear acceleration and angular velocity at various unknown places in the boat. The sensors were placed at various places in a simulated boat, after which the sensor orientations and positions were calculated using a nonlinear Least Squares method. After the sensors were positioned, an Extended Kalman Filter (EKF) was implemented on each sensor, after which the speed of the POI was calculated as the fused estimate of all EKFs. By changing the number of sensors and their sampling times, the best compromise between accuracy, computational load and number of sensors was found. The results prove that it is fully possible to estimate the vertical speed of the POI using only four 6DOF IMU:s using a sampling time of 50 or 100 ms, depending on how accurate the user wants the estimated positions of the sensors to be. However, there are still many ways in which the method used can be improved to geta better estimate.

For complex satellite missions that rely on agile and high-precision manoeuvres, the low-friction aspect of the space environment is a critical component in understanding the attitude control dynamics of the spacecraft. The Kinesthetic Node and Autonomous Table-Top Emulator (KNATTE) is a three-degree-of-freedom frictionless vehicle that serves as the foundation of a multipurpose platform for real-time spacecraft hardware-in-the-loop experiments, and allows emulation of these conditions in two dimensions with the purpose of validating various guidance, navigation, and control algorithms. The data acquisition of the vehicle depends on a computer vision system (CVS) that yields position and attitude data, but also suffers from unpredictable blackout events. To complement such measurements, KNATTE incorporates an inertial measurement unit (IMU) that yields accelerometer, gyroscope, and magnetometer data. This study describes a multisensor data fusion approach to obtain accurate attitude information by combining the measurements from the CVS and the IMU using nonlinear Kalman filter algorithms. To do this, the data fusion algorithms are developed and tested in a Matlab/Simulink environment. After that, the algorithms are adapted to the KNATTE platform and their performance is confirmed in various conditions. Through this work, the accuracy and efficiency of the approach can be checked by numerical simulation and real-time experiments. In addition, the quality of the CVS measurements are further improved by the introduction of a neural network to the image processing pipeline of the original system.

In this thesis we were interested in combining functional connectivity (from functional Magnetic Resonance Imaging) and structural connectivity (from Diffusion Tensor Imaging) with a data fusion approach. While data fusion approaches provide an abundance of information they are underutilized due to their complexity. To solve this problem, we integrated the ease of a neuroimaging toolbox, known as the Functional And Tractographic Analysis Toolbox (FATCAT) with a data fusion approach known as the anatomically weighted functional connectivity (awFC) approach - to produce a practical and more efficient pipeline. We studied the connectivity within resting-state networks of different populations using this novel pipeline. We performed separate analyses with traditional structural and functional connectivity for comparison with the awFC findings - across all three projects. In the first study we evaluated the awFC of participants with major depressive disorder compared to controls. We observed significant connectivity differences in the default mode network (DMN) and the ventral attention network (VAN). In the second study we studied the awFC of MDD remitters compared to non-remitters at baseline and week-8 (post antidepressant), and evaluated awFC in remitters longitudinally from baseline to to week-8. We found significant group differences in the DMN, VAN, and frontoparietal network (FPN) for remitters and non-remitters at week-8. We also found significant awFC longitudinally from baseline to week-8 in the dorsal attention network (DAN) and FPN. We also tested the associations between connectivity strength and cognition. In the third study we studied the awFC in children exposed to pre- and postnatal adversity compared to controls. We observed significant differences in the DMN, FPN, VAN, DAN, and limbic network (LIM). We also assessed the association between connectivity strength in middle childhood and motor and behavioural scores at age 3. Therefore, the FATCAT-awFC pipeline, we designed was capable of identifying group differences in RSN in a practical and more efficient manner. / Thesis / Doctor of Philosophy (PhD)

No description available.

Maintaining the quality of indoor environments in educational facilities is crucial for student comfort, health, well-being, and learning performance. Amidst the growing recognition of the impact of indoor environmental conditions on occupant comfort, health, and well-being, there has been an increasing focus on the assessment and modeling of Indoor Environmental Quality (IEQ). Despite considerable advancements, current IEQ modeling and assessment methodologies often prioritize and limit to singular comfort metrics, potentially neglect- ing the comprehensive and holistic factors associated with occupant comfort and health. Furthermore, existing indoor environment maintenance practices and building systems for educational facilities often fail to include feedback from occupants (e.g., students and fac- ulty) and exhibit limited adaptability to their needs. This calls for more inclusive and occupant-centric IEQ assessment models that cover a broader spectrum of environmental parameters and occupant needs. To address the gaps, this thesis proposes a novel Artificial Intelligence of Things (AIoT)-based IEQ assessment framework that bridges gaps by uti- lizing multimodal data fusion and deep learning-based prediction and classification models. These models are developed to utilize real-time multidimensional IEQ data, non-intrusive occupant feedback (MFCC features from audio recordings, video/thermal features extracted by Vision Transformer (ViT)), and self-reported comfort and health levels, placing a focus on occupant-centric and data-driven decision-making for intelligent educational facilities. The proposed framework was evaluated and validated at Virginia Tech Blacksburg campus, achieving a 91.9% in R2 score in predicting future IEQ conditions and 97% and 96% accuracy in comfort and health-based IEQ conditions classifications. / Master of Science

Systems biology aims to understand cellular behavior in terms of the spatiotemporal interactions among cellular components, such as genes, proteins and metabolites. Comprehensive visualization tools for exploring multivariate data are needed to gain insight into the physiological processes reflected in these molecular profiles. Data fusion methods are required to integratively study high-throughput transcriptomics, metabolomics and proteomics data combined before systems biology can live up to its potential. In this work I explored mathematical and statistical methods and visualization tools to resolve the prominent issues in the nature of systems biology data fusion and to gain insight into these comprehensive data. In order to choose and apply multivariate methods, it is important to know the distribution of the experimental data. Chi square Q-Q plot and violin plot were applied to all M. truncatula data and V. vinifera data, and found most distributions are right-skewed (Chapter 2). The biplot display provides an effective tool for reducing the dimensionality of the systems biological data and displaying the molecules and time points jointly on the same plot. Biplot of M. truncatula data revealed the overall system behavior, including unidentified compounds of interest and the dynamics of the highly responsive molecules (Chapter 3). The phase spectrum computed from the Fast Fourier transform of the time course data has been found to play more important roles than amplitude in the signal reconstruction. Phase spectrum analyses on in silico data created with two artificial biochemical networks, the Claytor model and the AB2 model proved that phase spectrum is indeed an effective tool in system biological data fusion despite the data heterogeneity (Chapter 4). The difference between data integration and data fusion are further discussed. Biplot analysis of scaled data were applied to integrate transcriptome, metabolome and proteome data from the V. vinifera project. Phase spectrum combined with k-means clustering was used in integrative analyses of transcriptome and metabolome of the M. truncatula yeast elicitation data and of transcriptome, metabolome and proteome of V. vinifera salinity stress data. The phase spectrum analysis was compared with the biplot display as effective tools in data fusion (Chapter 5). The results suggest that phase spectrum may perform better than the biplot. This work was funded by the National Science Foundation Plant Genome Program, grant DBI-0109732, and by the Virginia Bioinformatics Institute. / Ph. D.

Tracking human hand motions has raised more attention due to the recent advancements of virtual reality (Rheingold, 1991) and prosthesis control (Antfolk et al., 2010). Surface electromyography (sEMG) has been the predominant method for sensing electrical activity in biomechanical studies, and has also been applied to motion tracking in recent years. While most studies focus on the classification of human hand motions within a predefined motion set, the prediction of continuous finger joint angles and wrist angles remains a challenging endeavor. In this research, a biomechanical knowledge-driven data fusion strategy is proposed to predict finger joint angles and wrist angles. This strategy combines time series data of sEMG signals and simulated muscle features, which can be extracted from a biomechanical model available in OpenSim (Delp et al., 2007). A support vector regression (SVR) model is used to firstly predict muscle features from sEMG signals and then to predict joint angles from the estimated muscle features. A set of motion data containing 10 types of motions from 12 participants was collected from an institutional review board approved experiment. A hypothesis was tested to validate whether adding the simulated muscle features would significantly improve the prediction performance. The study indicates that the biomechanical knowledge-driven data fusion strategy will improve the prediction of new types of human hand motions. The results indicate that the proposed strategy significantly outperforms the benchmark date-driven model especially when the users were performing unknown types of motions from the model training stage. The proposed model provides a possible approach to integrate the simulation models and data fusion models in human factors and ergonomics. / Master of Science

Many countries around the world are undergoing massive urbanization campaigns at an unprecedented rate, heralded by promises of economical prosperity and bolstered population health and well-being. Projections indicate that by 2050, nearly 68% of the world populace will reside in these urban environments. However, rapid growth at such an exceptional scale poses unique challenges pertaining to environmental quality and food production, which can negate the effectiveness of the aforementioned boons. As such, there is an emphasis on mitigating these negative effects through the construction of smart and connected communities (S&CC), which integrate both artificial intelligence (AI) and the Internet of Things (IoT). This coupling of intelligent technologies also poses interesting system design challenges pertaining to the fusion of the diverse, heterogeneous datasets available to IoT environments, and the ability to learn multiple S&CC problem sets concurrently. Attention-based Transformer networks are of particular interest given their success across diverse fields of natural language processing (NLP), computer vision, time-series regression, and multi-modal data fusion in recent years. This begs the question whether Transformers can be further diversified to leverage fusions of IoT data sources for heterogeneous multi-task learning in S&CC trade spaces. This is a fundamental question that this thesis seeks to answer. Indeed, the key contribution of this thesis is the design and application of Transformer networks for developing AI systems in emerging smart cities. This is executed within a collaborative U.S.-Indonesia effort between Virginia Tech, the University of Colorado Boulder, the Universitas Gadjah Mada, and the Institut Teknologi Bandung with the goal of growing smart and sustainable garden alleys in Makassar City, Indonesia. Specifically, a proof-of-concept AI nerve-center is proposed using a backbone of pure-encoder Transformer architectures to learn a diverse set of tasks such as multivariate time-series regression, visual plant disease classification, and image-time-series fusion. To facilitate the data fusion tasks, an effective algorithm is also proposed to synthesize heterogeneous feature sets, such as multivariate time-series and time-correlated images. Moreover, a hyperparameter tuning framework is also proposed to standardize and automate model training regimes. Extensive experimentation shows that the proposed Transformer-based systems can handle various input data types via custom sequence embedding techniques, and are naturally suited to learning a diverse set of tasks. Further, the results also show that multi-task learners increase both memory and computational efficiency while maintaining comparable performance to both single-task variants, and non-Transformer baselines. This demonstrates the flexibility of Transformer networks to learn from a fusion of IoT data sources, their applicability in S&CC trade spaces, and their further potential for deployment on edge computing devices. / Master of Science / Many countries around the world are undergoing massive urbanization campaigns at an unprecedented rate, heralded by promises of economical prosperity and bolstered population health and well-being. Projections indicate that by 2050, nearly 68% of the world populace will reside in these urban environments. However, rapid growth at such an exceptional scale poses unique environmental and food cultivation challenges. Hence, there is a focus on reducing these negative effects through building smart and connected communities (S&CC). The term connected is derived from the integration of small, low-cost devices which gather information from the surrounding environment, called the Internet of Things (IoT). Likewise, smart is a term derived from the integration of artificial intelligence (AI), which is used to make informed decisions based on IoT-collected information. This coupling of intelligent technologies also poses its own unique challenges pertaining to the blending of IoT data with highly diverse characteristics. Of specific interest is the design of AI models that can not only learn from a fusion of this diverse information, but also learn to perform multiple tasks in parallel. Attention-based networks are a relatively new category of AI which learn to focus on, or attend to, the most important portions of an arbitrary data sequence. Transformers are AI models which are designed using attention as their backbone, and have been employed to much success in many fields in recent years. This success begs the question whether Transformers can be further extended to put the smart in S&CC. The overarching goal of this thesis is to design and implement a Transformer-based AI system for emerging smart cities. In particular, this is accomplished within a U.S.-Indonesia collaborative effort with the goal of growing smart and sustainable garden alleys in Makassar City, Indonesia.

Mit der vorliegenden Dissertation soll ein Beitrag zur Steigerung der Genauigkeit und Zuverlässigkeit einer sensorgestützten Objekterkennung im Fahrzeugumfeld geleistet werden. Aufbauend auf einem Erfassungssystem, bestehend aus einer Stereokamera und einem Mehrzeilen-Laserscanner, werden teils neu entwickelte Verfahren für die gesamte Verarbeitungskette vorgestellt. Zusätzlich wird ein neuartiges Framework zur Fusion heterogener Sensordaten eingeführt, welches über eine Zusammenführung der Fusionsergebnisse aus den unterschiedlichen Verarbeitungsebenen in der Lage ist, die Objektbestimmung zu verbessern. Nach einer Beschreibung des verwendeten Sensoraufbaus werden die entwickelten Verfahren zur Kalibrierung des Sensorpaares vorgestellt. Bei der Segmentierung der räumlichen Punktdaten werden bestehende Verfahren durch die Einbeziehung von Messgenauigkeit und Messspezifik des Sensors erweitert. In der anschließenden Objektverfolgung wird neben einem neuartigen berechnungsoptimierten Ansatz zur Objektassoziierung ein Modell zur adaptiven Referenzpunktbestimmung und –Verfolgung beschrieben. Durch das vorgestellte Fusions-Framework ist es möglich, die Sensordaten wahlweise auf drei unterschiedlichen Verarbeitungsebenen (Punkt-, Objekt- und Track-Ebene) zu vereinen. Hierzu wird ein sensorunabhängiger Ansatz zur Fusion der Punktdaten dargelegt, der im Vergleich zu den anderen Fusionsebenen und den Einzelsensoren die genaueste Objektbeschreibung liefert. Für die oberen Fusionsebenen wurden unter Ausnutzung der konkurrierenden Sensorinformationen neuartige Verfahren zur Bestimmung und Reduzierung der Detektions- und Verarbeitungsfehler entwickelt. Abschließend wird beschrieben, wie die fehlerreduzierenden Verfahren der oberen Fusionsebenen mit der optimalen Objektbeschreibung der unteren Fusionsebene für eine optimale Objektbestimmung zusammengeführt werden können. Die Effektivität der entwickelten Verfahren wurde durch Simulation oder in realen Messszenarien überprüft. / With the present thesis a contribution to the increase of the accuracy and reliability of a sensor-supported recognition and tracking of objects in a vehicle’s surroundings should be made. Based on a detection system, consisting of a stereo camera and a laser scanner, novel developed procedures are introduced for the whole processing chain of the sensor data. In addition, a new framework is introduced for the fusion of heterogeneous sensor data. By combining the data fusion results from the different processing levels the object detection can be improved. After a short description of the used sensor setup the developed procedures for the calibration and mutual orientation are introduced. With the segmentation of the spatial point data existing procedures are extended by the inclusion of measuring accuracy and specificity of the sensor. In the subsequent object tracking a new computation-optimized approach for the association of the related object hypotheses is presented. In addition, a model for a dynamic determination and tracking of an object reference point is described which exceeds the classical tracking of the object center in the track accuracy. By the introduced fusion framework it is possible to merge the sensor data at three different processing levels (point, object and track level). A sensor independent approach for the low fusion of point data is demonstrated which delivers the most precise object description in comparison to the other fusion levels and the single sensors. For the higher fusion levels new procedures were developed to discover and clean up the detection and processing mistakes benefiting from the competing sensor information. Finally it is described how the fusion results of the upper and lower levels can be brought together for an ideal object description. The effectiveness of the newly developed methods was checked either by simulation or in real measurement scenarios.

Master of Science / Department of Computing and Information Sciences / Gurdip Singh / Severe energy constraints and limited computing abilities of the nodes in a network present a major challenge in the design and deployment of a wireless sensor network. This thesis aims to present energy efficient algorithms for data fusion and information aggregation in a sensor network. The various methodologies of data fusion presented in this thesis intend to reduce the data traffic within a network by mapping the sensor network application task graph onto a sensor network topology. Partitioning of an application into sub-tasks that can be mapped onto the nodes of a sensor network offers opportunities to reduce the overall energy consumption of a sensor network. The first approach proposes a grid based coordinated incremental data fusion and routing with heterogeneous nodes of varied computational abilities. In this approach high performance nodes arranged in a mesh like structure spanning the network topology, are present amongst the resource constrained nodes. The sensor network protocol performance, measured in terms of hop-count is analysed for various grid sizes of the high performance nodes. To reduce network traffic and increase the energy efficiency in a randomly deployed sensor network, distributed clustering strategies which consider network density and structure similarity are applied on the network topology. The clustering methods aim to improve the energy efficiency of the sensor network by dividing the network into logical clusters and mapping the fusion points onto the clusters. Routing of network information is performed by inter-cluster and intra-cluster routing.