No description available.

Maintaining the quality of indoor environments in educational facilities is crucial for student comfort, health, well-being, and learning performance. Amidst the growing recognition of the impact of indoor environmental conditions on occupant comfort, health, and well-being, there has been an increasing focus on the assessment and modeling of Indoor Environmental Quality (IEQ). Despite considerable advancements, current IEQ modeling and assessment methodologies often prioritize and limit to singular comfort metrics, potentially neglect- ing the comprehensive and holistic factors associated with occupant comfort and health. Furthermore, existing indoor environment maintenance practices and building systems for educational facilities often fail to include feedback from occupants (e.g., students and fac- ulty) and exhibit limited adaptability to their needs. This calls for more inclusive and occupant-centric IEQ assessment models that cover a broader spectrum of environmental parameters and occupant needs. To address the gaps, this thesis proposes a novel Artificial Intelligence of Things (AIoT)-based IEQ assessment framework that bridges gaps by uti- lizing multimodal data fusion and deep learning-based prediction and classification models. These models are developed to utilize real-time multidimensional IEQ data, non-intrusive occupant feedback (MFCC features from audio recordings, video/thermal features extracted by Vision Transformer (ViT)), and self-reported comfort and health levels, placing a focus on occupant-centric and data-driven decision-making for intelligent educational facilities. The proposed framework was evaluated and validated at Virginia Tech Blacksburg campus, achieving a 91.9% in R2 score in predicting future IEQ conditions and 97% and 96% accuracy in comfort and health-based IEQ conditions classifications. / Master of Science

In the forest products manufacturing industry, recent price increases in the cost of high-quality lumber together with the reduced availability of this resource have forced manufacturers to utilize lower grade hardwood lumber in their manufacturing operations. This use of low quality lumber means that the labor involved in converting this lumber to usable parts is also increased because it takes more time to remove the additional defects that occur in the lower grade material. Simultaneously, labor costs have gone up and availability of skilled workers capable of getting a high yield of usable parts has markedly decreased. To face this increasingly complex and competitive environment, the industry has a critical need for efficient and cost-effective new processing equipment that can replace human operators who locate and identify defects that need to be removed in lumber and then remove these defects when cutting the lumber into rough parts. This human inspection process is laborious, inconsistent and subjective in nature due to the demands of making decisions very rapidly in a noisy and tiring environment. Hence, an automatic sawing system that could remove defects in lumber while creating maximum yield, offers significant opportunities for increasing profits of this industry. The difficult part in designing an automatic sawing system is creating an automatic inspection system that can detect critical features in wood that affect the quality of the rough parts. Many automatic inspection systems have been proposed and studied for the inspection of wood or wood products. But, most of these systems utilize a single sensing modality, e.g., a single optical sensor or an X-ray imaging system. These systems cannot detect all critical defects in wood. This research work reported in this dissertation is the first aimed at creating a vision system utilizes three imaging modalities: a color imaging system, a laser range profiling system and an X-ray imaging system. The objective of in designing this vision system is to detect and identify: 1) surface features such as knots, splits, stains; 2) geometry features such as wane, thin board; and 3) internal features such as voids, knots. The laser range profiling system is used to locate and identify geometry features. The X-ray imaging system is primarily used to detect features such as knots, splits and interior voids. The color imaging system is mainly employed to identify surface features. In this vision system a number of methodologies are used to improve processing speed and identification accuracy. The images from different sensing modalities are analyzed in a special order to offset the larger amount of image data that comes from the multiple sensors and that must be analyzed. The analysis of laser image is performed first. It is used to find defects that have insufficient thickness. These defects are then removed from consideration in the subsequent analysis of the X-ray image. Removing these defects from consideration in the analysis of the X-ray image not only improves the accuracy of detecting and identifying defects but also reduces the amount of time needed to analyze the X-ray image. Similarly, defect areas such as knot and mineral streak that are found in the analysis of the X-ray image are removed from consideration in the analysis of the color image. A fuzzy logic algorithm -- the approaching degree method-- is used to assign defect labels. The fuzzy logic approach is used to mimic human behavior in identifying defects in hardwood lumber. The initial results obtained from this vision system demonstrate the feasibility of locating and identifying all the major defects that occur in hardwood lumber. This was even true during the initial hardware development phase when only images of unsatisfactory quality from a limited lumber of samples were available. The vision system is capable of locating and identifying defects at the production speed of two linear feet per second that is typical in most hardwood secondary manufacturing plants. This vision system software was designed to run on a relative slow computer (200 MHz Pentium processor) with aid of special image processing hardware, i.e., the MORRPH board that was also designed at Virginia Tech. / Ph. D.

Systems biology aims to understand cellular behavior in terms of the spatiotemporal interactions among cellular components, such as genes, proteins and metabolites. Comprehensive visualization tools for exploring multivariate data are needed to gain insight into the physiological processes reflected in these molecular profiles. Data fusion methods are required to integratively study high-throughput transcriptomics, metabolomics and proteomics data combined before systems biology can live up to its potential. In this work I explored mathematical and statistical methods and visualization tools to resolve the prominent issues in the nature of systems biology data fusion and to gain insight into these comprehensive data. In order to choose and apply multivariate methods, it is important to know the distribution of the experimental data. Chi square Q-Q plot and violin plot were applied to all M. truncatula data and V. vinifera data, and found most distributions are right-skewed (Chapter 2). The biplot display provides an effective tool for reducing the dimensionality of the systems biological data and displaying the molecules and time points jointly on the same plot. Biplot of M. truncatula data revealed the overall system behavior, including unidentified compounds of interest and the dynamics of the highly responsive molecules (Chapter 3). The phase spectrum computed from the Fast Fourier transform of the time course data has been found to play more important roles than amplitude in the signal reconstruction. Phase spectrum analyses on in silico data created with two artificial biochemical networks, the Claytor model and the AB2 model proved that phase spectrum is indeed an effective tool in system biological data fusion despite the data heterogeneity (Chapter 4). The difference between data integration and data fusion are further discussed. Biplot analysis of scaled data were applied to integrate transcriptome, metabolome and proteome data from the V. vinifera project. Phase spectrum combined with k-means clustering was used in integrative analyses of transcriptome and metabolome of the M. truncatula yeast elicitation data and of transcriptome, metabolome and proteome of V. vinifera salinity stress data. The phase spectrum analysis was compared with the biplot display as effective tools in data fusion (Chapter 5). The results suggest that phase spectrum may perform better than the biplot. This work was funded by the National Science Foundation Plant Genome Program, grant DBI-0109732, and by the Virginia Bioinformatics Institute. / Ph. D.

Tracking human hand motions has raised more attention due to the recent advancements of virtual reality (Rheingold, 1991) and prosthesis control (Antfolk et al., 2010). Surface electromyography (sEMG) has been the predominant method for sensing electrical activity in biomechanical studies, and has also been applied to motion tracking in recent years. While most studies focus on the classification of human hand motions within a predefined motion set, the prediction of continuous finger joint angles and wrist angles remains a challenging endeavor. In this research, a biomechanical knowledge-driven data fusion strategy is proposed to predict finger joint angles and wrist angles. This strategy combines time series data of sEMG signals and simulated muscle features, which can be extracted from a biomechanical model available in OpenSim (Delp et al., 2007). A support vector regression (SVR) model is used to firstly predict muscle features from sEMG signals and then to predict joint angles from the estimated muscle features. A set of motion data containing 10 types of motions from 12 participants was collected from an institutional review board approved experiment. A hypothesis was tested to validate whether adding the simulated muscle features would significantly improve the prediction performance. The study indicates that the biomechanical knowledge-driven data fusion strategy will improve the prediction of new types of human hand motions. The results indicate that the proposed strategy significantly outperforms the benchmark date-driven model especially when the users were performing unknown types of motions from the model training stage. The proposed model provides a possible approach to integrate the simulation models and data fusion models in human factors and ergonomics. / Master of Science

Many countries around the world are undergoing massive urbanization campaigns at an unprecedented rate, heralded by promises of economical prosperity and bolstered population health and well-being. Projections indicate that by 2050, nearly 68% of the world populace will reside in these urban environments. However, rapid growth at such an exceptional scale poses unique challenges pertaining to environmental quality and food production, which can negate the effectiveness of the aforementioned boons. As such, there is an emphasis on mitigating these negative effects through the construction of smart and connected communities (S&CC), which integrate both artificial intelligence (AI) and the Internet of Things (IoT). This coupling of intelligent technologies also poses interesting system design challenges pertaining to the fusion of the diverse, heterogeneous datasets available to IoT environments, and the ability to learn multiple S&CC problem sets concurrently. Attention-based Transformer networks are of particular interest given their success across diverse fields of natural language processing (NLP), computer vision, time-series regression, and multi-modal data fusion in recent years. This begs the question whether Transformers can be further diversified to leverage fusions of IoT data sources for heterogeneous multi-task learning in S&CC trade spaces. This is a fundamental question that this thesis seeks to answer. Indeed, the key contribution of this thesis is the design and application of Transformer networks for developing AI systems in emerging smart cities. This is executed within a collaborative U.S.-Indonesia effort between Virginia Tech, the University of Colorado Boulder, the Universitas Gadjah Mada, and the Institut Teknologi Bandung with the goal of growing smart and sustainable garden alleys in Makassar City, Indonesia. Specifically, a proof-of-concept AI nerve-center is proposed using a backbone of pure-encoder Transformer architectures to learn a diverse set of tasks such as multivariate time-series regression, visual plant disease classification, and image-time-series fusion. To facilitate the data fusion tasks, an effective algorithm is also proposed to synthesize heterogeneous feature sets, such as multivariate time-series and time-correlated images. Moreover, a hyperparameter tuning framework is also proposed to standardize and automate model training regimes. Extensive experimentation shows that the proposed Transformer-based systems can handle various input data types via custom sequence embedding techniques, and are naturally suited to learning a diverse set of tasks. Further, the results also show that multi-task learners increase both memory and computational efficiency while maintaining comparable performance to both single-task variants, and non-Transformer baselines. This demonstrates the flexibility of Transformer networks to learn from a fusion of IoT data sources, their applicability in S&CC trade spaces, and their further potential for deployment on edge computing devices. / Master of Science / Many countries around the world are undergoing massive urbanization campaigns at an unprecedented rate, heralded by promises of economical prosperity and bolstered population health and well-being. Projections indicate that by 2050, nearly 68% of the world populace will reside in these urban environments. However, rapid growth at such an exceptional scale poses unique environmental and food cultivation challenges. Hence, there is a focus on reducing these negative effects through building smart and connected communities (S&CC). The term connected is derived from the integration of small, low-cost devices which gather information from the surrounding environment, called the Internet of Things (IoT). Likewise, smart is a term derived from the integration of artificial intelligence (AI), which is used to make informed decisions based on IoT-collected information. This coupling of intelligent technologies also poses its own unique challenges pertaining to the blending of IoT data with highly diverse characteristics. Of specific interest is the design of AI models that can not only learn from a fusion of this diverse information, but also learn to perform multiple tasks in parallel. Attention-based networks are a relatively new category of AI which learn to focus on, or attend to, the most important portions of an arbitrary data sequence. Transformers are AI models which are designed using attention as their backbone, and have been employed to much success in many fields in recent years. This success begs the question whether Transformers can be further extended to put the smart in S&CC. The overarching goal of this thesis is to design and implement a Transformer-based AI system for emerging smart cities. In particular, this is accomplished within a U.S.-Indonesia collaborative effort with the goal of growing smart and sustainable garden alleys in Makassar City, Indonesia.

In recent years, as the availability of remote sensing imagery of varying resolution has increased, merging images of differing spatial resolution has become a significant operation in the field of digital remote sensing. This practice, known as data fusion, is designed to enhance the spatial resolution of multispectral images by merging a relatively coarse-resolution image with a higher resolution panchromatic image of the same geographic area. This study examines properties of fused images and their ability to preserve the spectral integrity of the original image. It analyzes five current data fusion techniques for three complex scenes to assess their performance. The five data fusion models used include one spatial domain model (High-Pass Filter), two algebraic models (Multiplicative and Brovey Transform), and two spectral domain models (Principal Components Transform and Intensity-Hue-Saturation). SPOT data were chosen for both the panchromatic and multispectral data sets. These data sets were chosen for the high spatial resolution of the panchromatic (10 meters) data, the relatively high spectral resolution of the multispectral data, and the low spatial resolution ratio of two to one (2:1). After the application of the data fusion techniques, each merged image was analyzed statistically, graphically, and for increased photointerpretive potential as compared with the original multispectral images. While all of the data fusion models distorted the original multispectral imagery to an extent, both the Intensity-Hue-Saturation Model and the High-Pass Filter model maintained the original qualities of the multispectral imagery to an acceptable level. The High-Pass Filter model, designed to highlight the high frequency spatial information, provided the most noticeable increase in spatial resolution. / Master of Science

Mit der vorliegenden Dissertation soll ein Beitrag zur Steigerung der Genauigkeit und Zuverlässigkeit einer sensorgestützten Objekterkennung im Fahrzeugumfeld geleistet werden. Aufbauend auf einem Erfassungssystem, bestehend aus einer Stereokamera und einem Mehrzeilen-Laserscanner, werden teils neu entwickelte Verfahren für die gesamte Verarbeitungskette vorgestellt. Zusätzlich wird ein neuartiges Framework zur Fusion heterogener Sensordaten eingeführt, welches über eine Zusammenführung der Fusionsergebnisse aus den unterschiedlichen Verarbeitungsebenen in der Lage ist, die Objektbestimmung zu verbessern. Nach einer Beschreibung des verwendeten Sensoraufbaus werden die entwickelten Verfahren zur Kalibrierung des Sensorpaares vorgestellt. Bei der Segmentierung der räumlichen Punktdaten werden bestehende Verfahren durch die Einbeziehung von Messgenauigkeit und Messspezifik des Sensors erweitert. In der anschließenden Objektverfolgung wird neben einem neuartigen berechnungsoptimierten Ansatz zur Objektassoziierung ein Modell zur adaptiven Referenzpunktbestimmung und –Verfolgung beschrieben. Durch das vorgestellte Fusions-Framework ist es möglich, die Sensordaten wahlweise auf drei unterschiedlichen Verarbeitungsebenen (Punkt-, Objekt- und Track-Ebene) zu vereinen. Hierzu wird ein sensorunabhängiger Ansatz zur Fusion der Punktdaten dargelegt, der im Vergleich zu den anderen Fusionsebenen und den Einzelsensoren die genaueste Objektbeschreibung liefert. Für die oberen Fusionsebenen wurden unter Ausnutzung der konkurrierenden Sensorinformationen neuartige Verfahren zur Bestimmung und Reduzierung der Detektions- und Verarbeitungsfehler entwickelt. Abschließend wird beschrieben, wie die fehlerreduzierenden Verfahren der oberen Fusionsebenen mit der optimalen Objektbeschreibung der unteren Fusionsebene für eine optimale Objektbestimmung zusammengeführt werden können. Die Effektivität der entwickelten Verfahren wurde durch Simulation oder in realen Messszenarien überprüft. / With the present thesis a contribution to the increase of the accuracy and reliability of a sensor-supported recognition and tracking of objects in a vehicle’s surroundings should be made. Based on a detection system, consisting of a stereo camera and a laser scanner, novel developed procedures are introduced for the whole processing chain of the sensor data. In addition, a new framework is introduced for the fusion of heterogeneous sensor data. By combining the data fusion results from the different processing levels the object detection can be improved. After a short description of the used sensor setup the developed procedures for the calibration and mutual orientation are introduced. With the segmentation of the spatial point data existing procedures are extended by the inclusion of measuring accuracy and specificity of the sensor. In the subsequent object tracking a new computation-optimized approach for the association of the related object hypotheses is presented. In addition, a model for a dynamic determination and tracking of an object reference point is described which exceeds the classical tracking of the object center in the track accuracy. By the introduced fusion framework it is possible to merge the sensor data at three different processing levels (point, object and track level). A sensor independent approach for the low fusion of point data is demonstrated which delivers the most precise object description in comparison to the other fusion levels and the single sensors. For the higher fusion levels new procedures were developed to discover and clean up the detection and processing mistakes benefiting from the competing sensor information. Finally it is described how the fusion results of the upper and lower levels can be brought together for an ideal object description. The effectiveness of the newly developed methods was checked either by simulation or in real measurement scenarios.

Master of Science / Department of Computing and Information Sciences / Gurdip Singh / Severe energy constraints and limited computing abilities of the nodes in a network present a major challenge in the design and deployment of a wireless sensor network. This thesis aims to present energy efficient algorithms for data fusion and information aggregation in a sensor network. The various methodologies of data fusion presented in this thesis intend to reduce the data traffic within a network by mapping the sensor network application task graph onto a sensor network topology. Partitioning of an application into sub-tasks that can be mapped onto the nodes of a sensor network offers opportunities to reduce the overall energy consumption of a sensor network. The first approach proposes a grid based coordinated incremental data fusion and routing with heterogeneous nodes of varied computational abilities. In this approach high performance nodes arranged in a mesh like structure spanning the network topology, are present amongst the resource constrained nodes. The sensor network protocol performance, measured in terms of hop-count is analysed for various grid sizes of the high performance nodes. To reduce network traffic and increase the energy efficiency in a randomly deployed sensor network, distributed clustering strategies which consider network density and structure similarity are applied on the network topology. The clustering methods aim to improve the energy efficiency of the sensor network by dividing the network into logical clusters and mapping the fusion points onto the clusters. Routing of network information is performed by inter-cluster and intra-cluster routing.

Since the origin of the automotive at the end of the 19th century, the traffic flow is subject to a constant increase and, unfortunately, involves a constant augmentation of road accidents. Research studies such as the one performed by the World Health Organization, show alarming results about the number of injuries and fatalities due to these accidents. To reduce these figures, a solution lies in the development of Advanced Driver Assistance Systems (ADAS) which purpose is to help the Driver in his driving task. This research topic has been shown to be very dynamic and productive during the last decades. Indeed, several systems such as Anti-lock Braking System (ABS), Electronic Stability Program (ESP), Adaptive Cruise Control (ACC), Parking Manoeuvre Assistant (PMA), Dynamic Bending Light (DBL), etc. are yet market available and their benefits are now recognized by most of the drivers. This first generation of ADAS are usually designed to perform a specific task in the Controller/Vehicle/Environment framework and thus requires only microscopic information, so requires sensors which are only giving local information about an element of the Vehicle or of its Environment. On the opposite, the next ADAS generation will have to consider more aspects, i.e. information and constraints about of the Vehicle and its Environment. Indeed, as they are designed to perform more complex tasks, they need a global view about the road context and the Vehicle configuration. For example, longitudinal control requires information about the road configuration (straight line, bend, etc.) and about the eventual presence of other road users (vehicles, trucks, etc.) to determine the best reference speed. [...]