Vision is widely acknowledged as the most critical and complex human sense, with visual data constituting roughly 90% of the information transmitted to the brain. As a result, the ability to classify image information is pivotal for enabling computers to interpret the visual world and execute tasks on our behalf. Convolutional neural networks (CNNs) have significantly advanced image classification in recent years, although their training with a vast number of parameters remains challenging, impacting recognition performance. Methods like attention mechanisms have emerged to prioritize extensive training data, which is often difficult to acquire. The limited availability of training data constitutes a significant challenge that forms the core focus of our research. Recent research has begun exploring the integration of predefined filters within CNNs to alleviate the learning burden. However, the integration of these filters, particularly with attention mechanisms, remains an ongoing area of investigation. This dissertation aims to explore effective strategies in this domain.

No description available.

Spatiotemporal Predictive Learning (SPL) is an essential research topic involving many practical and real-world applications, e.g., motion detection, video generation, precipitation forecasting, and traffic flow prediction. The problems and challenges of this field come from numerous data characteristics in both time and space domains, and they vary depending on the specific task. For instance, spatial analysis refers to the study of spatial features, such as spatial location, latitude, elevation, longitude, the shape of objects, and other patterns. From the time domain perspective, the temporal analysis generally illustrates the time steps and time intervals of data points in the sequence, also known as interval recording or time sampling. Typically, there are two types of time sampling in temporal analysis: regular time sampling (i.e., the time interval is assumed to be fixed) and the irregular time sampling (i.e., the time interval is considered arbitrary) related closely to the continuous-time prediction task when data are in continuous space. Therefore, an efficient spatiotemporal predictive method has to model spatial features properly at the given time sampling types. In this thesis, by taking advantage of Machine Learning (ML) and Deep Learning (DL) methods, which have achieved promising performance in many complicated computational tasks, we propose three DL-based models used for Spatiotemporal Sequence Prediction (SSP) with several types of time sampling. First, we design the Trajectory Gated Recurrent Unit Attention (TrajGRU-Attention) with novel attention mechanisms, namely Motion-based Attention (MA), to improve the performance of the standard Convolutional Recurrent Neural Networks (ConvRNNs) in the SSP tasks. In particular, the TrajGRU-Attention model can alleviate the impact of the vanishing gradient, which leads to the blurry effect in the long-term predictions and handle both regularly sampled and irregularly sampled time series. Consequently, this model can work effectively with different scenarios of spatiotemporal sequential data, especially in the case of time series with missing time steps. Second, by taking the idea of Neural Ordinary Differential Equations (NODEs), we propose Trajectory Gated Recurrent Unit integrating Ordinary Differential Equation techniques (TrajGRU-ODE) as a continuous time-series model. With Ordinary Differential Equation (ODE) techniques and the TrajGRU neural network, this model can perform continuous-time spatiotemporal prediction tasks and generate resulting output with high accuracy. Compared to TrajGRU-Attention, TrajGRU-ODE benefits from the development of efficient and accurate ODE solvers. Ultimately, we attempt to combine those two models to create TrajGRU-Attention-ODE. NODEs are still in their early stage of research, and recent ODE-based models were designed for many relatively simple tasks. In this thesis, we will train the models with several video datasets to verify the ability of the proposed models in practical applications. To evaluate the performance of the proposed models, we select four available spatiotemporal datasets based on the complexity level, including the MovingMNIST, MovingMNIST++, and two real-life datasets: the weather radar HKO-7 and KTH Action. With each dataset, we train, validate, and test with distinct types of time sampling to justify the prediction ability of our models. In summary, the experimental results on the four datasets indicate the proposed models can generate predictions properly with high accuracy and sharpness. Significantly, the proposed models outperform state-of-the-art ODE-based approaches under SSP tasks with different circumstances of interval recording.

<p dir="ltr">To address the challenges of increasing global food demand, climate change, and resource constraints, significant advances are required in plant breeding, sustainable agricultural practices, and technological solutions. This dissertation examines the use of remotely sensed data from unmanned aerial vehicles (UAVs) integrated with deep learning models that incorporate temporal attention mechanisms to improve the accuracy and explainability of yield prediction in plant breeding and management trials. This study leverages a multimodal remote sensing dataset, including hyperspectral, LiDAR, and environmental data, to mitigate challenges related to early-season prediction, model explainability, and broad applicability.</p><p dir="ltr">The study consisted of three themes: identification of relevant features within hyperspectral and LiDAR datasets for models, exploration of temporal attention mechanisms to improve model interpretability, and achievement of robust yield prediction generalization across varied temporal periods and geographic areas. The research investigates the utility of Shapley Additive Explanations (SHAP) for feature selection, isolating key features derived from RS data that improve model performance without sacrificing interpretability. Attention-based DL architectures, including stacked Long Short-Term Memory (LSTM) networks, are implemented to capture temporal dynamics and align model predictions with biologically significant growth stages. Transfer learning and domain adaptation are investigated to improve the generalization of yield prediction models under diverse growing conditions and with limited training data.</p><p dir="ltr">The SHAP-based feature selection successfully decreased input dimensionality without sacrificing LSTM model accuracy; concurrently, attention mechanisms highlighted the temporal significance of features, correlating with physiological phases of maize growth. Supervised approaches and semi-supervised/unsupervised generative methods for domain adaptation demonstrated potential for robust cross-environment prediction, enhancing scalability and practical utility. This research contributes to the understanding of how multi-modal remote sensing data and deep learning techniques can be utilized to address crop yield prediction. This research suggests improvements to sustainable agricultural practices are possible, specifically within plant breeding and crop production management.</p>