As the world explores opportunities to develop offshore renewable energy capacity, there will be a growing need for pre-construction biological surveys and post-construction monitoring in the challenging marine environment. Underwater video is a powerful tool to facilitate such surveys, but the interpretation of the imagery is costly and time-consuming. Emerging technologies have improved automated analysis of underwater video, but these technologies are not yet accurate or accessible enough for widespread adoption in the scientific community or industries that might benefit from these tools. To address these challenges, prior research developed a website that allows to: (1) Quickly play and annotate underwater videos, (2) Create a short tracking video for each annotation that shows how an annotated concept moves in time, (3) Verify the accuracy of existing annotations and tracking videos, (4) Create a neural network model from existing annotations, and (5) Automatically annotate unwatched videos using a model that was previously created. It uses both validated and unvalidated annotations and automatically generated annotations from trackings to count the number of Rathbunaster californicus (starfish) and Strongylocentrotus fragilis (sea urchin) with count accuracy of 97% and 99%, respectively, and F1 score accuracy of 0.90 and 0.81, respectively. The thesis explores several improvements to the model above. First, a method to sync JavaScript video frames to a stable Python environment. Second, reinforcement training using marine biology experts and the verification feature. Finally, a hierarchical method that allows the model to combine predictions of related concepts. On average, this method improved the F1 scores from 0.42 to 0.45 (a relative increase of 7%) and count accuracy from 58% to 69% (a relative increase of 19%) for the concepts Umbellula Lindahli and Funiculina.

3D scene understanding is an important problem that has experienced great progress in recent years, in large part due to the development of state-of-the-art methods for 3D object detection. However, the performance of 3D object detectors can suffer in scenarios where extreme occlusion of objects is present, or the number of object classes is large. In this paper, we study the problem of inferring 3D counts from densely packed scenes with heterogeneous objects. This problem has applications to important tasks such as inventory management or automatic crop yield estimation. We propose a novel regression-based method, CountNet3D, that uses mature 2D object detectors for finegrained classi- fication and localization, and a PointNet backbone for geo- metric embedding. The network processes fused data from images and point clouds for end-to-end learning of counts. We perform experiments on a novel synthetic dataset for inventory management in retail, which we construct and make publicly available to the community. We also have a proprietary dataset we've collected of real-world scenes. In addition we run experiments to quantify the uncertainty of the models and evaluate the confidence of our predic- tions. Our results show that regression-based 3D counting methods systematically outperform detection-based meth- ods, and reveal that directly learning from raw point clouds greatly assists count estimation under extreme occlusion.

Object counting is a process of determining the quantity of specific objects in images. Accurate object counting is key for various applications in image understanding. The common applications are traffic monitoring, crowd management, wildlife migration monitoring, cell counting in medical images, plant and insect counting in agriculture, etc. Occlusions, complex backgrounds, changes in scale, and variations in object appearance in real-world settings make object counting challenging. This dissertation explores a progression of techniques to achieve robust localization and counting under diverse image modalities. The exploration initiates with addressing the challenges of vehicular target localization in cluttered environments using infrared (IR) imagery. We propose a network, called TCRNet-2, that processes target and clutter information in two parallel channels and then combines them to optimize the target-to-clutter ratio (TCR) metric. Next, we explore class-agnostic object counting in RGB images using vision transformers. The primary motivation for this work is that most current methods excel at counting known object types but struggle with unseen categories. To solve these drawbacks, we propose a class-agnostic object counting method. We introduce a dual-branch architecture with interconnected cross-attention that generates feature pyramids for robust object representations, and a dedicated feature aggregator module that further improves performance. Finally, we propose a novel framework that leverages vision-language models (VLM) for zero-shot object counting. While our earlier class-agnostic counting method demonstrates high efficacy in generalized counting tasks, it relies on user-defined exemplars of target objects, presenting a limitation. Additionally, the previous zero-shot counting method was a reference-less approach, which limits the ability to control the selection of the target object of interest in multi-class scenarios. To address these shortcomings, we propose to utilize vision-language models for zero-shot counting where object categories of interest can be specified by text prompts.

Le problème du dénombrement d’un grand nombre de très petits objets en mouvement dans les vidéos est un contexte applicatif jusqu’à présent peu étudié.Dans ce cadre, la difficulté réside essentiellement dans le fait qu’en raison de leurs très petites tailles apparentes dans la vidéo, il n’est pas possible de définir un modèle géométrique fiable de ces objets. Or, les travaux existants dans le domaine de la détection d’objets dans des vidéo, utilisent souvent un tel modèle géométrique des objets d’intérêt. Les méthodes de détection existantes ne sont de ce fait pas applicables directement dans le cadre de la détection de tels très petits objets. Dans le cadre de cette thèse, il est proposé une méthodologie complète permettant la détection de nombreux petits objets, avec un cadre applicatif visant plus particulièrement la détection et le comptage d’oiseaux migrateurs dans une vidéo. Le principe innovant, proposé en tant qu’une solution de ce problème, consiste à associer des coefficients de fiabilité de détection aux objets pour les dénombrer tout en évitant de prendre en compte de trop nombreuses fausses détections. Un algorithme hiérarchique analysant l’aspect spatio-temporel d’objets (leurs apparence et l’évolution dans le temps) dans une vidéo à l’aide de méthodes de traitement d’images, de statistique et de la logique floue est ainsi proposé. Le but des coefficients de fiabilité est d’estimer la probabilité que les paramètres d’une détection correspondent aux paramètres attendus pour les objets d’intérêt. Finalement, l’ensemble des coefficients est converti en une valeur qui évalue la séquence du traitement d’un objet. La somme de ces valeurs correspond au nombre d’objets d’intérêt dans une vidéo. Les résultats obtenus montrent que les bonnes détections sont pour la plupart comprises dans le dénombrement avec des coefficients de fiabilité égaux ou proche de 1, et où les fausses détections sont supprimées ou sous-pondérés avec des coefficients de fiabilité plus faible. Les résultats de comptage dans des vidéos contenant de très nombreux oiseaux sont proches de la vérité terrain, ce qui prouve la validité de la solution proposée comme un moyen de dénombrement automatique d’objets dans des vidéos. / The problem of counting of big volumes of very small moving objects in videos is a domain, which was not studied to date. The difficulty of this application consists essentially in the fact, that because of very small sizes of objects, apparent in the videos, it is impossible to define a reliable geometric model of these objects. The researches, existing in the domain of object detection in videos frequently use a geometrical model of objects of interest.For this reason, the existing methods of object detection cannot be applied for the detection of very small objects in the study case. This thesis proposes a complete methodology, allowing the detection of very small objects in videos, and designed particularly the detection and counting of migrating birds in videos. An innovative principle and the solution of this problem consist in association of coefficients of detection reliability to the objects, in order to count them, avoiding counting of many false detections. The solution proposes a hierarchical algorithm, which analyses the spatial and temporal aspects of objects (their appearance and evolution in time) in a video, by the means of methods of image processing, statistics, and fuzzy logic. The aim of the reliability coefficients is to estimate the probability, that the parameters of a detected objects conform to the expected parameters of the objects of interest. Finally, the coefficients are put together and converted into a value, which evaluates the sequence of processing, applied to detect an object. The sum of these values corresponds to the number of the objects of interest in a video. The results show, that the most of correct detections are characterized in the counting by the reliability coefficient equal or close to 1. The results show, that the most of correct detections have their reliability coefficients close to 1, and the false detection are deleted or have low reliability coefficients. The counting results in the videos with numerous groups of migrating birds are close to the ground trough. This validates the proposed solution as a method of automatic counting of objects in videos.

The aim of this thesis is to design an algorithm for an automatic track counting of an image of etched track detector made of CR-39 polymer. Tracks are produced by alpha particles. Chemically etched detector is imaged using a microscope resulting in 64 images of segments on the surface of the detector. Circle shaped tracks in the images have to be detected and counted. This thesis evaluates the utilization of circle hough transform for circle detection. The final software should automate a detector track counting and should also account for defects in the image and contamination of detector surface. The software will produce a measurement report with a total track count in each segment.