Radiologists are often plagued by limited time for completing their work, with an ever increasing workload. A picture archiving and communication system (PACS) is a platform for daily image reviewing that improves their work environment, and on that platform for example spinal MR images can be reviewed. When reviewing spinal images a radiologist wants vertebrae labels, and in Sectra's PACS platform there is a good opportunity for implementing an automatic method for spinal labeling. In this thesis a method for performing automatic spinal labeling, called a vertebrae classifier, is presented. This method should remove the need for radiologists to perform manual spine labeling, and could be implemented in Sectra's PACS software to improve radiologists overall work experience.Spine labeling is the process of marking vertebrae centres with a name on a spinal image. The method proposed in this thesis for performing that process was developed using a machine learning approach for vertebrae detection in sagittal MR images. The developed classifier works for both the lumbar and the cervical spine, but it is optimized for the lumbar spine. During the development three different methods for the purpose of vertebrae detection were evaluated. Detection is done on multiple sagittal slices. The output from the detection is then labeled using a pictorial structure based algorithm which uses a trained model of the spine to correctly assess correct labeling. The suggested method achieves 99.6% recall and 99.9% precision for the lumbar spine. The cervical spine achieves slightly worse performance, with 98.1% for both recall and precision. This result was achieved by training the proposed method on 43 images and validated with 89 images for the lumbar spine. The cervical spine was validated using 26 images. These results are promising, especially for the lumbar spine. However, further evaluation is needed to test the method in a clinical setting. / Radiologer får bara mindre och mindre tid för att utföra sina arbetsuppgifter, då arbetsbördan bara blir större. Ett picture archiving and communication system (PACS) är en platform där radiologer kan undersöka medicinska bilder, däribland magnetic resonance (MR) bilder av ryggraden. När radiologerna tittar på dessa bilder av ryggraden vill de att kotorna ska vara markerade med sina namn, och i Sectra's PACS platform finns det en bra möjlighet för att implementera en automatisk metod för att namnge ryggradens kotor på bilden. I detta examensarbete presenteras en metod för att automatiskt markera alla kotorna utifrån saggitala MR bilder. Denna metod kan göra så att radiologer inte längre behöver manuellt markera kotor, och den skulle kunna implementeras i Sectra's PACS för att förbättra radiologernas arbetsmiljö. Det som menas med att markera kotor är att man ger mitten av alla kotor ett namn utifrån en MR bild på ryggraden. Metoden som presenteras i detta arbete kan utföra detta med hjälp av ett "machine learning" arbetssätt. Metoden fungerar både för övre och nedre delen av ryggraden, men den är optimerad för den nedre delen. Under utvecklingsfasen var tre olika metoder för att detektera kotor evaluerade. Resultatet från detektionen är sedan använt för att namnge alla kotor med hjälp av en algoritm baserad på pictorial structures, som använder en tränad model för att kunna evaluera vad som bör anses vara korrekt namngivning. Metoden uppnår 99.6% recall och 99.9% precision för nedre ryggraden. För övre ryggraden uppnås något sämre resultat, med 98.1% vad gäller både recall och precision. Detta resultat uppnådes då metoden tränades på 43 bilder och validerades på 89 bilder för nedre ryggraden. För övre ryggraden användes 26 stycken bilder. Resultaten är lovande, speciellt för den nedre delen. Dock måste ytterligare utvärdering göras för metoden i en klinisk miljö.

La thèse s'inscrit dans l'étude de la structuration des forêts à partir des propriétés de la canopée telles que décrites par la distribution spatiale ou la taille des houppiers des arbres dominants. L'approche suivie est fondée sur la théorie des Processus Ponctuels Marqués (PPM) qui permet de modéliser ces houppiers comme des disques sur images considérées comme un espace 2D. Le travail a consisté à évaluer le potentiel des PPM pour détecter automatiquement les houppiers d'arbres dans des images optiques de très résolution spatiale acquises sur des forêts de mangroves et des plantations d'Eucalyptus. Pour les mangroves, nous avons également travaillé sur des images simulées de réflectance et des données Lidar. Différentes adaptations (paramétrage, modèles d'énergie) de la méthode de PPM ont été testées et comparées grâce à des indices quantitatifs de comparaison entre résultats de la détection et références de positionnement issues du terrain, de photo-interprétation ou de maquettes forestières.Dans le cas des mangroves, les tailles de houppier estimées par détection restent cohérentes avec les sorties des modèles allométriques disponibles. Les résultats thématiques indiquent que la détection par PPM permet de cartographier dans une jeune plantation d'Eucalyptus la densité locale d'arbres dont la taille des houppiers est proche de la résolution spatiale de l'image (0.5m). Cependant, la qualité de la détection diminue quand le couvert se complexifie. Ce travail dresse plusieurs pistes de recherche tant mathématique, comme la prise en compte des objets de forme complexe, que thématiques, comme l'apport des informations forestières à des échelles pertinentes pour la mise au point de méthodes de télédétection. / This PhD work aims at providing information on the forest structure through the analysis of canopy properties as described by the spatial distribution and the crown size of dominant trees. Our approach is based on the Marked Point Processes (MPP) theory, which allows modeling tree crowns observed in remote sensing images by discs belonging a two dimensional space. The potential of MPP to detect the trees crowns automatically is evaluated by using very high spatial resolution optical satellite images of both Eucalyptus plantations and mangrove forest. Lidar and simulated reflectance images are also analyzed for the mangrove application. Different adaptations (parameter settings, energy models) of the MPP method are tested and compared through the development of quantitative indices that allow comparison between detection results and tree references derived from the field, photo-interpretation or the forest mockups.In the case of mangroves, the estimated crown sizes from detections are consistent with the outputs from the available allometric models. Other results indicate that tree detection by MPP allows mapping, the local density of trees of young Eucalyptus plantations even if crown size is close to the image spatial resolution (0.5m). However, the quality of detection by MPP decreases with canopy closeness. To improve the results, further work may involve MPP detection using objects with finer shapes and forest data measurements collected at the tree plant scale.

Bien que devenant un environnement de plus en plus riche technologiquement, la salle opératoire reste un endroit où la sécurité des patients n'est pas assurée à 100% comme le montre le nombre toujours conséquent d'erreurs chirurgicales. La nécessité de développer des systèmes intelligents au bloc opératoire apparait comme croissante. Un des éléments clés pour ce type de système est la reconnaissance du processus chirurgical, passant par une identification précise des outils chirurgicaux utilisés. L'objectif de cette thèse a donc porté sur la détection en temps-réel des outils chirurgicaux dans des vidéos 2D provenant de microscopes. Devant l'absence de jeux de données de référence, qui plus est dans un contexte neurochirurgical, la première contribution de la thèse a donc été la création d'un nouvel ensemble d'images de chirurgies du cerveau et du rachis cervical, mis à disposition en ligne. Comme seconde contribution, deux approches différentes ont été proposées permettant de détecter des outils chirurgicaux via des techniques d'analyse d'image. Tout d'abord, le SquaresChnFtrs adapté, basé sur une des méthodes de détection de piétons les plus performantes de la littérature. Notre deuxième méthode, le ShapeDetector, est une approche à deux niveaux n'utilisant aucune contrainte ou hypothèse a priori sur le nombre, la position, ou la forme des outils dans l'image. Par rapport aux travaux précédents du domaine, nous avons choisi de représenter les détections potentielles par des polygones plutôt que par des rectangles, obtenant ainsi des détections plus précises. Pour intégration dans des systèmes médicaux, une optimisation de la vitesse de calcul a été effectuée via un usage optimal du CPU, du GPU, et de méthodes ad-hoc. Pour des vidéos de résolution 612x480 pixels, notre ShapeDetector est capable d'effectuer les détections à une vitesse maximale de 8 Hz. Pour la validation de nos méthodes, nous avons pris en compte trois paramètres: la position globale, la position de l'extrémité, et l'orientation des détections. Les méthodes ont été classées et comparées avec des méthodes de référence compétitives. Pour la détection des tubes d'aspiration, nous avons obtenu un taux de manqué de 15% pour un taux de faux positifs par image de 0.1, comparé à un taux de manqué de 55% pour le SquaresChnFtrs adapté. L'orientation future du travail devra porter sur l'intégration des informations 3D, l'amélioration de la couche de labélisation sémantique, et la classification des outils chirurgicaux. Pour finir, un enrichissement du jeu de données et des annotations de plus haute précision seront nécessaires. / Despite modern-life technological advances and tremendous progress made in surgical techniques including MIS, today's OR is facing many challenges remaining yet to be addressed. The development of CAS systems integrating the SPM methodology was born as a response from the medical community, with the long-term objective to create surgical cockpit systems. Being able to identify surgical tools in use is a key component for systems relying on the SPM methodology. Towards that end, this thesis work has focused on real-time surgical tool detection from microscope 2D images. From the review of the literature, no validation data-sets have been elected as benchmarks by the community. In addition, the neurosurgical context has been addressed only once. As such, the first contribution of this thesis work consisted in the creation of a new surgical tool data-set, made freely available online. Two methods have been proposed to tackle the surgical tool detection challenge. First, the adapted SquaresChnFtrs, evolution of one of the best computer vision state-of-the-art approach for pedestrian detection. Our second contribution, the ShapeDetector, is fully data-driven and performs detection without the use of prior knowledge regarding the number, shape, and position of tools in the image. Compared to previous works, we chose to represent candidate detections with bounding polygons instead of bounding boxes, hence providing more fitting results. For integration into medical systems, we performed different code optimization through CPU and GPU use. Speed gain and accuracy loss from the use of ad-hoc optimization strategies have been thoroughly quantified to find an optimal trade-off between speed and accuracy. Our ShapeDetector is running in-between 5 and 8Hz for 612x480 pixel video sequences.We validated our approaches using a detailed methodology covering the overall tool location, tip position, and orientation. Approaches have been compared and ranked conjointly with a set of competitive baselines. For suction tube detections, we achieved a 15% miss-rate at 0.1 FPPI, compared to a 55% miss-rate for the adapted SquaresChnFtrs. Future works should be directed toward the integration of 3D feature extraction to improve detection performance but also toward the refinement of the semantic labelling step. Coupling the tool detection task to the tool classification in one single framework should be further investigated. Finally, increasing the data-set in diversity, number of tool classes, and detail of annotations is of interest.

Detecção de objetos é um problema clássico em visão computacional, presente em aplicações como vigilância automatizada, análise de imagens médicas e recuperação de informação. Dentre as abordagens existentes na literatura para resolver esse problema, destacam-se métodos baseados em reconhecimento de pontos-chave que podem ser interpretados como diferentes implementações de um mesmo arcabouço. O objetivo desta pesquisa de doutorado é desenvolver e avaliar uma versão generalizada desse arcabouço, na qual reconhecimento de pontos-chave é substituído por reconhecimento de grafos-chave. O potencial da pesquisa reside na riqueza de informação que um grafo pode apresentar antes e depois de ser reconhecido. A dificuldade da pesquisa reside nos problemas que podem ser causados por essa riqueza, como maldição da dimensionalidade e complexidade computacional. Três contribuições serão incluídas na tese: a descrição detalhada de um arcabouço para detecção de objetos baseado em grafos-chave, implementações fiéis que demonstram sua viabilidade e resultados experimentais que demonstram seu desempenho. / Object detection is a classic problem in computer vision, present in applications such as automated surveillance, medical image analysis and information retrieval. Among the existing approaches in the literature to solve this problem, we can highlight methods based on keypoint recognition that can be interpreted as different implementations of a same framework. The objective of this PhD thesis is to develop and evaluate a generalized version of this framework, on which keypoint recognition is replaced by keygraph recognition. The potential of the research resides in the information richness that a graph can present before and after being recognized. The difficulty of the research resides in the problems that can be caused by this richness, such as curse of dimensionality and computational complexity. Three contributions are included in the thesis: the detailed description of a keygraph-based framework for object detection, faithful implementations that demonstrate its feasibility and experimental results that demonstrate its performance.

abstract: Autonomous Vehicles (AVs), or self-driving cars, are poised to have an enormous impact on the automotive industry and road transportation. While advances have been made towards the development of safe, competent autonomous vehicles, there has been inadequate attention to the control of autonomous vehicles in unanticipated situations, such as imminent crashes. Even if autonomous vehicles follow all safety measures, accidents are inevitable, and humans must trust autonomous vehicles to respond appropriately in such scenarios. It is not plausible to program autonomous vehicles with a set of rules to tackle every possible crash scenario. Instead, a possible approach is to align their decision-making capabilities with the moral priorities, values, and social motivations of trustworthy human drivers.Toward this end, this thesis contributes a simulation framework for collecting, analyzing, and replicating human driving behaviors in a variety of scenarios, including imminent crashes. Four driving scenarios in an urban traffic environment were designed in the CARLA driving simulator platform, in which simulated cars can either drive autonomously or be driven by a user via a steering wheel and pedals. These included three unavoidable crash scenarios, representing classic trolley-problem ethical dilemmas, and a scenario in which a car must be driven through a school zone, in order to examine driver prioritization of reaching a destination versus ensuring safety. Sample human driving data in CARLA was logged from the simulated car’s sensors, including the LiDAR, IMU and camera. In order to reproduce human driving behaviors in a simulated vehicle, it is necessary for the AV to be able to identify objects in the environment and evaluate the volume of their bounding boxes for prediction and planning. An object detection method was used that processes LiDAR point cloud data using the PointNet neural network architecture, analyzes RGB images via transfer learning using the Xception convolutional neural network architecture, and fuses the outputs of these two networks. This method was trained and tested on both the KITTI Vision Benchmark Suite dataset and a virtual dataset exclusively generated from CARLA. When applied to the KITTI dataset, the object detection method achieved an average classification accuracy of 96.72% and an average Intersection over Union (IoU) of 0.72, where the IoU metric compares predicted bounding boxes to those used for training. / Dissertation/Thesis / Masters Thesis Mechanical Engineering 2020

Architectural floor plans are scale-accurate 2D drawings of one level of a building, seen from above, which convey structural and semantic information related to rooms, walls, symbols, textual data, etc. They consist of lines, curves, symbols, and textual markings, showing the relationships between rooms and all physical features, required for the proper construction or renovation of the building. First, this thesis provides a thorough study of state-of-the-art on symbol spotting methods for architectural drawings, an application domain providing the document image analysis and graphic recognition communities with an interesting set of challenges linked to the sheer complexity and density of embedded information, that have yet to be resolved. Second, we propose a hybrid method that capitalizes on strengths of both vector-based and pixel-based symbol spotting techniques. In the description phase, the salient geometric constituents of a symbol are extracted by a variety of vectorization techniques, including a proposed voting-based algorithm for finding partial ellipses. This enables us to better handle local shape irregularities and boundary discontinuities, as well as partial occlusion and overlap. In the matching phase, the spatial relationship between the geometric primitives is encoded via a primitive-aware proximity graph. A statistical approach is then used to rapidly yield a coarse localization of symbols within the plan. Localization is further refined with a pixel-based step implementing a modified cross-correlation function. Experimental results on the public SESYD synthetic dataset and real-world images demonstrate that our approach clearly outperforms other popular symbol spotting approaches. Traditional on-the-fly symbol spotting methods are unable to address the semantic challenge of graphical notation variability, i.e. low intra-class symbol similarity, an issue that is particularly important in architectural floor plan analysis. The presence of occlusion and clutter, characteristic of real-world plans, along with a varying graphical symbol complexity from almost trivial to highly complex, also pose challenges to existing spotting methods. Third, we address all the above issues by leveraging recent advances in deep learning-based neural networks and adapting an object detection framework based on the YOLO (You Only Look Once) architecture. We propose a training strategy based on tiles, avoiding many issues particular to deep learning-based object detection networks related to the relatively small size of symbols compared to entire floor plans, aspect ratios, and data augmentation. Experimental results demonstrate that our method successfully detects architectural symbols with low intra-class similarity and of variable graphical complexity, even in the presence of heavy occlusion and clutter. / Graduate

<p>Deep learning solutions to computer vision tasks have revolutionized many industries in recent years, but embedded systems have too many restrictions to take advantage of current state-of-the-art configurations. Typical embedded processor hardware configurations must meet very low power and memory constraints to maintain small and lightweight packaging, and the architectures of the current best deep learning models are too computationally intensive for these hardware configurations. Current research shows that convolutional neural networks (CNNs) can be deployed with a few architectural modifications on Field-Programmable Gate Arrays (FPGAs) resulting in minimal loss of accuracy, similar or decreased processing speeds, and lower power consumption when compared to general-purpose Central Processing Units (CPUs) and Graphics Processing Units (GPUs). This research contributes further to these findings with the FPGA implementation of a YOLOv4 object detection model that was developed with the use of transfer learning. The transfer-learned model uses the weights of a model pre-trained on the MS-COCO dataset as a starting point then fine-tunes only the output layers for detection on more specific objects of five classes. The model architecture was then modified slightly for compatibility with the FPGA hardware using techniques such as weight quantization and replacing unsupported activation layer types. The model was deployed on three different hardware setups (CPU, GPU, FPGA) for inference on a test set of images. It was found that the FPGA was able to achieve real-time inference speeds of 33.77 frames-per-second, a speedup of 7.74 frames-per-second when compared to GPU deployment. The model also consumed 96% less power than a GPU configuration with only approximately 4% average loss in accuracy across all 5 classes. The results are even more striking when compared to CPU deployment, with 131.7-times speedup in inference throughput. CPUs have long since been outperformed by GPUs for deep learning applications but are used in most embedded systems. These results further illustrate the advantages of FPGAs for deep learning inference on embedded systems even when transfer learning is used for an efficient end-to-end deployment process. This work advances current state-of-the-art with the implementation of a YOLOv4 object detection model developed with transfer learning for FPGA deployment.</p>

Automatic Passenger Counting (APC) systems are some of the many Internet-Of-Things (IoT) applications and have been increasingly adopted by public transportation companies in recent years. APCs provide valuable data that can be used to give an real time passenger count, which can be a convenient service and allow customers to plan their travels accordingly. The provided data is also valuable for resource streamlining and planning, which potentially increases revenues for the public transportation companies. This thesis briefly studies and evaluates different APC technologies, highlights the advantages and disadvantages of these, and presents an Edge-prototype based on Computer Vision and Object Detection. The presented APC was tested in a lab environment and with recordings of people walking in and out of a designated area in the lab. Test results from the lab environment show that the presented low-cost APC efficiently detects passengers with an accuracy of 98.6% on pre-recorded videos. The APC was also tested in real time and the results show that the low-cost APC only achieved an accuracy of 66.7%. This work has laid the ground for further development and testing in a public transport environment.

Indianapolis / E-scooters are ubiquitous and their number keeps escalating, increasing their interactions with other vehicles on the road. E-scooter riders have an atypical behavior that varies enormously from other vulnerable road users, creating new challenges for vehicle active safety systems and automated driving functionalities. The detection of e-scooter riders by other vehicles is the first step in taking care of the risks. This research presents a novel vision-based system to differentiate between e-scooter riders and regular pedestrians and a benchmark dataset for e-scooter riders in natural environments. An efficient system pipeline built using two existing state-of-the-art convolutional neural networks (CNN), You Only Look Once (YOLOv3) and MobileNetV2, performs detection of these vulnerable e-scooter riders.

Sensors, as a crucial part of autonomous driving, are primarily used for perceiving the environment. The recent deep learning development of different sensors has demonstrated the ability of machines recognizing and understanding their surroundings. Automotive radar, as a primary sensor for self-driving vehicles, is well-known for its robustness against variable lighting and weather conditions. Compared with camera-based deep learning development, Object detection using automotive radars has not been explored to its full extent. This can be attributed to the lack of public radar datasets. In this thesis, we collect a novel radar dataset that contains radar data in the form of Range-Azimuth-Doppler tensors along with the bounding boxes on the tensor for dynamic road users, category labels, and 2D bounding boxes on the Cartesian Bird-EyeView range map. To build the dataset, we propose an instance-wise auto-annotation algorithm. Furthermore, a novel Range-Azimuth-Doppler based multi-class object detection deep learning model is proposed. The algorithm is a one-stage anchor-based detector that generates both 3D bounding boxes and 2D bounding boxes on Range-AzimuthDoppler and Cartesian domains, respectively. Our proposed algorithm achieves 56.3% AP with IOU of 0.3 on 3D bounding box predictions, and 51.6% with IOU of 0.5 on 2D bounding box predictions. Our dataset and the code can be found at https://github.com/ZhangAoCanada/RADDet.git.