Remote sensing helps many applications like precision irrigation, habitat mapping, and traffic monitoring. However, due to shortcomings of current remote sensing platforms - like high cost, low spatial, and temporal resolution - many applications do not have access to useful remote sensing data. A team at the Center for Self-Organizing and Intelligent Systems (CSOIS) together with the Utah Water Research Laboratory (UWRL) at Utah State University has been developing a new remote sensing platform to deal with these shortcomings in order to give more applications access to remote sensing data. This platform (AggieAir) is low cost, fully autonomous, easy to use, independent of a runway, has a fast turnover time, and a high spatial resolution. A program called the Geospatial Real-Time Aerial Image Display (gRAID) has also been developed to process the images taken from AggieAir. gRAID is able to correct the camera lens distortion, georeference, and display the images on a 3D globe, and export them in a conventional Geographic Information System (GIS) format for further processing. AggieAir and gRAID prove to be innovative and useful tools for remote sensing.

<p>In this thesis, semantic mapping is understood to be the process of putting a tag or label on objects or regions in a map. This label should be interpretable by and have a meaning for a human. The use of semantic information has several application areas in mobile robotics. The largest area is in human-robot interaction where the semantics is necessary for a common understanding between robot and human of the operational environment. Other areas include localization through connection of human spatial concepts to particular locations, improving 3D models of indoor and outdoor environments, and model validation.</p><p>This thesis investigates the extraction of semantic information for mobile robots in outdoor environments and the use of semantic information to link ground-level occupancy maps and aerial images. The thesis concentrates on three related issues: i) recognition of human spatial concepts in a scene, ii) the ability to incorporate semantic knowledge in a map, and iii) the ability to connect information collected by a mobile robot with information extracted from an aerial image.</p><p>The first issue deals with a vision-based virtual sensor for classification of views (images). The images are fed into a set of learned virtual sensors, where each virtual sensor is trained for classification of a particular type of human spatial concept. The virtual sensors are evaluated with images from both ordinary cameras and an omni-directional camera, showing robust properties that can cope with variations such as changing season.</p><p>In the second part a probabilistic semantic map is computed based on an occupancy grid map and the output from a virtual sensor. A local semantic map is built around the robot for each position where images have been acquired. This map is a grid map augmented with semantic information in the form of probabilities that the occupied grid cells belong to a particular class. The local maps are fused into a global probabilistic semantic map covering the area along the trajectory of the mobile robot.</p><p>In the third part information extracted from an aerial image is used to improve the mapping process. Region and object boundaries taken from the probabilistic semantic map are used to initialize segmentation of the aerial image. Algorithms for both local segmentation related to the borders and global segmentation of the entire aerial image, exemplified with the two classes ground and buildings, are presented. Ground-level semantic information allows focusing of the segmentation of the aerial image to desired classes and generation of a semantic map that covers a larger area than can be built using only the onboard sensors.</p>

In this thesis, semantic mapping is understood to be the process of putting a tag or label on objects or regions in a map. This label should be interpretable by and have a meaning for a human. The use of semantic information has several application areas in mobile robotics. The largest area is in human-robot interaction where the semantics is necessary for a common understanding between robot and human of the operational environment. Other areas include localization through connection of human spatial concepts to particular locations, improving 3D models of indoor and outdoor environments, and model validation. This thesis investigates the extraction of semantic information for mobile robots in outdoor environments and the use of semantic information to link ground-level occupancy maps and aerial images. The thesis concentrates on three related issues: i) recognition of human spatial concepts in a scene, ii) the ability to incorporate semantic knowledge in a map, and iii) the ability to connect information collected by a mobile robot with information extracted from an aerial image. The first issue deals with a vision-based virtual sensor for classification of views (images). The images are fed into a set of learned virtual sensors, where each virtual sensor is trained for classification of a particular type of human spatial concept. The virtual sensors are evaluated with images from both ordinary cameras and an omni-directional camera, showing robust properties that can cope with variations such as changing season. In the second part a probabilistic semantic map is computed based on an occupancy grid map and the output from a virtual sensor. A local semantic map is built around the robot for each position where images have been acquired. This map is a grid map augmented with semantic information in the form of probabilities that the occupied grid cells belong to a particular class. The local maps are fused into a global probabilistic semantic map covering the area along the trajectory of the mobile robot. In the third part information extracted from an aerial image is used to improve the mapping process. Region and object boundaries taken from the probabilistic semantic map are used to initialize segmentation of the aerial image. Algorithms for both local segmentation related to the borders and global segmentation of the entire aerial image, exemplified with the two classes ground and buildings, are presented. Ground-level semantic information allows focusing of the segmentation of the aerial image to desired classes and generation of a semantic map that covers a larger area than can be built using only the onboard sensors.

Essa pesquisa trata do crescimento urbano e implicações ambientais de uma área na bacia do Guarapiranga. Essa área abrange parte dos municípios de São Paulo, Itapecerica da Serra e Embu Guaçu. O intuito é apresentar a evolução urbana da área entre os anos 1962 e 2001, com análises de fotografias áreas. Em uma escala intra-urbana, a pesquisa apresenta a interpretação de fotografias áreas para o ano de 2001 criando a carta de áreas homogêneas urbanas. Os principais dados dessa pesquisa são para o ano de 2001, de modo que pudemos os indicadores sócio-demográficos dos dados do censo do IBGE realizado para o ano de 2000.Com todos esses dados, podemos apontar os vetores de crescimento urbano da área, assim como seus problemas sócio-ambientais. / This research is about the urban growing in an area wich is considered model in the Guarapiranga´s Basin. This area encloses the citys of São Paulo, Itapecerica da Serra and Embu Guaçu. The intention is present the urban evolution of the area between the years 1962 and 2001, with analyses of aerial photography. In intra urban scale, the research presents the aerial photography\'s urban interpretation for the year 2001, making the maps of the urban homogenous zones areas. The main data or this research is about the year 2001, in the way that we could have the social demography in the IBGE´s censo data for the year 2000. With all this data, we have got the growing expansion city vectors as also the social and ambient problems.

Highway assets, including traffic signs, traffic signals, light poles, and guardrails, are important components of transportation networks. They guide, warn and protect drivers, and regulate traffic. To manage and maintain the regular operation of the highway system, state departments of transportation (DOTs) need reliable and up-to-date information about the location and condition of highway assets. Different methodologies have been employed to collect road inventory data. Currently, ground-based technologies are widely used to help DOTs to continually update their road database, while air-based methods are not commonly used. One possible reason is that the initial investment for air-based methods is relatively high; another is the lack of a systematic and effective approach to extract road features from raw airborne light detection and ranging (LiDAR) data and aerial image data. However, for large-area inventories (e.g., a whole state highway inventory), the total cost of using aerial mapping is actually much lower than other methods considering the time and personnel needed. Moreover, unmanned aerial vehicles (UAVs) are easily accessible and inexpensive, which makes it possible to reduce costs for aerial mapping. The focus of this project is to analyze the capability and strengths of airborne data collection system in highway inventory data collection. In this research, a field experiment was conducted by the Remote Sensing Service Laboratory (RSSL), Utah State University (USU), to collect airborne data. Two kinds of methodologies were proposed for data processing, namely ArcGIS-based algorithm for airborne LiDAR data, and MATLAB-based procedure for aerial photography. The results proved the feasibility and high efficiency of airborne data collection method for updating highway inventory database.

One of the main challenges of the integration of unmanned aerial vehicles (UAVs) into today’s society is the risk of in-flight failures, such as motor failure, occurring in populated areas that can result in catastrophic accidents. We propose a framework to manage the consequences of an in-flight system failure and to bring down the aircraft safely without causing any serious accident to people, property, and the UAV itself. This can be done in three steps: a) Detecting a failure, b) Finding a safe landing spot, and c) Navigating the UAV to the safe landing spot. In this thesis, we will look at part b. Specifically, we are working to develop an active system that can detect landing sites autonomously without any reliance on UAV resources. To detect a safe landing site, we are using a deep learning algorithm named "You Only Look Once" (YOLO) that runs on a Jetson Xavier NX computing module, which is connected to a camera, for image processing. YOLO is trained using the DOTA dataset and we show that it can detect landing spots and obstacles effectively. Then by avoiding the detected objects, we find a safe landing spot. The effectiveness of this algorithm will be shown first by comprehensive simulations. We also plan to experimentally validate this algorithm by flying a UAV and capturing ground images, and then applying the algorithm in real-time to see if it can effectively detect acceptable landing spots.

Optical lithography is an indispensible step in the process flow of Design for Manufacturability (DFM). Optical lithography simulation is a compute intensive task and simulation performance, or lack thereof can be a determining factor in time to market. Thus, the efficiency of lithography simulation is of paramount importance. Coherent decomposition is a popular simulation technique for aerial imaging simulation. In this thesis, we propose an approximate simulation technique based on the 2D wavelet transform and use a number of optimization methods to further improve polygon edge detection. Results show that the proposed method suffers from an average error of less than 6% when compared with the coherent decomposition method. The benefits of the proposed method are (i) > 20X increase in performance and more importantly (ii) it allows very large circuits to be simulated while some commercial tools are severely capacity limited and cannot even simulate a circuit as small as ISCAS-85 benchmark C17. Approximate simulation is quite attractive for layout optimization where it may be used in a loop and may even be acceptable for final layout verification.

Modern semiconductor manufacturing requires photolithographic printing of subillumination wavelength features in photoresist via electromagnetic energy scattered by complicated photomask designs. This results in aerial images which are subject to constructive and destructive wave interference, as well as electromagnetic resonances in the photomask features. This thesis proposes a 3-D full-wave frequency domain nonconformal Octree mesh based Finite Element Method (OFEM) electromagnetic scattering solver in combination with Fourier Optics to accurately simulate the entire projection photolithography system, from illumination source to final image intensity in the photoresist layer. A rapid 1-irregular octree based geometry model mesher is developed and shown to perform remarkably well compared to a tetrahedral mesher. A special set of nonconformal 1st and 2nd order hierarchal OFEM basis functions is presented, and 1st order numerical results show good performance compared to tetrahedral FEM. Optical and modern photomask phenomenology is examined, including optical proximity correction (OPC) with thick PEC metal layer, and chromeless phase inversion (PI) masks.

Cette thèse aborde des problèmes liés à la localisation et reconnaissance d'objets urbains dans des images aériennes de très haute définition. Les objets urbains se caractérisent par une représentation très variable en terme de forme, texture et couleur. De plus, ils sont présents de multiples fois sur les images à analyser et peuvent être collés les uns aux autres. Pour effectuer la localisation et reconnaissance automatiquement des différents objets nous proposons d'utiliser des approches d'apprentissage supervisé. De part leurs caractéristiques, les objets urbains sont difficilement détectables et les approches classiques de détections n'offrent pas de performances satisfaisantes. Nous avons proposé l'utilisation d'un réseau de séparateurs à vaste marge (SVM) afin de mieux fusionner les informations issues des différentes résolutions et donc d'améliorer la représentativité de l'objet urbain. L'utilisation de réseau de SVM permet d'améliorer les performances mais à un coût calculatoire important. Nous avons alors proposé d'utiliser un chemin d'activation permettant de réduire la complexité sans perdre en efficacité. Ce chemin va activer le réseau de manière séquentielle et stoppera l'exploration lorsque la probabilité de détection d'un objet est importante. Dans le cas d'une localisation basée sur l'extraction de caractéristiques puis la classification, la réduction calculatoire est d'un facteur cinq. Par la suite, nous avons montré que nous pouvons combiner le réseau de SVM avec les cartes de caractéristiques issues de réseaux de neurones convolutifs. Cette architecture combinée avec le chemin d'activation permet une réduction théorique du coût d'activation pouvant aller jusqu'à 97% avec un gain de performances d'environ 8% sur les données utilisées. Les méthodes développées ont pour objectif d'être intégrées dans un logiciel de la société Berger-Levrault afin de faciliter et d'améliorer la gestion de cadastre dans les collectivités locales. / This thesis deals with the problems of automatic localization and recognition of urban objects in high-definition aerial images. Urban object detection is a challenging problem because they vary in appearance, color and size. Moreover, there are many urban objects which can be very close to each other in an image. The localization and the automatic recognition of different urban objects, considering these characteristics, are very difficult to detect and classical image processing algorithms do not lead to good performances. We propose then to use the supervised learning approach. In a first time, we have built a Support Vector Machine (SVM) network to merge different resolutions in an efficient way. However, this method highly increases the computational cost. We then proposed to use an “activation path” which reduces the complexity without any loss of efficiency. This path activates sequentially the network and stops the exploration when an urban object has a high probability of detection. In the case of localizations based on a feature extraction step followed by a classification step, this may reduce by a factor 5 the computational cost. Thereafter, we show that we can combine an SVM network with feature maps which have been extracted by a Convolutional Neural Network. Such an architecture associated with the activation path increased the performance by 8% on our database while giving a theoretical reduction of the computational costs up to 97%. We implemented all these new methods in order to be integrated in the software framework of Berger-Levrault company, to improve land registry for local communities.

This work investigates the landscape of aerial image stereo matching (AISM) methods suitable for large scale forest variable estimation. AISM methods are an important source of remotely collected information used in modern forestry to keep track of a growing forest's condition. A total of 17 AISM methods are investigated, out of which 4 are evaluated by processing a test data set consisting of three aerial images. The test area is located in southern Sweden, consisting of mainly Norway Spruce and Scots Pine. From the resulting point clouds and height raster images, a total of 30 different metrics of both height and density types are derived. Linear regression is used to fit functions from metrics derived from AISM data to a set of forest variables including tree height (HBW), tree diameter (DBW), basal area, volume. As ground truth, data collected by dense airborne laser scanning is used. Results are presented as RMSE and standard deviation concluded from the linear regression. For tree height, tree diameter, basal area, volume the RMSE ranged from 7.442% to 10.11%, 11.58% to 13.96%, 32.01% to 35.10% and 34.01% to 38.26% respectively. The results concluded that all four tested methods achieved comparable estimation quality although showing small differences among them. Keystone and SURE performed somewhat better while MicMac placed third and Photoscan achieved the less accurate result.