People describe and explore space with a strong emphasis on the visual senses, yet modelling the field of view has received little attention within the realm of Location Based Services (LBS), in part due to the lack of useful data. Advances in data capture, such as Light Detection and Ranging (LiDAR), provide new opportunities to build digital city models and expand the range of applications which use visibility analysis. This thesis capitalises on these advances with the development of a visibility model to support a number of innovative LBS functions in an urban region and particular focus is given to the visibility model‟s supporting role in the formation of referring expressions, the descriptive phrases used to identify objects in a scene, which are relevant when delivering spatial information to the user through a speech based interface. Speech interfaces are particularly useful to mobile users with restricted screen viewing opportunities, such as navigational support for motorists and a wider range of tasks including delivering information to urban pedestrians. As speech recognition accuracies improve so new interaction opportunities will allow users to relate to their surroundings and retrieve information on buildings in view through spoken descriptions. The papers presented in this thesis work towards this goal, by translating spatial information into a form which matches the user‟s perspective and can be delivered over a speech interface. The foundation is the development of a new visual exposure model for use in urban areas, able to calculate a number of metrics about Features of Interest (FOIs), including the façade area visible and the percentage on the skyline. The impact of urban vegetation as a semi-permeable visual barrier is also considered, and how visual exposure calculations may be adjusted to accommodate under canopy and through canopy views. The model may be used by pedestrian LBSs, or applied to vehicle navigation tasks to determine how much of a route ahead is in view for a car driver, identifying the sections with limited visibility or the best places for an overtaking manoeuvre. Delivering information via a speech interface requires FOI positions to be defined according to projective space relating to the user‟s viewpoint, rather than topological or metric space, and this is handled using a new egocentric model. Finally descriptions of the FOIs are considered, including a method to automatically collect façade colours by excluding foreground objects, and a model to determine the most appropriate description to direct the LBS user‟s attention to a FOI in view.

Les modèles numériques de terrain, cas particulier de modèles numériques de surfaces, n'ont pas la même erreur quadratique moyenne en planimétrie qu'en altimétrie. Différentes solutions ont été envisagées pour déterminer séparément l'erreur en altimétrie et l'erreur planimétrique, disposant, bien entendu, d'un modèle numérique plus précis comme référence. La démarche envisagée consiste à déterminer les paramètres des ellipsoïdes d'erreur, centrées dans la surface de référence. Dans un premier temps, l'étude a été limitée aux profils de référence avec l'ellipse d'erreur correspondante. Les paramètres de cette ellipse sont déterminés à partir des distances qui séparent les tangentes à l'ellipse du centre de cette même ellipse. Remarquons que cette distance est la moyenne quadratique des distances qui séparent le profil de référence des points du modèle numérique à évaluer, c'est à dire la racine de la variance marginale dans la direction normale à la tangente. Nous généralisons à l'ellipsoïde de révolution. C'est le cas ou l'erreur planimétrique est la même dans toutes les directions du plan horizontal (ce n'est pas le cas des MNT obtenus, par exemple, par interférométrie radar). Dans ce cas nous montrons que le problème de simulation se réduit à l'ellipse génératrice et la pente du profil correspondant à la droite de pente maximale du plan appartenant à la surface de référence. Finalement, pour évaluer les trois paramètres d'un ellipsoïde, cas où les erreurs dans les directions des trois axes sont différentes (MNT obtenus par Interférométrie SAR), la quantité des points nécessaires pour la simulation doit être importante et la surface tr ès accidentée. Le cas échéant, il est difficile d'estimer les erreurs en x et en y. Néanmoins, nous avons remarqué, qu'il s'agisse de l'ellipsoïde de révolution ou non, que dans tous les cas, l'estimation de l'erreur en z (altimétrie) donne des résultats tout à fait satisfaisants. / A Digital Surface Model (DSM) is a numerical surface model which is formed by a set of points, arranged as a grid, to study some physical surface, Digital Elevation Models (DEM), or other possible applications, such as a face, or some anatomical organ, etc. The study of the precision of these models, which is of particular interest for DEMs, has been the object of several studies in the last decades. The measurement of the precision of a DSM model, in relation to another model of the same physical surface, consists in estimating the expectancy of the squares of differences between pairs of points, called homologous points, one in each model which corresponds to the same feature of the physical surface. But these pairs are not easily discernable, the grids may not be coincident, and the differences between the homologous points, corresponding to benchmarks in the physical surface, might be subject to special conditions such as more careful measurements than on ordinary points, which imply a different precision. The generally used procedure to avoid these inconveniences has been to use the squares of vertical distances between the models, which only address the vertical component of the error, thus giving a biased estimate when the surface is not horizontal. The Perpendicular Distance Evaluation Method (PDEM) which avoids this bias, provides estimates for vertical and horizontal components of errors, and is thus a useful tool for detection of discrepancies in Digital Surface Models (DSM) like DEMs. The solution includes a special reference to the simplification which arises when the error does not vary in all horizontal directions. The PDEM is also assessed with DEM's obtained by means of the Interferometry SAR Technique

Building information extraction and reconstruction from satellite images is an essential task for many applications related to 3D city modeling, planning, disaster management, navigation, and decision-making. Building information can be obtained and interpreted from several data, like terrestrial measurements, airplane surveys, and space-borne imagery. However, the latter acquisition method outperforms the others in terms of cost and worldwide coverage: Space-borne platforms can provide imagery of remote places, which are inaccessible to other missions, at any time. Because the manual interpretation of high-resolution satellite image is tedious and time consuming, its automatic analysis continues to be an intense field of research. At times however, it is difficult to understand complex scenes with dense placement of buildings, where parts of buildings may be occluded by vegetation or other surrounding constructions, making their extraction or reconstruction even more difficult. Incorporation of several data sources representing different modalities may facilitate the problem. The goal of this dissertation is to integrate multiple high-resolution remote sensing data sources for automatic satellite imagery interpretation with emphasis on building information extraction and refinement, which challenges are addressed in the following: Building footprint extraction from Very High-Resolution (VHR) satellite images is an important but highly challenging task, due to the large diversity of building appearances and relatively low spatial resolution of satellite data compared to airborne data. Many algorithms are built on spectral-based or appearance-based criteria from single or fused data sources, to perform the building footprint extraction. The input features for these algorithms are usually manually extracted, which limits their accuracy. Based on the advantages of recently developed Fully Convolutional Networks (FCNs), i.e., the automatic extraction of relevant features and dense classification of images, an end-to-end framework is proposed which effectively combines the spectral and height information from red, green, and blue (RGB), pan-chromatic (PAN), and normalized Digital Surface Model (nDSM) image data and automatically generates a full resolution binary building mask. The proposed architecture consists of three parallel networks merged at a late stage, which helps in propagating fine detailed information from earlier layers to higher levels, in order to produce an output with high-quality building outlines. The performance of the model is examined on new unseen data to demonstrate its generalization capacity. The availability of detailed Digital Surface Models (DSMs) generated by dense matching and representing the elevation surface of the Earth can improve the analysis and interpretation of complex urban scenarios. The generation of DSMs from VHR optical stereo satellite imagery leads to high-resolution DSMs which often suffer from mismatches, missing values, or blunders, resulting in coarse building shape representation. To overcome these problems, a methodology based on conditional Generative Adversarial Network (cGAN) is developed for generating a good-quality Level of Detail (LoD) 2 like DSM with enhanced 3D object shapes directly from the low-quality photogrammetric half-meter resolution satellite DSM input. Various deep learning applications benefit from multi-task learning with multiple regression and classification objectives by taking advantage of the similarities between individual tasks. Therefore, an observation of such influences for important remote sensing applications such as realistic elevation model generation and roof type classification from stereo half-meter resolution satellite DSMs, is demonstrated in this work. Recently published deep learning architectures for both tasks are investigated and a new end-to-end cGAN-based network is developed, which combines different models that provide the best results for their individual tasks. To benefit from information provided by multiple data sources, a different cGAN-based work-flow is proposed where the generative part consists of two encoders and a common decoder which blends the intensity and height information within one network for the DSM refinement task. The inputs to the introduced network are single-channel photogrammetric DSMs with continuous values and pan-chromatic half-meter resolution satellite images. Information fusion from different modalities helps in propagating fine details, completes inaccurate or missing 3D information about building forms, and improves the building boundaries, making them more rectilinear. Lastly, additional comparison between the proposed methodologies for DSM enhancements is made to discuss and verify the most beneficial work-flow and applicability of the resulting DSMs for different remote sensing approaches.

Automatic three-dimensional (3D) building model reconstruction using remote sensing data is crucial in applications which require large-scale and frequent building model updates, such as disaster monitoring and urban management, to avoid huge manual efforts and costs. Recent advances in the availability of very high-resolution satellite data together with efficient data acquisition and large area coverage have led to an upward trend in their applications for 3D building model reconstructions. In this dissertation, a novel multistage hybrid automatic 3D building model reconstruction approach is proposed which reconstructs building models in level of details 2 (LOD2) based on digital surface model (DSM) data generated from the very high-resolution stereo imagery of the WorldView-2 satellite. This approach uses DSM data in combination with orthorectified panchromatic (PAN) and pan-sharpened data of multispectral satellite imagery to overcome the drawbacks of DSM data, such as blurred building boundaries, rough building shapes unwanted failures in the roof geometries. In the first stage, the rough building boundaries in the DSM-based building masks are refined by classifying the geometrical features of the corresponding PAN images. The refined boundaries are then simplified in the second stage through a parameterization procedure which represents the boundaries by a set of line segments. The main orientations of buildings are then determined, and the line segments are regularized accordingly. The regularized line segments are then connected to each other based on a rule-based method to form polygonal building boundaries. In the third stage, a novel technique is proposed to decompose the building polygons into a number of rectangles under the assumption that buildings are usually composed of rectangular structures. In the fourth stage, a roof model library is defined, which includes flat, gable, half-hip, hip, pyramid and mansard roofs. These primitive roof types are then assigned to the rectangles based on a deep learning-based classification method. In the fifth stage, a novel approach is developed to reconstruct watertight parameterized 3D building models based on the results of the previous stages and normalized DSM (nDSM) of satellite imagery. In the final stage, a novel approach is proposed to optimize building parameters based on an exhaustive search, so that the two-dimensional (2D) distance between the 3D building models and the building boundaries (obtained from building masks and PAN image) as well as the 3D normal distance between the 3D building models and the 3D point clouds (obtained from nDSM) are minimized. Different parts of the building blocks are then merged through a newly proposed intersection and merging process. All corresponding experiments were conducted on four areas of the city of Munich including 208 buildings and the results were evaluated qualitatively and quantitatively. According to the results, the proposed approach could accurately reconstruct 3D models of buildings, even the complex ones with several inner yards and multiple orientations. Furthermore, the proposed approach provided a high level of automation by the limited number of primitive roof model types required and by performing automatic parameter initialization. In addition, the proposed boundary refinement method improved the DSM-based building masks specified by 8 % in area accuracy. Furthermore, the ridge line directions and roof types were detected accurately for most of the buildings. The combination of the first three stages improved the accuracy of the building boundaries by 70 % in comparison to using line segments extracted from building masks without refinement. Moreover, the proposed optimization approach could achieve in most cases the best combinations of 2D and 3D geometrical parameters of roof models. Finally, the intersection and merging process could successfully merge different parts of the complex building models.