The resolutions of acquired image and volume data are ever increasing. However, the resolutions of commodity display devices remain limited. This leads to an increasing gap between data and display resolutions. To bridge this gap, the standard approach is to employ output-sensitive operations on multi-resolution data representations. Output-sensitive operations facilitate interactive applications since their required computations are proportional only to the size of the data that is visible, i.e., the output, and not the full size of the input. Multi-resolution representations, such as image mipmaps, and volume octrees, are crucial in providing these operations direct access to any subset of the data at any resolution corresponding to the output. Despite its widespread use, this standard approach has some shortcomings in three important application areas, namely non-linear image operations, multi-resolution volume rendering, and large-scale image exploration. This dissertation presents new multi-resolution representations for large-scale images and volumes that address these shortcomings. Standard multi-resolution representations require low-pass pre-filtering for anti- aliasing. However, linear pre-filters do not commute with non-linear operations. This becomes problematic when applying non-linear operations directly to any coarse resolution levels in standard representations. Particularly, this leads to inaccurate output when applying non-linear image operations, e.g., color mapping and detail-aware filters, to multi-resolution images. Similarly, in multi-resolution volume rendering, this leads to inconsistency artifacts which manifest as erroneous differences in rendering outputs across resolution levels. To address these issues, we introduce the sparse pdf maps and sparse pdf volumes representations for large-scale images and volumes, respectively. These representations sparsely encode continuous probability density functions (pdfs) of multi-resolution pixel and voxel footprints in input images and volumes. We show that the continuous pdfs encoded in the sparse pdf map representation enable accurate multi-resolution non-linear image operations on gigapixel images. Similarly, we show that sparse pdf volumes enable more consistent multi-resolution volume rendering compared to standard approaches, on both artificial and real world large-scale volumes. The supplementary videos demonstrate our results. In the standard approach, users heavily rely on panning and zooming interactions to navigate the data within the limits of their display devices. However, panning across the whole spatial domain and zooming across all resolution levels of large-scale images to search for interesting regions is not practical. Assisted exploration techniques allow users to quickly narrow down millions to billions of possible regions to a more manageable number for further inspection. However, existing approaches are not fully user-driven because they typically already prescribe what being of interest means. To address this, we introduce the patch sets representation for large-scale images. Patches inside a patch set are grouped and encoded according to similarity via a permutohedral lattice (p-lattice) in a user-defined feature space. Fast set operations on p-lattices facilitate patch set queries that enable users to describe what is interesting. In addition, we introduce an exploration framework—GigaPatchExplorer—for patch set-based image exploration. We show that patch sets in our framework are useful for a variety of user-driven exploration tasks in gigapixel images and whole collections thereof.

Inverse problems, foundational in applied sciences, involve deducing system inputs from specific output observations. These problems find applications in diverse domains such as aerospace engineering, weather prediction, and oceanography. However, their solution often requires complex numerical simulations and substantial computational resources. Modern machine learning based approaches have emerged as an alternative and flexible methodology for solving these types of problems, however their generalization power often comes at the cost of working with large descriptive datasets, a requirement that many applications cannot afford. This thesis proposes and explores the novel Deep Multi-resolution Operator Network (DMON), inspired by the recently developed DeepONet architecture. The DMON model is designed to solve inverse problems related to chaotic non-linear systems with low-resolution data through intelligently utilizing high-resolution data from a similar system. Performance of the DMON model and the proposed selection mechanisms are evaluated on two chaotic systems, a double pendulum and turbulent flow around a cylinder, with improvements observed under idealized scenarios whereby high and low-resolution inputs are manually paired, along with minor improvements when this pairing is conducted through the proposed the latent space comparison selection mechanism. / Master of Science / In everyday life, we often encounter the challenge of determining the cause behind something we observe. For instance, meteorologists infer weather patterns based on limited atmospheric data, while doctors use X-rays and CT scans to reconstruct images representing the insides of our bodies. Solving these so called ``inverse problems'' can be difficult, particularly when the process is chaotic such as the weather, whereby small changes result in much larger ones over time. In this thesis, we propose a novel method using artificial intelligence and high-resolution simulation data to aid in solving these types of problems. Our proposed method is designed to work well even when we only have access to a small amount of information, or the information available isn't very detailed. Because of this there are potential applications of the proposed method across a wide range of fields, particularly those where acquiring detailed information is difficult, expensive, or impossible.

<p> It is possible to interpret multi-resolution analysis from both Fourier-domain and temporal/spatial domain stand-points. While a Fourier-domain interpretation helps in designing a powerful machinery for multi-resolution refinement on regular point-sets and lattices, most of its techniques cannot be directly generalized to the case of irregular sampling. Therefore, in this thesis we provide a new definition and formulation of multi-resolution refinement, based on a temporal/spatial-domain understanding, that is general enough to allow multi-resolution approximation of different spaces of functions by processing samples (or observations) that can be irregularly distributed or even obtained using different sampling methods. We then continue to provide a construction for designing and implementing classes of refinement schemes in these general settings. The framework for multi-resolution refinement that we discuss includes and extends the existing mathematical machinery for multi-resolution analysis; and the suggested construction unifies many of the schemes currently in use, and, more importantly, allows designing schemes for many new settings. </p> / Thesis / Master of Applied Science (MASc)

O trabalho apresentado por essa dissertação busca contribuir com a atenuação do problema da correspondência em visão estéreo a partir de uma abordagem local de soluções. São usadas duas estratégias como solução às ambigüidades e às oclusões da cena: a análise multi-resolução das imagens empregando a estrutura piramidal, e a força de agrupamento perceptual, conhecida como Gestalt theory na psicologia. Inspirado no sistema visual humano, a visão estéreo é uma área de grande interesse em visão computacional, e está relacionada à recuperação de informações tridimensionais de uma cena a partir de imagens da mesma. Para isso, as imagens são capturadas em posições diferentes para o futuro relacionamento das várias projeções de um mesmo ponto 3D. Apesar de ser estudada há quase quatro décadas, ela ainda apresenta problemas de difícil solução devido às dificuldades relacionadas às distorções produzidas pela mudança da perspectiva de visualização. Dentre esses problemas destacam-se os relacionados à oclusão de pontos e também à ambigüidade gerada pela repetição ou ausência de textura nas imagens. Esses por sua vez compõem a base do problema estéreo, chamado de problema da correspondência. Os resultados obtidos são equivalentes aos obtidos por técnicas globais, com a vantagem de requerer menor complexidade computacional. O uso da teoria de agrupamento perceptual faz desse trabalho um método moderno de estimação de disparidades, visto que essa técnica é alvo de atenção especial em recentes estudos na área de visão computacional. / This work aims to give a contribution to the correspondence problem using a local approach. Two strategies are used as solution to ambiguities and occlusions: the multi-resolution analysis with irnages pyramids and the other is the perceptual grouping weight, called Gestalt theory in the psychology. Inspired by human vision system, the stereo vision is an very important area in computer vision. It is related with the 3D information recovery from a pair of images. The images are captured frorn different positions to hereafter association of the 3D point projections. Although it has being studied for quite a long time, stereo vision presents some difficult problems, related to the change of visualisation perspective. Among the different problems originated from point of view changes, occlusions and ambiguities have special attention and compose the foundation of stereo problem, named correspondence problem. The results obtained were closer to the ones generated by global techniques, with the advantage of requiring less computational complexity. The use of Gestalt theory makes this a modern disparity estimation method, as this theory has been received special attention in computer vision researchs.

Multi-resolution visualization (MRV) systems are widely used for handling large amounts of information. These systems look different but they share many common features. The visualization research community lacks a general framework that summarizes the common features among the wide variety of MRV systems in order to help in MRV system design, analysis, and enhancement. This dissertation proposes such a general framework. This framework is based on the definition that a MRV system is a visualization system that visually represents perceptions in different levels of detail and allows users to interactively navigate among the representations. The visual representations of a perception are called a view. The framework is composed of two essential components: view simulation and interactive visualization. View simulation means that an MRV system simulates views of non-existing perceptions through simplification on the data structure or the graphics generation process. This is needed when the perceptions provided to the MRV system are not at the user's desired level of detail. The framework identifies classes of view simulation approaches and describes them in terms of simplification operators and operands (spaces). The simplification operators are further divided into four categories, namely sampling operators, aggregation operators, approximation operators, and generalization operators. Techniques in these categories are listed and illustrated via examples. The simplification operands (spaces) are also further divided into categories, namely data space and visualization space. How different simplification operators are applied to these spaces is also illustrated using examples. Interactive visualization means that an MRV system visually presents the views to users and allows users to interactively navigate among different views or within one view. Three types of MRV interface, namely the zoomable interface, the overview + context interface, and the focus + detail interface, are presented with examples. Common interaction tools used in MRV systems, such as zooming and panning, selection, distortion, overlap reduction, previewing, and dynamic simplification are also presented. A large amount of existing MRV systems are used as examples in this dissertation, including several MRV systems developed by the author based on the general framework. In addition, a case study that analyzes and suggests possible improvements for an existing MRV system is described. These examples and the case study reveal that the framework covers the common features of a wide variety of existing MRV systems, and helps users analyze and improve existing MRV systems as well as design new MRV systems.

O trabalho apresentado por essa dissertação busca contribuir com a atenuação do problema da correspondência em visão estéreo a partir de uma abordagem local de soluções. São usadas duas estratégias como solução às ambigüidades e às oclusões da cena: a análise multi-resolução das imagens empregando a estrutura piramidal, e a força de agrupamento perceptual, conhecida como Gestalt theory na psicologia. Inspirado no sistema visual humano, a visão estéreo é uma área de grande interesse em visão computacional, e está relacionada à recuperação de informações tridimensionais de uma cena a partir de imagens da mesma. Para isso, as imagens são capturadas em posições diferentes para o futuro relacionamento das várias projeções de um mesmo ponto 3D. Apesar de ser estudada há quase quatro décadas, ela ainda apresenta problemas de difícil solução devido às dificuldades relacionadas às distorções produzidas pela mudança da perspectiva de visualização. Dentre esses problemas destacam-se os relacionados à oclusão de pontos e também à ambigüidade gerada pela repetição ou ausência de textura nas imagens. Esses por sua vez compõem a base do problema estéreo, chamado de problema da correspondência. Os resultados obtidos são equivalentes aos obtidos por técnicas globais, com a vantagem de requerer menor complexidade computacional. O uso da teoria de agrupamento perceptual faz desse trabalho um método moderno de estimação de disparidades, visto que essa técnica é alvo de atenção especial em recentes estudos na área de visão computacional. / This work aims to give a contribution to the correspondence problem using a local approach. Two strategies are used as solution to ambiguities and occlusions: the multi-resolution analysis with irnages pyramids and the other is the perceptual grouping weight, called Gestalt theory in the psychology. Inspired by human vision system, the stereo vision is an very important area in computer vision. It is related with the 3D information recovery from a pair of images. The images are captured frorn different positions to hereafter association of the 3D point projections. Although it has being studied for quite a long time, stereo vision presents some difficult problems, related to the change of visualisation perspective. Among the different problems originated from point of view changes, occlusions and ambiguities have special attention and compose the foundation of stereo problem, named correspondence problem. The results obtained were closer to the ones generated by global techniques, with the advantage of requiring less computational complexity. The use of Gestalt theory makes this a modern disparity estimation method, as this theory has been received special attention in computer vision researchs.

The low-level interpretation of images provides constraints on 3D surface shape at multiple resolutions, but typically only at scattered locations over the visual field. Subsequent visual processing can be facilitated substantially if the scattered shape constraints are immediately transformed into visible-surface representations that unambiguously specify surface shape at every image point. The required transformation is shown to lead to an ill-posed surface reconstruction problem. A well-posed variational principle formulation is obtained by invoking 'controlled continuity,' a physically nonrestrictive (generic) assumption about surfaces which is nonetheless strong enough to guarantee unique solutions. The variational principle, which admits an appealing physical interpretation, is locally discretized by applying the finite element method to a piecewise, finite element representation of surfaces. This forms the mathematical basis of a unified and general framework for computing visible-surface representations. The computational framework unifies formal solutions to the key problems of (i) integrating multiscale constraints on surface depth and orientation from multiple visual sources, (ii) interpolating these scattered constraints into dense, piecewise smooth surfaces, (iii) discovering surface depth and orientation discontinuities and allowing them to restrict interpolation appropriately, and (iv) overcoming the immense computational burden of fine resolution surface reconstruction. An efficient surface reconstruction algorithm is developed. It exploits multiresolution hierarchies of cooperative relaxation processes and is suitable for implementation on massively parallel networks of simple, locally interconnected processors. The algorithm is evaluated empirically in a diversity of applications.

Multi-resolution analysis has been a very popular technique in the recent years. Wavelets have been used extensively to perform multi resolution image expansion and analysis. DCT, however, has been used to compress image but not for multi resolution image analysis. This thesis is an attempt to explore the possibilities of using DCT for multi-resolution image analysis. Naive implementation of block DCT for multi-resolution expansion has many difficulties that lead to signal distortion. One of the main causes of distortion is the blocking artifacts that appear when reconstructing images transformed by DCT. The new algorithm is based on line DCT which eliminates the need for block processing. The line DCT is one dimensional array based on cascading the image rows and columns in one transform operation. Several images have been used to test the algorithm at various resolution levels. The reconstruction mean square error rate is used as an indication to the success of the method. The proposed algorithm has also been tested against the traditional block DCT.

The rise in digital technology has also rose the use of digital images. The digital imagesrequire much storage space. The compression techniques are used to compress the dataso that it takes up less storage space. In this regard wavelets play important role. Inthis thesis, we studied the Haar wavelet system, which is a complete orthonormal systemin L2(R): This system consists of the functions j the father wavelet, and y the motherwavelet. The Haar wavelet transformation is an example of multiresolution analysis. Ourpurpose is to use the Haar wavelet basis to compress an image data. The method ofaveraging and differencing is used to construct the Haar wavelet basis. We have shownthat averaging and differencing method is an application of Haar wavelet transform. Afterdiscussing the compression by using Haar wavelet transform we used another method tocompress that is based on singular value decomposition. We used mathematical softwareMATLAB to compress the image data by using Haar wavelet transformation, and singularvalue decomposition.

Distributed three-dimensional (3D) graphics applications exhibit both resemblance and uniqueness in comparison with conventional streaming media applications. The resemblance relates to the large data volume and the bandwidth-limited and error-prone transmission channel. The uniqueness is due to the polygon-based representation of 3D geometric meshes and their accompanying attributes such as textures. This specific data format introduces sophisticated rendering computation to display graphics models and therefore places an additional constraint on the streaming application. The objective of this research is to provide scalable, error-resilient, and time-efficient solutions for high-quality 3D graphics applications in distributed and resource-constrained environments. Resource constraints range from rate-limited and error-prone channels to insufficient data-reception, computing, and display capabilities of client devices. Optimal resource treatment with transmission and rendering scalability is important under such circumstances. The proposed research consists of three milestones. In the first milestone, we develop a joint mesh and texture optimization framework for scalable transmission and rendering of textured 3D models. Then, we address network behaviors and develop a hybrid retransmission and error protection mechanism for the on-demand delivery of 3D models. Next, we advance from individual 3D models to 3D scene databases, which contain numerous objects interacting in one geometric space, and study joint application and transport approaches. By properly addressing the properties of 3D scenes represented in multi-resolution hierarchies, we develop a joint source and channel coding method and a multi-streaming framework for streaming the content-rich 3D scene databases toward optimized transmission and rendering scalability under resource constraints.