Advancement in high-throughput microscopy technology such as the Knife-Edge Scanning Microscopy (KESM) is enabling the production of massive amounts of high-resolution and high-quality volumetric data of biological microstructures. To fully utilize these data, they should be efficiently distributed to the scientific research community through the Internet and should be easily visualized, annotated, and analyzed. Given the volumetric nature of the data, visualizing them in 3D is important. However, since we cannot assume that every end user has high-end hardware, an approach that has minimal hardware and software requirements will be necessary, such as a standard web browser running on a typical personal computer. There are several web applications that facilitate the viewing of large collections of images. Google Maps and Google Maps-like interfaces such as Brainmaps.org allow users to pan and zoom 2D images efficiently. However, they do not yet support the rendering of volumetric data in their standard web interface. The goal of this thesis is to develop a light-weight volumetric image viewer using existing web technologies such as HTML, CSS and JavaScript while exploiting the properties of stereo vision to facilitate the viewing and annotations of volumetric data. The choice of stereogram over other techniques was made since it allows the usage of raw image stacks produced by the 3D microscope without any extra computation on the data at all. Operations to generate stereo images using 2D image stacks include distance attenuation and binocular disparity. By using HTML and JavaScript that are computationally cheap, we can accomplish both tasks dynamically in a standard web browser, by overlaying the images with intervening semi-opaque layers. The annotation framework has also been implemented and tested. In order for annotation to work in this environment, it should also be in the form of stereogram and should aid the merging of stereo pairs. The current technique allows users to place a mark (dot) on one image stack, and its projected position onto the other image stack is calculated dynamically on the client side. Other extra metadata such as textual descriptions can be entered by the user as well. To cope with the occlusion problem caused by changes in the z direction, the structure traced by the user will be displayed on the side, together with the data stacks. Using the same stereo-gram creation techniques, the traces made by the user is dynamically generated and shown as stereogram. We expect the approach presented in this thesis to be applicable to a broader scientific domain, including geology and meteorology.

With advances in imaging and communication systems, there is increased use of multi-dimensional images. Examples include multi-view image/video, hyperspectral image/video and dynamic volume imaging in CT/MRI/Ultrasound. These datasets consume even larger amounts of resources for transmission or storage compared to 2-D images. Hence, it is vital to have efficient compression methods for multi-dimensional images. In this dissertation, first, a JPEG2000 Part-2 compliant scheme is proposed for compressing multi-dimensional datasets for any dimension N>=3. Secondly, a novel view-compensated compression method is investigated for remote visualization of volumetric data. Experimental results indicate superior compression performance compared to state-of-the-art compression standards. Thirdly, a new scalable low complexity coder is designed that sacrifices some compression efficiency to get substantial gain in throughput. Potential use of the scalable low complexity coder is illustrated for two applications: Airborne video transmission and remote volume visualization.

Volume rendering is a research area within scientific visualisation, where images are computed from volumetric data sets for visual exploration. Such data sets are typically generated by Computer aided Tomography, Magnetic Resonance Imaging, Positron Emission Tomography or gained from simulations. The data sets are usually interpreted using optical models that assign optical properties to the volume and define the illumination and shading behaviour. Volume rendering techniques may be divided into three classes: object-order, image-order or hybrid methods. Image-order or ray casting methods shoot rays from the view plane into the volume and simulate the variation of light intensities along those rays. Object-order techniques traverse the volume data set and project each volume element onto the view plane. Hybrid volume rendering techniques combine these two approaches. A very popular object-order rendering method is called splatting. This technique traverses the volume data set and projects the optical properties of each volume element onto the view plane. This thesis consists of two parts. The first part introduces two new splatting methods, collectively called high-resolution splatting, which are based on standard splatting. Both high-resolution splatting methods correct errors of splatting by applying major modifications. We propose the first method, called fast high-resolution splatting, as an alternative to standard splatting. It may be used for quick previewing, since it is faster than standard splatting and the resulting images are significantly sharper. Our second method, called complete high-resolution splatting, improves the volume reconstruction, which results in images that are very close to those produced by ray casting methods. The second part of the thesis incorporates wavelet analysis into high-resolution splatting. Wavelet analysis is a mathematical theory that decomposes volumes into multi-resolution hierarchies, which may be used to find coherence within volumes. The combination of wavelets with the high-resolution splatting method has the two advantages. Firstly the extended splatting method, called high-resolution wavelet splatting, can be directly applied to wavelet transformed volume data sets without performing an inverse transform. Secondly when visualising wavelet compressed volumes, only a small fraction of the wavelet coefficients need to be projected. For all three versions of the new high-resolution splatting method, complexity analyses, comprehensive error and performance analyses as well as implementation details are discussed.

Volume rendering is a research area within scientific visualisation, where images are computed from volumetric data sets for visual exploration. Such data sets are typically generated by Computer aided Tomography, Magnetic Resonance Imaging, Positron Emission Tomography or gained from simulations. The data sets are usually interpreted using optical models that assign optical properties to the volume and define the illumination and shading behaviour. Volume rendering techniques may be divided into three classes: object-order, image-order or hybrid methods. Image-order or ray casting methods shoot rays from the view plane into the volume and simulate the variation of light intensities along those rays. Object-order techniques traverse the volume data set and project each volume element onto the view plane. Hybrid volume rendering techniques combine these two approaches. A very popular object-order rendering method is called splatting. This technique traverses the volume data set and projects the optical properties of each volume element onto the view plane. This thesis consists of two parts. The first part introduces two new splatting methods, collectively called high-resolution splatting, which are based on standard splatting. Both high-resolution splatting methods correct errors of splatting by applying major modifications. We propose the first method, called fast high-resolution splatting, as an alternative to standard splatting. It may be used for quick previewing, since it is faster than standard splatting and the resulting images are significantly sharper. Our second method, called complete high-resolution splatting, improves the volume reconstruction, which results in images that are very close to those produced by ray casting methods. The second part of the thesis incorporates wavelet analysis into high-resolution splatting. Wavelet analysis is a mathematical theory that decomposes volumes into multi-resolution hierarchies, which may be used to find coherence within volumes. The combination of wavelets with the high-resolution splatting method has the two advantages. Firstly the extended splatting method, called high-resolution wavelet splatting, can be directly applied to wavelet transformed volume data sets without performing an inverse transform. Secondly when visualising wavelet compressed volumes, only a small fraction of the wavelet coefficients need to be projected. For all three versions of the new high-resolution splatting method, complexity analyses, comprehensive error and performance analyses as well as implementation details are discussed.

Volume rendering is a research area within scientific visualisation, where images are computed from volumetric data sets for visual exploration. Such data sets are typically generated by Computer aided Tomography, Magnetic Resonance Imaging, Positron Emission Tomography or gained from simulations. The data sets are usually interpreted using optical models that assign optical properties to the volume and define the illumination and shading behaviour. Volume rendering techniques may be divided into three classes: object-order, image-order or hybrid methods. Image-order or ray casting methods shoot rays from the view plane into the volume and simulate the variation of light intensities along those rays. Object-order techniques traverse the volume data set and project each volume element onto the view plane. Hybrid volume rendering techniques combine these two approaches. A very popular object-order rendering method is called splatting. This technique traverses the volume data set and projects the optical properties of each volume element onto the view plane. This thesis consists of two parts. The first part introduces two new splatting methods, collectively called high-resolution splatting, which are based on standard splatting. Both high-resolution splatting methods correct errors of splatting by applying major modifications. We propose the first method, called fast high-resolution splatting, as an alternative to standard splatting. It may be used for quick previewing, since it is faster than standard splatting and the resulting images are significantly sharper. Our second method, called complete high-resolution splatting, improves the volume reconstruction, which results in images that are very close to those produced by ray casting methods. The second part of the thesis incorporates wavelet analysis into high-resolution splatting. Wavelet analysis is a mathematical theory that decomposes volumes into multi-resolution hierarchies, which may be used to find coherence within volumes. The combination of wavelets with the high-resolution splatting method has the two advantages. Firstly the extended splatting method, called high-resolution wavelet splatting, can be directly applied to wavelet transformed volume data sets without performing an inverse transform. Secondly when visualising wavelet compressed volumes, only a small fraction of the wavelet coefficients need to be projected. For all three versions of the new high-resolution splatting method, complexity analyses, comprehensive error and performance analyses as well as implementation details are discussed.

Volume rendering is a research area within scientific visualisation, where images are computed from volumetric data sets for visual exploration. Such data sets are typically generated by Computer aided Tomography, Magnetic Resonance Imaging, Positron Emission Tomography or gained from simulations. The data sets are usually interpreted using optical models that assign optical properties to the volume and define the illumination and shading behaviour. Volume rendering techniques may be divided into three classes: object-order, image-order or hybrid methods. Image-order or ray casting methods shoot rays from the view plane into the volume and simulate the variation of light intensities along those rays. Object-order techniques traverse the volume data set and project each volume element onto the view plane. Hybrid volume rendering techniques combine these two approaches. A very popular object-order rendering method is called splatting. This technique traverses the volume data set and projects the optical properties of each volume element onto the view plane. This thesis consists of two parts. The first part introduces two new splatting methods, collectively called high-resolution splatting, which are based on standard splatting. Both high-resolution splatting methods correct errors of splatting by applying major modifications. We propose the first method, called fast high-resolution splatting, as an alternative to standard splatting. It may be used for quick previewing, since it is faster than standard splatting and the resulting images are significantly sharper. Our second method, called complete high-resolution splatting, improves the volume reconstruction, which results in images that are very close to those produced by ray casting methods. The second part of the thesis incorporates wavelet analysis into high-resolution splatting. Wavelet analysis is a mathematical theory that decomposes volumes into multi-resolution hierarchies, which may be used to find coherence within volumes. The combination of wavelets with the high-resolution splatting method has the two advantages. Firstly the extended splatting method, called high-resolution wavelet splatting, can be directly applied to wavelet transformed volume data sets without performing an inverse transform. Secondly when visualising wavelet compressed volumes, only a small fraction of the wavelet coefficients need to be projected. For all three versions of the new high-resolution splatting method, complexity analyses, comprehensive error and performance analyses as well as implementation details are discussed.

Modern medical imaging techniques provide an increasing amount of high-dimensional and high-resolution image data that need to be visualized, analyzed, and interpreted for diagnostic and treatment planning purposes. As a consequence, efficient ways of exploring these images are needed. In order to work with specific patient cases, it is necessary to be able to work directly with the medical image volumes and to generate the relevant 3D structures directly as they are needed for visualization and analysis. This requires efficient tools for segmentation, i.e., separation of objects from each other and from the background. Segmentation is hard to automate due to, e.g., high shape variability of organs and limited contrast between tissues. Manual segmentation, on the other hand, is tedious and error-prone. An approach combining the merits from automatic and manual methods is semi-automatic segmentation, where the user interactively provides input to the methods. For complex medical image volumes, the interactive part can be highly 3D oriented and is therefore dependent on the user interface. This thesis presents methods for interactive segmentation and visualization where true 3D interaction with haptic feedback and stereo graphics is used. Well-known segmentation methods such as fast marching, fuzzy connectedness, live-wire, and deformable models, have been tailored and extended for implementation in a 3D environment where volume visualization and haptics are used to guide the user. The visualization is accelerated with graphics hardware and therefore allows for volume rendering in stereo at interactive rates. The haptic feedback is rendered with constraint-based direct volume haptics in order to convey information about the data that is hard to visualize and thereby facilitate the interaction. The methods have been applied to real medical images, e.g., 3D liver CT data and 4D breast MR data with good results. To provide a tool for future work in this area, a software toolkit containing the implementations of the developed methods has been made publicly available.

The graphics hardware have evolved from simple feed forward triangle rasterization devices to flexible, programmable, and powerful parallel processors. This evolution allows the researchers to use graphics processing units (GPU) for both general purpose computations and advanced graphics rendering. Sophisticated GPUs hold great opportunities for the acceleration of computationally expensive photorealistic rendering methods. Rendering of photorealistic images in real-time is a challenge. In this work, we investigate efficient ways to utilize GPUs for real-time photorealistic rendering. Specifically, we studied uniform grid based ray tracing acceleration methods and GPU friendly traversal algorithms. We show that our method is faster than or competitive to other GPU based ray tracing acceleration techniques. The proposed approach is also applicable to the fast rendering of volumetric data. Additionally, we devised GPU based solutions for real-time stereoscopic image generation which can be used in companion with GPU based ray tracers.

O uso de dados volumétricos é bastante comum em diversas áreas da ciência, como Medicina, Física e Meteorologia. São exemplos típicos os dados provenientes de dispositivos de tomografia computadorizada ou ressonância magnética e os obtidos através de estimação de fenômenos físicos pelo uso de sensores diversos ou de simulação numérica. Tais dados apresentam-se, freqüentemente, sob a forma de uma grade tridimensional regular, onde cada elemento possui um valor escalar ou multidimensional (uma tupla de valores). Outras topologias também podem ser usadas para exprimir a disposição espacial dos valores. A visualização de dados volumétricos, importante na compreensão destes, é um processo não-trivial e, em decorrência, diversas técnicas foram propostas para abordar o problema. Visualização direta de volumes é uma abordagem em crescente popularização que representa visualmente os dados, conservando sua estrutura tridimensional, sem extrair geometrias intermediárias. Esse processo exige o mapeamento dos atributos dos elementos de volume para propriedades ópticas, permitindo a geração de imagens através da aplicação de um algoritmo de visualização, que pode implementar um modelo de iluminação. Tal mapeamento é definido por uma função, conhecida como função de transferência, que determina valores de atributos ópticos para cada valor encontrado no volume. Essa função desenvolve, portanto, um importante papel na visualização, pois define a visibilidade das estruturas presentes no volume — normalmente valendo-se do atributo opacidade — e também o aspecto destas na imagem final. Contudo, a definição de uma boa função de transferência, capaz de produzir imagens informativas, é um processo complexo que deve ser simplificado com o apoio de ferramentas adequadas. A simples especificação manual de uma função de transferência é um processo iterativo de tentativa e erro, em decorrência da dificuldade de compreensão do relacionamento entre a função utilizada e a imagem gerada, especialmente quando se trata de dados multidimensionais, que implicam funções de transferência com maior número de dimensões. Diante da necessidade de agilizar e simplificar a especificação de funções de transferência, abordagens semi-automáticas e automáticas para geração de funções foram propostas, exigindo do usuário esforço de interação reduzido ou nulo. Entretanto, as propostas existentes deixam a desejar na simplicidade, interatividade ou flexibilidade. O presente trabalho propõe técnicas de especificação de funções de transferência, para volumes escalares e multidimensionais, baseadas na automatização parcial do processo e simplificação do espaço de interação usado na definição das funções.Como principais contribuições, são apresentados uma eficaz combinação de técnicas complementares para especificação de funções de transferência para volumes escalares; e um método de especificação de funções de transferência para volumes multidimensionais que reúne o potencial de classificação dos mapas auto-organizáveis com a capacidade de decisão não-binária acerca davisibilidade e aspecto de voxels pertinente às funções de transferência tradicionais. / Volume data are very often used in several areas of science, such as medicine, physics and meteorology. Typical examples are data provided by computed tomography, magnetic resonance imaging or estimation of physical phenomena through numerical simulation or sensors. Such data are often provided as regular three-dimensional grids where each element has a scalar or higher-dimensional value, though other topologies may also be employed to express the position of the values in the three-dimensional space. Visualizing volume data is very important in understanding the conveyed information, but it is also a hard task. Thus, many approaches to this problem have been developed. Direct volume rendering is a set of visualization techniques that have become very popular because they can visually represent volume data, keeping their three-dimensional structure, without extracting intermediate geometries. Such processes require a mapping from voxels’ attributes to optical attributes, which allows generating images from the data through the application of a visualization algorithm that implements an illumination model, which is often very simple. This mapping, known as transfer function, associates each volume element with values of optical properties. Therefore, transfer functions play an important role in defining the visibility and the aspect of structures inside a volume, typically using opacity and color, respectively, as optical attributes. However, the design of a good transfer function, capable of generating informative images, is a complex task which must be simplified as much as possible through the support of suitable tools. A simple manual design process is a trial-and-error effort, due to the difficulty of understanding the relationship between the transfer function and the generated image, specially when dealing with multi-dimensional volume data, which require transfer functions with a wide domain. The need to accelerate and simplify the transfer function design led to the development of several automatic and semi-automatic approaches to the problem, which can reduce or eliminate the user’s interaction effort. However, the existent proposals lack in simplicity, interactivity or flexibility. This work outlines transfer function design methods for visualization of scalar volume data and multi-dimensional volume data. We propose techniques based on partial automation of the design process and simplification of the interaction space used in TF specification. Our main contributions are an effective combination of complementary techniques for specifying transfer functions for scalar volumes; and a multi-dimensional transfer function design method that brings together the classification capabilities of self-organizing maps and the transfer functions’ ability of non-binary decision on voxels’ visibility and aspect.

O uso de dados volumétricos é bastante comum em diversas áreas da ciência, como Medicina, Física e Meteorologia. São exemplos típicos os dados provenientes de dispositivos de tomografia computadorizada ou ressonância magnética e os obtidos através de estimação de fenômenos físicos pelo uso de sensores diversos ou de simulação numérica. Tais dados apresentam-se, freqüentemente, sob a forma de uma grade tridimensional regular, onde cada elemento possui um valor escalar ou multidimensional (uma tupla de valores). Outras topologias também podem ser usadas para exprimir a disposição espacial dos valores. A visualização de dados volumétricos, importante na compreensão destes, é um processo não-trivial e, em decorrência, diversas técnicas foram propostas para abordar o problema. Visualização direta de volumes é uma abordagem em crescente popularização que representa visualmente os dados, conservando sua estrutura tridimensional, sem extrair geometrias intermediárias. Esse processo exige o mapeamento dos atributos dos elementos de volume para propriedades ópticas, permitindo a geração de imagens através da aplicação de um algoritmo de visualização, que pode implementar um modelo de iluminação. Tal mapeamento é definido por uma função, conhecida como função de transferência, que determina valores de atributos ópticos para cada valor encontrado no volume. Essa função desenvolve, portanto, um importante papel na visualização, pois define a visibilidade das estruturas presentes no volume — normalmente valendo-se do atributo opacidade — e também o aspecto destas na imagem final. Contudo, a definição de uma boa função de transferência, capaz de produzir imagens informativas, é um processo complexo que deve ser simplificado com o apoio de ferramentas adequadas. A simples especificação manual de uma função de transferência é um processo iterativo de tentativa e erro, em decorrência da dificuldade de compreensão do relacionamento entre a função utilizada e a imagem gerada, especialmente quando se trata de dados multidimensionais, que implicam funções de transferência com maior número de dimensões. Diante da necessidade de agilizar e simplificar a especificação de funções de transferência, abordagens semi-automáticas e automáticas para geração de funções foram propostas, exigindo do usuário esforço de interação reduzido ou nulo. Entretanto, as propostas existentes deixam a desejar na simplicidade, interatividade ou flexibilidade. O presente trabalho propõe técnicas de especificação de funções de transferência, para volumes escalares e multidimensionais, baseadas na automatização parcial do processo e simplificação do espaço de interação usado na definição das funções.Como principais contribuições, são apresentados uma eficaz combinação de técnicas complementares para especificação de funções de transferência para volumes escalares; e um método de especificação de funções de transferência para volumes multidimensionais que reúne o potencial de classificação dos mapas auto-organizáveis com a capacidade de decisão não-binária acerca davisibilidade e aspecto de voxels pertinente às funções de transferência tradicionais. / Volume data are very often used in several areas of science, such as medicine, physics and meteorology. Typical examples are data provided by computed tomography, magnetic resonance imaging or estimation of physical phenomena through numerical simulation or sensors. Such data are often provided as regular three-dimensional grids where each element has a scalar or higher-dimensional value, though other topologies may also be employed to express the position of the values in the three-dimensional space. Visualizing volume data is very important in understanding the conveyed information, but it is also a hard task. Thus, many approaches to this problem have been developed. Direct volume rendering is a set of visualization techniques that have become very popular because they can visually represent volume data, keeping their three-dimensional structure, without extracting intermediate geometries. Such processes require a mapping from voxels’ attributes to optical attributes, which allows generating images from the data through the application of a visualization algorithm that implements an illumination model, which is often very simple. This mapping, known as transfer function, associates each volume element with values of optical properties. Therefore, transfer functions play an important role in defining the visibility and the aspect of structures inside a volume, typically using opacity and color, respectively, as optical attributes. However, the design of a good transfer function, capable of generating informative images, is a complex task which must be simplified as much as possible through the support of suitable tools. A simple manual design process is a trial-and-error effort, due to the difficulty of understanding the relationship between the transfer function and the generated image, specially when dealing with multi-dimensional volume data, which require transfer functions with a wide domain. The need to accelerate and simplify the transfer function design led to the development of several automatic and semi-automatic approaches to the problem, which can reduce or eliminate the user’s interaction effort. However, the existent proposals lack in simplicity, interactivity or flexibility. This work outlines transfer function design methods for visualization of scalar volume data and multi-dimensional volume data. We propose techniques based on partial automation of the design process and simplification of the interaction space used in TF specification. Our main contributions are an effective combination of complementary techniques for specifying transfer functions for scalar volumes; and a multi-dimensional transfer function design method that brings together the classification capabilities of self-organizing maps and the transfer functions’ ability of non-binary decision on voxels’ visibility and aspect.