The research presented in this thesis describes a substantial part of the design of a prototypical surgical training simulator. The results are intended to be applied in future simulators used to educate and train surgeons for bone milling operations. In earlier work we have developed a haptic bone milling surgery simulator prototype based on three degrees-of-freedom force feedback. The contributions presented here constitute an extension to that work by further developing the haptic algorithms to enable six degrees-of-freedom (6-DOF) haptic feedback. Such feedback is crucial for a realistic haptic experience when interacting in a more complex virtual environment, particularly in milling applications.The main contributions of this thesis are:The developed 6-DOF haptic algorithm is based on the work done by Barbic and James, but differs in that the algorithm is modified and optimized for milling applications. The new algorithm handles the challenging problem of real-time rendering of volume data changes due to material removal, while fulfilling the requirements on stability and smoothness of the kind of haptic applications that we approach. The material removal algorithm and the graphic rendering presented here are based on the earlier research. The new 6-DOF haptic milling algorithm is characterized by voxel-based collision detection, penalty-based and constraint-based haptic feedback, and by using a virtual coupling for stable interaction.Milling a hole in an object in the virtual environment or dragging the virtual tool along the surface of a virtual object shall generate realistic contact force and torque in the correct directions. These are important requirements for a bone milling simulator to be used as a future training tool in the curriculum of surgeons. The goal of this thesis is to present and state the quality of a newly developed 6-DOF haptic milling algorithm. The quality of the algorithm is confirmed through a verification test and a face validity study performed in collaboration with the Division of Orthopedics at the Karolinska University Hospital. In a simulator prototype, the haptic algorithm is implemented together with a new 6-DOF haptic device based on parallel kinematics. This device is developed with workspace, transparency and stiffness characteristics specifically adapted to the particular procedure. This thesis is focuses on the 6-DOF haptic algorithm. / QC 20120226

<p>A prototype of a haptic and virtual reality simulator has been developed for simulation of the bone milling and material removal process occurring in several operations, e.g. temporal bone surgery or dental milling. The milling phase of an operation is difficult, safety critical and very time consuming. Reduction of operation time by only a few percent would in the long run save society large expenses. In order to reduce operation time and to provide surgeons with an invaluable practicing environment, this licentiate thesis discusses the introduction of a simulator system to be used in both surgeon curriculum and in close connection to the actual operations.</p><p>The virtual reality and haptic feedback topics still constitute a young and unexplored area. It has only been active for about 10-15 years for medical applications. High risk training on real patients and the change from open surgery to endoscopic procedures have enforced the introduction of haptic and virtual reality simulators for training of surgeons. Increased computer power and the similarity to the successful aviation simulators also motivate to start using simulators for training of surgical skills.</p><p>The research focus has been twofold: 1) To develop a well working VR-system for realistic graphical representation of the skull itself including the changes resulting from milling, and 2) to find an efficient algorithm for haptic feedback to mimic the milling procedure using the volumetric Computer Tomography (CT) data of the skull. The developed haptic algorithm has been verified and tested in the simulator. The visualization of the milling process is rendered at a graphical frame rate of 30 Hz and the haptic rendering loop is updated at 1000 Hz. Test results show that the real-time demands are fulfilled. The visual and haptic implementations have been the two major steps to reach the over all goal with this research project.</p><p>A survey study is also included where the use of VR and haptic simulators in the surgical curriculum is investigated. The study starts with a historical perspective of the VR and haptic topics and is built up by answering different questions related to this topic and the implementation of simulators at the medical centres. The questions are of general concern for those developing surgical VR and haptic simulators.</p><p>Suggested future work includes modelling, development and validation of the haptic forces occurring in the milling process and, based on this, implementation in the simulator system. Also, further development of the simulator should be done in close cooperation with surgeons in order to get appropriate feedback for further improvements of the functionality and performance of the simulator.</p>

abstract: Colorectal cancer is the second-highest cause of cancer-related deaths in the United States with approximately 50,000 estimated deaths in 2015. The advanced stages of colorectal cancer has a poor five-year survival rate of 10%, whereas the diagnosis in early stages of development has showed a more favorable five-year survival rate of 90%. Early diagnosis of colorectal cancer is achievable if colorectal polyps, a possible precursor to cancer, are detected and removed before developing into malignancy. The preferred method for polyp detection and removal is optical colonoscopy. A colonoscopic procedure consists of two phases: (1) insertion phase during which a flexible endoscope (a flexible tube with a tiny video camera at the tip) is advanced via the anus and then gradually to the end of the colon--called the cecum, and (2) withdrawal phase during which the endoscope is gradually withdrawn while colonoscopists examine the colon wall to find and remove polyps. Colonoscopy is an effective procedure and has led to a significant decline in the incidence and mortality of colon cancer. However, despite many screening and therapeutic advantages, 1 out of every 4 polyps and 1 out of 13 colon cancers are missed during colonoscopy. There are many factors that contribute to missed polyps and cancers including poor colon preparation, inadequate navigational skills, and fatigue. Poor colon preparation results in a substantial portion of colon covered with fecal content, hindering a careful examination of the colon. Inadequate navigational skills can prevent a colonoscopist from examining hard-to-reach regions of the colon that may contain a polyp. Fatigue can manifest itself in the performance of a colonoscopist by decreasing diligence and vigilance during procedures. Lack of vigilance may prevent a colonoscopist from detecting the polyps that briefly appear in the colonoscopy videos. Lack of diligence may result in hasty examination of the colon that is likely to miss polyps and lesions. To reduce polyp and cancer miss rates, this research presents a quality assurance system with 3 components. The first component is an automatic polyp detection system that highlights the regions with suspected polyps in colonoscopy videos. The goal is to encourage more vigilance during procedures. The suggested polyp detection system consists of several novel modules: (1) a new patch descriptor that characterizes image appearance around boundaries more accurately and more efficiently than widely-used patch descriptors such HoG, LBP, and Daisy; (2) A 2-stage classification framework that is able to enhance low level image features prior to classification. Unlike the traditional way of image classification where a single patch undergoes the processing pipeline, our system fuses the information extracted from a pair of patches for more accurate edge classification; (3) a new vote accumulation scheme that robustly localizes objects with curvy boundaries in fragmented edge maps. Our voting scheme produces a probabilistic output for each polyp candidate but unlike the existing methods (e.g., Hough transform) does not require any predefined parametric model of the object of interest; (4) and a unique three-way image representation coupled with convolutional neural networks (CNNs) for classifying the polyp candidates. Our image representation efficiently captures a variety of features such as color, texture, shape, and temporal information and significantly improves the performance of the subsequent CNNs for candidate classification. This contrasts with the exiting methods that mainly rely on a subset of the above image features for polyp detection. Furthermore, this research is the first to investigate the use of CNNs for polyp detection in colonoscopy videos. The second component of our quality assurance system is an automatic image quality assessment for colonoscopy. The goal is to encourage more diligence during procedures by warning against hasty and low quality colon examination. We detect a low quality colon examination by identifying a number of consecutive non-informative frames in videos. We base our methodology for detecting non-informative frames on two key observations: (1) non-informative frames most often show an unrecognizable scene with few details and blurry edges and thus their information can be locally compressed in a few Discrete Cosine Transform (DCT) coefficients; however, informative images include much more details and their information content cannot be summarized by a small subset of DCT coefficients; (2) information content is spread all over the image in the case of informative frames, whereas in non-informative frames, depending on image artifacts and degradation factors, details may appear in only a few regions. We use the former observation in designing our global features and the latter in designing our local image features. We demonstrated that the suggested new features are superior to the existing features based on wavelet and Fourier transforms. The third component of our quality assurance system is a 3D visualization system. The goal is to provide colonoscopists with feedback about the regions of the colon that have remained unexamined during colonoscopy, thereby helping them improve their navigational skills. The suggested system is based on a new 3D reconstruction algorithm that combines depth and position information for 3D reconstruction. We propose to use a depth camera and a tracking sensor to obtain depth and position information. Our system contrasts with the existing works where the depth and position information are unreliably estimated from the colonoscopy frames. We conducted a use case experiment, demonstrating that the suggested 3D visualization system can determine the unseen regions of the navigated environment. However, due to technology limitations, we were not able to evaluate our 3D visualization system using a phantom model of the colon. / Dissertation/Thesis / Doctoral Dissertation Biomedical Informatics 2015

Nesta tese apresentamos uma metodologia de mapeamento capaz de gerar representações em termos de grafos para sistemas biológicos de conectividade complexa. Tais sistemas são inicialmente armazenados na forma de imagens digitais e em seguida submetidos a um pré-processamento com objetivo de padronizar as imagens. As imagens pré-processadas são então utilizadas para gerar modelos tridimensionais dos sistemas de interesse. Um algoritmo de propagação de rótulos é utilizado para extrair os esqueletos dos modelos volumétricos e estes esqueletos são por fim, representados por um grafo, composto por vértices e arestas. Os vértices e arestas desse grafo armazenam propriedades do sistema original, como posição, comprimento e diâmetro, bem como as características topológicas de tais sistemas. Finalmente, os grafos resultantes são estudados através da teoria das redes complexas, dentro de um contexto específico para cada sistema. Nossos procedimentos foram aplicados com sucesso a diferentes sistemas biológicos, como artérias caríotidas, árvores arteriais, estruturas mitocondriais e poros em amostras de solo. / In the present work, we developed a mapping methodology able to build a graph representation for biological branched systems. Initially, such systems are stored as digital images and then they undergo a pre-processing in order to standardize the images. The pre-processed imagens are used to build tridimensional models of the interested systems. A label-propagation algorithm is used to extract the skeleton from the volumetric models and these skeletons are then represented by a graph, composed by nodes and edges. The nodes and edges of these graphs store properties of the original system, such as spatial position, lengths and diameter, as well as the topological features of such systems. Finally, the graphs are studied by using the complex networks theory within a specific context for each system. Our procedures were apllied sucefully to different biological systems, such as carotid artery, arterial trees, mitocondrial structure and pores in soil samples.

Detta examensarbete syftar till att undersöka hur aktörerna kommunicerar effektivt med verksamheten i projektering med hjälp av en 3D-visualisering. Rapporten är resultatet av en fallstudie utförd på skol- och sjukhusprojekt. I fallstudien har kvalitativa intervjuer genomförts på respondenter som innehar olika roller inom de studerande projekten. Intervjuerna har syftat till att ta reda på om 3D-visualisering har används inom projektet och om det har påverkat kommunikationen mellan aktörerna och verksamheten. Intervjuerna har även syftat till att ta reda på om en 3D-visualisering bidrar till en större förståelse hos verksamheten för projektet. Resultatet redovisar att 3D-visualisering används mot verksmaheten i skolprojketet men inte i sjukhusprojektet. Resultatet visar även på att en 3D-visualisering kan bidra till en större förståelse för projektet hos verksamheten samt aktörernas intsällning till verksamhetens delaktighet i projektet och om/hur aktörerna nyttjar 3D-visualisering vid kommunikationen mot verksamheten. / The purpose of this thesis is to investigate if 3D-visualization can affect of effectiveness of communication. The report is the result of a case study performed on one school- and two hospital project. In the case study qualitative interviews with respondents who have different roles in the studied project were conducted. The interviews aim to investigate if 3D-visualization affect the commutation between the organizations and the other stakeholders in the project. The interviews also aim to investigate if the use of a 3D-visualization affects the organizations understanding of the project. The thesis results shows that a 3D-visualization was used in the school project but not in the hospital projects. The results also shows that a 3D-visualization can affect the understandning of the project for the organization and the stakeholder’s attitude for the participation of the organizations in the project and how/if the stakeholder´s use of 3D-visualization to communicate with the organizations.

BUDI (Building Urban Designs Interactively) is an integrated 3D visualization and remote collaboration platform for complex urban design tasks. Users with different backgrounds can remotely engage in the entire design cycle, improving the quality of the end result. In this paper, I consider the trade-offs encountered when trying to make spatial-based collaboration seamless. Specifically, I detail the multi-dimensional data visualization and interaction the platform provides, and outline how users can interact with and analyze various aspects of urban design. In $BUDI$, the display and interactive environment was designed to seamlessly expand beyond a traditional two-dimensional surface into a fully immersive three-dimensional space. Clients on various devices connect with servers for different functionalities tailored for different user groups. A demonstration with a local urban planning use-case shows the costs and benefits of $BUDI$ as a spatial-based collaborative platform. A performance evaluation with remote collaboration shows how the platform can meet the requirements for real-time and seamless collaboration. / Graduate

A prototype of a haptic and virtual reality simulator has been developed for simulation of the bone milling and material removal process occurring in several operations, e.g. temporal bone surgery or dental milling. The milling phase of an operation is difficult, safety critical and very time consuming. Reduction of operation time by only a few percent would in the long run save society large expenses. In order to reduce operation time and to provide surgeons with an invaluable practicing environment, this licentiate thesis discusses the introduction of a simulator system to be used in both surgeon curriculum and in close connection to the actual operations. The virtual reality and haptic feedback topics still constitute a young and unexplored area. It has only been active for about 10-15 years for medical applications. High risk training on real patients and the change from open surgery to endoscopic procedures have enforced the introduction of haptic and virtual reality simulators for training of surgeons. Increased computer power and the similarity to the successful aviation simulators also motivate to start using simulators for training of surgical skills. The research focus has been twofold: 1) To develop a well working VR-system for realistic graphical representation of the skull itself including the changes resulting from milling, and 2) to find an efficient algorithm for haptic feedback to mimic the milling procedure using the volumetric Computer Tomography (CT) data of the skull. The developed haptic algorithm has been verified and tested in the simulator. The visualization of the milling process is rendered at a graphical frame rate of 30 Hz and the haptic rendering loop is updated at 1000 Hz. Test results show that the real-time demands are fulfilled. The visual and haptic implementations have been the two major steps to reach the over all goal with this research project. A survey study is also included where the use of VR and haptic simulators in the surgical curriculum is investigated. The study starts with a historical perspective of the VR and haptic topics and is built up by answering different questions related to this topic and the implementation of simulators at the medical centres. The questions are of general concern for those developing surgical VR and haptic simulators. Suggested future work includes modelling, development and validation of the haptic forces occurring in the milling process and, based on this, implementation in the simulator system. Also, further development of the simulator should be done in close cooperation with surgeons in order to get appropriate feedback for further improvements of the functionality and performance of the simulator. / QC 20101112

Fractures and fracture networks govern the mechanical and fluid flow behavior of rock masses. Tunneling and other rock mechanics applications therefore require the characterization of rock fractures based on geological data. Field investigations produce only a limited amount of data from boreholes, outcrops, cut slopes, and geophysical surveys. In tunneling, the process of excavation creates a priceless opportunity to gather more data during construction. Typically, however, these data are not utilized due to the impedance of sampling and analysis on the flow of construction, and safety concerns with sampling within unlined tunnel sections. However, the use of this additional data would increase the overall safety, quality, and cost savings of tunneling. This study deals with several aspects of the above, with the goal of creating methods and tools to allow engineers and geologists to gather and analysis fracture data in tunnels without interrupting the excavation and without compromising safety. Distribution-independent trace density and mean trace length estimators are developed using principles of stereology. An optimization technique is developed utilizing Differential Evolution to infer fracture size and shape from trace data obtained on two or more nonparallel sampling planes. A method of producing nearly bias free empirical trace length CDF's is also introduced. These new methods and tools were validated using Monte Carlo simulations. A field study was conducted in an existing tunnel allowing the above methods and tools to be further validated and tested. A relational database was developed to aid in storage, retrieval, and analysis of field data. Fracture models were built and updated using fracture data from within the tunnel. Utilization of state of the art imaging techniques allowed for remote sampling and analysis, which were enhanced by the use of 3d visualization techniques. / Ph. D.

Les systèmes distribués, tels que les grilles, sont utilisés aujourd’hui pour l’exécution des grandes applications parallèles. Quelques caractéristiques de ces systèmes sont l’interconnexion complexe de ressources qui pourraient être présent et de la facile passage à l’échelle. La complexité d’interconnexion vient, par exemple, d’un nombre plus grand de directives de routage pour la communication entre les processus et une latence variable dans le temps. La passage à l’échelle signifie que des ressources peuvent être ajoutées indéfiniment simplement en les reliant à l’infrastructure existante. Ces caractéristiques influencent directement la façon dont la performance des applications parallèles doit être analysée. Les techniques de visualisation traditionnelles pour cette analyse sont généralement basées sur des diagrammes de Gantt que disposent la liste des composants de l’application verticalement et metent la ligne du temps sur l’axe horizontal. Ces représentations visuelles ne sont généralement pas adaptés à l’analyse des applications exécutées en parallèle dans les grilles. La première raison est qu’elles n’ont pas été conçues pour offrir aux développeurs une analyse qui montre aussi la topologie du réseau des ressources. La deuxième raison est que les techniques de visualisation traditionnels ne s’adaptent pas bien quand des milliers d’entités doivent être analysés ensemble. Cette thèse tente de résoudre les problèmes des techniques traditionnelles dans la visualisation des applications parallèles. L’idée principale est d’exploiter le domaine de la visualisation de l’information et essayer d’appliquer ses concepts dans le cadre de l’analyse des programmes parallèles. Portant de cette idée, la thèse propose deux modèles de visualisation : les trois dimensions et le modèle d’agrégation visuelle. Le premier peut être utilisé pour analyser les programmes parallèles en tenant compte de la topologie du réseau. L’affichage lui-même se compose de trois dimensions, où deux sont utilisés pour indiquer la topologie et la troisième est utilisée pour représenter le temps. Le second modèle peut être utilisé pour analyser des applications parallèles comportant un très grand nombre de processsus. Ce deuxième modèle exploite une organisation hiérarchique des données utilisée par une technique appelée Treemap pour représenter visuellement la hiérarchie. Les implications de cette thèse sont directement liées à l’analyse et la compréhension des applications parallèles exécutés dans les systèmes distribués. Elle améliore la compréhension des modes de communication entre les processus et améliore la possibilité d’assortir les motifs avec cette topologie de réseau réel sur des grilles. Bien que nous utilisons abondamment l’exemple de la topologie du réseau, l’approche pourrait être adapté, avec presque pas de changements, à l’interconnexion fourni par un middleware d’une interconnexion logique. Avec la technique d’agrégation, les développeurs sont en mesure de rechercher des patterns et d’observer le comportement des applications à grande échelle. / Sistemas distribuídos tais como grids são usados hoje para a execução de aplicações paralelas com um grande número de processos. Algumas características desses sistemas são a presença de uma complexa rede de interconexão e a escalabilidade de recursos. A complexidade de rede vem, por exemplo, de largura de banda e latências variáveis ao longo do tempo. Escalabilidade é a característica pela qual novos recursos podem ser adicionados em um grid apenas através da conexão em uma infraestrutura pré-existente. Estas características influenciam a forma como o desempenho de aplicações paralelas deve ser analisado. Esquemas tradicionais de visualização de desempenho são usualmente baseados em gráficos Gantt com uma dimensão para listar entidades monitoradas e outra para o tempo. Visualizações como essa não são apropriadas para a análise de aplicações paralelas executadas em grid. A primeira razão para tal é que elas não foram concebidas para oferecer ao desenvolvedor uma análise que mostra a topologia dos recursos e a relação disso com a aplicação. A segunda razão é que técnicas tradicionais não são escaláveis quando milhares de entidades monitoradas devem ser analisadas conjuntamente. Esta tese tenta resolver estes problemas encontrados em técnicas de visualização tradicionais para a análise de aplicações paralelas. A idéia principal consiste em explorar técnicas da área de visualização da informação e aplicá-las no contexto de análise de programas paralelos. Levando em conta isto, esta tese propõe dois modelos de visualização: o de três dimensões e o modelo de agregação visual. O primeiro pode ser utilizado para analisar aplicações levando-se em conta a topologia da rede dos recursos. A visualização em si é composta por três dimensões, onde duas são usadas para mostrar a topologia e a terceira é usada para representar o tempo. O segundo modelo pode ser usado para analisar aplicações paralelas com uma grande quantidade de processos. Ela explora uma organização hierárquica dos dados de monitoramento e uma técnica de visualização chamada Treemap para representar visualmente a hierarquia. Os dois modelos representam uma nova forma de analisar aplicação paralelas visualmente, uma vez que eles foram concebidos para larga-escala e sistemas distribuídos complexos, como grids. As implicações desta tese estão diretamente relacionadas à análise e ao entendimento do comportamento de aplicações paralelas executadas em sistemas distribuídos. Um dos modelos de visualização apresentados aumenta a compreensão dos padrões de comunicação entre processos e oferece a possibilidade de observar tal padrão com a topologia de rede. Embora a topologia de rede seja usada, a abordagem pode ser adaptada sem grandes mudanças para levar em conta interconexões lógicas de bibliotecas de comunicação. Com a técnica de agregação apresentada nesta tese, os desenvolvedores são capazes de observar padrões de aplicações paralelas de larga escala. / Highly distributed systems such as grids are used today for the execution of large-scale parallel applications. Some characteristics of these systems are the complex resource interconnection that might be present and the scalability. The interconnection complexity comes from the different number of hops to provide communication among applications processes and differences in network latencies and bandwidth. The scalability means that the resources can be added indefinitely just by connecting them to the existing infrastructure. These characteristics influence directly the way parallel applications performance must be analyzed. Current traditional visualization schemes to this analysis are usually based on Gantt charts with one dimension to list the monitored entities and the other dimension dedicated to time. These visualizations are generally not suited to parallel applications executed in grids. The first reason is that they were not built to offer to the developer an analysis that also shows the network topology of the resources. The second reason is that traditional visualization techniques do not scale well when thousands of monitored entities must be analyzed together. This thesis tries to overcome the issues encountered on traditional visualization techniques for parallel applications. The main idea behind our efforts is to explore techniques from the information visualization research area and to apply them in the context of parallel applications analysis. Based on this main idea, the thesis proposes two visualization models: the three-dimensional and the visual aggregation model. The former might be used to analyze parallel applications taking into account the network topology of the resources. The visualization itself is composed of three dimensions, where two of them are used to render the topology and the third is used to represent time. The later model can be used to analyze parallel applications composed of several thousands of processes. It uses hierarchical organization of monitoring data and an information visualization technique called Treemap to represent that hierarchy. Both models represent a novel way to visualize the behavior of parallel applications, since they are conceived considering large-scale and complex distributed systems, such as grids. The implications of this thesis are directly related to the analysis and understanding of parallel applications executed in distributed systems. It enhances the comprehension of patterns in communication among processes and improves the possibility of matching this patterns with real network topology of grids. Although we extensively use the network topology example, the approach could be adapted with almost no changes to the interconnection provided by a middleware of a logical interconnection. With the scalable visualization technique, developers are able to look for patterns and observe the behavior of large-scale applications.

The Local Climate Change Visioning Project is using 3D visualization to bring climate change action to life for local communities.