As a non-robotic minimally invasive surgery, endoscopic surgery is one of the widely used surgeries for the medical domain to reduce the risk of infection, incisions, and the discomfort of the patient. The endoscopic surgery procedure, also named surgical workflow in this work, can be divided into different sub-phases. During the procedure, the surgeon inserts a thin, flexible tube with a video camera through a small incision or a natural orifice like the mouth or nostrils. The surgeon can utilize tiny surgical instruments while viewing organs on the computer monitor through these tubes. The surgery only allows a limited number of instruments simultaneously appearing in the body, requiring a sufficient instrument preparation method. Therefore, surgical workflow anticipation, including surgical instrument and phase anticipation, is essential for an intra-operative decision-support system. It deciphers the surgeon's behaviors and the patient's status to forecast surgical instrument and phase occurrence before they appear, supporting instrument preparation and computer-assisted intervention (CAI) systems. In this work, we investigate an unexplored surgical workflow anticipation problem by proposing an Instrument Interaction Aware Anticipation Network (IIA-Net). Spatially, it utilizes rich visual features about the context information around the instrument, i.e., instrument interaction with their surroundings. Temporally, it allows for a large receptive field to capture the long-term dependency in the long and untrimmed surgical videos through a causal dilated multi-stage temporal convolutional network. Our model enforces an online inference with reliable predictions even with severe noise and artifacts in the recorded videos. Extensive experiments on Cholec80 dataset demonstrate the performance of our proposed method exceeds the state-of-the-art method by a large margin (1.40 v.s. 1.75 for inMAE and 2.14 v.s. 2.68 for eMAE).

In todays complex society many companies have large and complex workflows which may involve several different tools. This report will look into the workflow of a big programming company, from when a mistake is found in the code until it has been corrected and see how the workflow can be improved. / <p>Examensarbetet är utfört vid Institutionen för teknik och naturvetenskap (ITN) vid Tekniska fakulteten, Linköpings universitet</p>

The aim of the project was to compare the workflow for an Elekta Linac with and without the surfacescanning system Catalyst and describe pros and cons with both workflows. The findings in the reportcan be used as decision support in development of Elekta products and workflow improvements. The method for the project was to do interviews, observations and time measurements at Södersjukhuset(not using Catalyst) and Sundsvalls sjukhus (using Catalyst). The workflows were graded in an as-sessment protocol covering time efficiency, comfort, noise, resources, reliability, cost, dosage and sideeffects. Different workflow scenarios were simulated in AnyLogic. The result of the project was that, according to our protocol, the workflow with Catalyst was ratedhigher than without it. The simulations in Anylogic showed that minimizing gaps in the treatment sched-ule generated the same number of patients treated per day, if the positioning could not be done faster.The simulations also showed that removing position verification with cone beam computer tomography(CBCT), an imaging system which is used in addition to the Catalyst system, would increase the numberof treated patients with approximately 33%. The conclusion was that there were no great differences in time efficiency between the workflows. How-ever, considering the higher reliability and comfort for the patient, optical surface scanning can improvethe positioning for Elekta Linac and is therefore worth implementing. Minimizing treatment gaps wouldnot improve the workflow. Removing the use of CBCT would increase the number of treated patientsper day.

Robotic assistance is applied in orthopedic interventions for pedicle screw placement (PSP). While current robots do not act autonomously, they are expected to have higher autonomy under surgeon supervision in the mid-term. Augmented reality (AR) is promising to support this supervision and to enable human–robot interaction (HRI). To outline a futuristic scenario for robotic PSP, the current workflow was analyzed through literature review and expert discussion. Based on this, a hypothetical workflow of the intervention was developed, which additionally contains the analysis of the necessary information exchange between human and robot. A video see-through AR prototype was designed and implemented. A robotic arm with an orthopedic drill mock-up simulated the robotic assistance. The AR prototype included a user interface to enable HRI. The interface provides data to facilitate understanding of the robot’s ”intentions”, e.g., patient-specific CT images, the current workflow phase, or the next planned robot motion. Two-dimensional and three-dimensional visualization illustrated patient-specific medical data and the drilling process. The findings of this work contribute a valuable approach in terms of addressing future clinical needs and highlighting the importance of AR support for HRI.

This project has been initiated in cooperation with a Swedish manufacturing company. Due to increased demand and competition, the company wants to streamline its production process, increase the degree of automation and visualize specific workflows. By creating a frame of reference and a literature review, a theoretical basis for methods and concepts which has been utilized throughout the project has been obtained. With the help of the identified methods and methodologies, a current state analysis was performed. Using traditional Lean tools such as Genchi genbutsu, Ishikawa diagram and a 5-why analysis, in combination with time studies and interviews, the current state of the studied system was successfully mapped and analysed. With the help of the current state analysis, which served as a conceptual model, a simulation model of the current state was created in order to handle the large variety and the complexity of the system. The simulation model was validated and verified in order to ensure that it was “good enough” for the purpose of this project in the depiction of the real world system. During the experimental design, several improvement suggestions were created by utilizing methods such as brainstorming, Ishikawa diagram and a 5-why analysis. In a Kaizen event, onsite personnel had the opportunity to decide which suggestions that was fit for experimentation using simulation. With the result of the Kaizen event, experiments were performed in order to evaluate the proposed improvement suggestion. As a result, several new insights regarding improvements could be obtained, which provided several suggestions for an improved future state. Including a proposed automated cell. The analysis of the results did not entirely satisfy the aim of the project since certain factors could not be analysed, therefore the authors recommend that further studies are needed if proposed improvement suggestions are to be implemented.

Technological advancements in mass spectrometry allowing quantification of almost complete proteomes make proteomics a key platform for generating unique functional molecular data. Furthermore, the integrative analysis of genomic and proteomic data, termed proteogenomics, has emerged as a new field revealing insights into gene expression regulation, cell signalling, and disease processes. However, the lack of software tools for high-throughput integration and unbiased modification and variant detection hinder efforts for large-scale proteogenomics studies. The main objectives of this work are to address these issues by developing and applying new software tools and data analysis methods. Firstly, I address mapping of peptide sequences to reference genomes. I introduce a novel tool for high-throughput mapping and highlight its unique features facilitating quantitative and post-translational modification mapping alongside accounting for amino acid substitutions. The performance is benchmarked. Furthermore, I offer an additional tool that permits generation of web accessible hubs of genome wide mappings. To enable unbiased identification of post-translational modifications and amino acid substitutions for high resolution mass spectrometry data, I present algorithmic updates the mass tolerant blind spectrum comparison tool ’MS SMiV’. I demonstrate the applicability of the changes by benchmarking against a published mass tolerant database search of a high resolution tandem mass spectrometry dataset. I then present the application of ‘MS SMiV’ on a panel of 50 colorectal cancer cell lines. I show that the adaption of ‘MS SMiV’ outperforms traditional sequence database based identification of single amino acid variants. Furthermore, I highlight the utility of mass tolerant spectrum matching in combination with isobaric labelled quantitative proteomics in distinguishing between post-translational modifications and amino acid variants of similar mass. In the last part of this work I integrate both tools with a high-throughput proteogenomic identification pipeline and apply it to a pilot study of chondrocytes derived from 12 osteoarthritic individuals. I show the value of this approach in identifying variation between individuals and molecular levels and highlight them with individual examples. I show that multi-plexed proteogenomics can be used to infer genotypes of individuals.

Dans cette thèse, nous traitons des problèmes de la détection des personnes et de l'estimation de leurs poses dans la Salle Opératoire (SO), deux éléments clés pour le développement d'applications d'assistance chirurgicale. Nous percevons la salle grâce à des caméras RGB-D qui fournissent des informations visuelles complémentaires sur la scène. Ces informations permettent de développer des méthodes mieux adaptées aux difficultés propres aux SO, comme l'encombrement, les surfaces sans texture et les occlusions. Nous présentons des nouvelles approches qui tirent profit des informations temporelles, de profondeur et des vues multiples afin de construire des modèles robustes pour la détection des personnes et de leurs poses. Une évaluation est effectuée sur plusieurs jeux de données complexes enregistrés dans des salles opératoires avec une ou plusieurs caméras. Les résultats obtenus sont très prometteurs et montrent que nos approches surpassent les méthodes de l'état de l'art sur ces données cliniques. / In this thesis, we address the two problems of person detection and pose estimation in Operating Rooms (ORs), which are key ingredients in the development of surgical assistance applications. We perceive the OR using compact RGB-D cameras that can be conveniently integrated in the room. These sensors provide complementary information about the scene, which enables us to develop methods that can cope with numerous challenges present in the OR, e.g. clutter, textureless surfaces and occlusions. We present novel part-based approaches that take advantage of depth, multi-view and temporal information to construct robust human detection and pose estimation models. Evaluation is performed on new single- and multi-view datasets recorded in operating rooms. We demonstrate very promising results and show that our approaches outperform state-of-the-art methods on this challenging data acquired during real surgeries.