National and Kapodistrian University of Athens
Ανάλυση χειρουργικών δεξιοτήτων με τεχνικές υπολογιστικής όρασης και τεχνητής νοημοσύνης
Abstract
dc:descriptionThe assessment and improvement of surgical skills are fundamental pillars for ensuring the quality of surgical care and patient safety. Traditional training and assessment methods, which rely mainly on subjective observation by experienced instructors, present limitations such as lack of objectivity, expenditure of valuable time, and inability to provide detailed and quantied feedback. The present doctoral thesis falls within the eld of Surgical Data Science and investigates the application of advanced Computer Vision and Articial Intelligence (AI) techniques for the automated analysis and assessment of surgical skills. Initially, the thesis focuses on online surgical gesture recognition in laparoscopic training tasks. A new video dataset of 80 surgical sessions (40 per task: Peg Transfer and Knot Tying) is presented, and a hybrid architecture (C3DTrans) is proposed, combining 3D Convolutional Neural Networks (3D CNNs) for extracting spatiotemporal features and Transformers for modeling long-term dependencies. The proposed model is designed for causal, real-time inference, enabling the online prediction of surgical gestures as the video stream unfolds. Particular emphasis is placed on addressing the lack of large volumes of annotated data through the introduction of Self-Supervised Learning techniques, specically next frame prediction. Evaluation on the Peg Transfer and Knot Tying tasks demonstrated excellent performance, with gesture-level accuracy reaching 97.5% and 97.9%, respectively. Furthermore, Self-Supervised Learning enabled the model to maintain high performance even with signicantly reduced annotated data. Finally, the model yielded competitive results (75.8% accuracy) on the benchmark JIGSAWS dataset, conrming its generalizability. Subsequently, the research extends to skill assessment in open surgery, a eld lacking inherent sources of digital imagery. A new multimodal dataset was created from 20 participants (students, residents, and experts) performing three fundamental tasks: Knot Tying, Continuous Suturing, and Interrupted Suturing. Data collection included egocentric video and motion data from electromagnetic sensors. For establishing the ground truth based on the OSATS scale, two expert surgeons independently annotated the performances. A Deep Learning framework was developed and evaluated, utilizing the ResNet50 model in combination with various temporal analysis models (TCN, LSTM, Transformer) for the automatic prediction of the score. The results demonstrated the superiority of Transformer-based architectures, which achieved high correlation with the consensus score of experts, specically ρ = 0.90 for Knot Tying, ρ = 0.88 for Continuous Suturing, and ρ = 0.84 for Interrupted Suturing. Notably, the model’s correlation with the expert consensus was found to be comparable to or, in certain tasks, exceeding the inter-rater correlation between the two expert surgeons themselves (ρ = 0.90 for Knot Tying, ρ = 0.76 for Continuous Suturing, and ρ = 0.77 for Interrupted Suturing), proving the model’s ability to capture complex skill patterns. Additionally, the thesis presents complementary research regarding quantitative analysis of surgical performance through detailed video annotation, highlighting signicant dierentiating metrics between novices and experts. Finally, two studies are presented for the automated assessment of gallbladder vascularity as an indicator of surgical diculty. The rst study applies classical Multiple-Instance Learning (MIL) methods, while the second proposes an advanced MICNN architecture with attention mechanisms, achieving high accuracy (92.1%) in predicting expert judgments and bypassing the subjectivity and low inter-rater agreement typically observed in clinical evaluations. In summary, the present thesis contributes to the development of objective and automated assessment systems, leveraging modern machine learning methods. The results indicate that these systems have the potential to enhance surgical training by providing reliable and immediate feedback, and lay the foundations for integrating intelligent assistance systems into the modern operating room.
Degree
thesis:*- Grantor dc:publisher
- National and Kapodistrian University of Athens
- Year dc:date
- 2026
Author and committee
dc:creator, dc:contributor.*- Authors dc:creator
-
- Gazis, Athanasios
- Γαζής, Αθανάσιος
Subjects
dc:subject × 23- Επιστήμη Χειρουργιϰών Δεδομένων
- Υπολογιστική όραση
- Τεχνητή νοημοσύνη
- Αναγνώριση Χειρουργιϰών Χειρονομιών
- ξιολόγηση Χειρουργιϰών Δεξιοτή- των
- Λαπαροσκοπική χειρουργική
- Ανοιϰτή Χειρουργιϰή
- Βαθιά μάθηση
- Αυτο-επιβλεπόμενη μάθηση
- Surgical Data Science
- Computer vision
- Artificial intelligence
- Surgical Gesture Recognition
- Surgical Skill Assessment
- Laparoscopic surgery
- Open Surgery
- Deep learning
- Transformers
- Self-supervised learning
- Επιστήμη Ηλεκτρονικών Υπολογιστών και Πληροφορική
- Φυσικές Επιστήμες
- Computer and Information Sciences
- Natural Sciences
Rights
- Language dc:language
- gre
Identifiers
dc:identifier.*- Identifier
- 10.12681/eadd/61641