Eine Methode zur kontaktlosen Blutdruckbestimmung mittels video-basierter Extraktion von Photoplethysmographie-Kurven aus dem Gesicht von Patienten sollte evaluiert werden. Mittels Machine-Learning-Verfahren wurden anhand der extrahierten Kurve Blutdruckwerte geschätzt. Dafür kamen verschiedene neuronale Netzwerke zum Einsatz, der Fokus lag dabei auf dem ResNet, ein besonders tiefes faltendes neuronales Netzwerk. Die Netzwerkstrukturen wurden mit photoplethysmographischen Daten der MIMIC-III-Datenbank vortrainiert, bevor ihnen die videobasierten Daten zugeführt wurden. Die Datenerhebung der Videoaufnahmen erfolgte im Rahmen der klinischen Studie auf der Intensivstation des Universitätsklinikums Leipzig bei insgesamt 50 Patienten, von denen jeweils circa zweistündige Videoaufnahmen des Gesichts angefertigt wurden. Die Validierung der Methode erfolgte durch den Vergleich der geschätzten Blutdruckwerte mit den simultan über einen intraarteriellen Katheter gemessenen tatsächlichen Blutdruck-werten. Zusätzlich wurde der Einfluss von Personalisierung auf die Performance der Methode untersucht. Es konnte keine verlässliche Blutdruckschätzung erreicht werden, die geforderte Standards an Blutdruckmessgeräte erfüllt. Personalisierung führte jedoch zu einer signifikanten Verbesserung des mittleren absoluten Fehlers in der Vorhersagegenauigkeit des Algorithmus und stellt einen vielversprechenden Ansatz für zukünftige Methoden dar.:1 EINLEITUNG - 1 - 1.1 Entdeckung des Blutdrucks - 1 - 1.2 Anfänge instrumentalisierter Blutdruckmessung - 2 - 1.3 Weg zur Moderne - 3 - 1.3.1 Einführung der Armmanschette - 3 - 1.3.2 Oszillometrische Methode - 5 - 1.3.3 Invasive Blutdruckmessung - 5 - 1.4 Vitalparameter-Extraktion aus Videodaten - 7 - 1.4.1 Messprinzip - 7 - 1.4.2 Kontaktlose Photoplethysmographie - 8 - 1.4.3 Bildregionen zur Signalextraktion - 9 - 1.4.4 Aktuelle Ansätze - 10 - 2 ALLGEMEINE GRUNDLAGEN - 13 - 2.1 Haut - 13 - 2.2 Vitalparameter - 16 - 2.3 Herzaktion und Windkessel - 17 - 2.4 Strom- und Druckpuls - 18 - 2.5 Widerstandsgefäße und Ohm’sches Gesetz - 20 - 2.6 Kapillaren und Stoffaustausch - 21 - 2.7 Steuerung der kapillären Durchblutung - 22 - 2.8 Kurzfristige Regulation des Blutdrucks - 23 - 2.9 Langfristige Regulation des Blutdrucks - 24 - 2.10 Störungen des Blutdrucks - 26 - 2.10.1 Hypertension - 26 - 2.10.2 Hypotension und Schock - 28 - 2.10.2.1 Anaphylaktischer Schock - 29 - 2.10.2.2 Kardiogener Schock - 29 - 2.10.2.3 Hypovolämischer Schock - 30 - 2.10.2.4 Septischer Schock - 31 - 2.11 Künstliche neuronale Netzwerke - 32 - 2.11.1 Multilayer Perceptron - 32 - 2.11.2 Convolutional Neural networks - 33 - 2.11.3 Mathematische Funktionsweise - 35 - 2.12 Maschinelles Lernen - 36 - 2.12.1 Anwendungsbereiche - 38 - 3 MATERIALIEN UND METHODEN - 40 - 3.1 Studiendesign - 40 - 3.1.1 Patientenrekrutierung - 40 - 3.2 Daten-Akquisition - 41 - 3.2.1 Erhebung der Videodaten - 41 - 3.2.2 Arterielle Blutdruckmessung - 43 - 3.3 Datenverarbeitung - 44 - 3.3.1 Prozessierung der Videodaten - 44 - 3.3.2 Extraktion des rPPG-Signals - 47 - 3.3.3 Aufbereitung des rPPG-Signals - 47 - 3.4 Neuronale Netzwerkstrukturen - 48 - 3.4.1 PPG-basiertes Training - 49 - 3.4.2 Personalisierung - 51 - 3.4.3 Transfer learning - 52 - 3.5 Statistik - 52 - 4 ERGEBNISSE - 53 - 4.1 Patientenpopulation - 53 - 4.2 Blutdruckvorhersage - 55 - 5 DISKUSSION - 60 - 5.1 Patientenpopulation - 60 - 5.2 Trainingsdaten - 60 - 5.3 Klinisches Setting - 63 - 5.4 Datenvorbehandlung - 66 - 5.5 Ergebnisse - 67 - 5.6 Personalisierung - 71 - 6 SCHLUSSFOLGERUNG - 74 - 7 ZUSAMMENFASSUNG DER ARBEIT - 76 - 8 ABBILDUNGSVERZEICHNIS - 80 - 9 LITERATURVERZEICHNIS - 82 -

Les tests d'écoute subjectifs permettent d'évaluer la fiabilité de reproduction des systèmes de codage audio (codecs). Le projet présenté ici vise à évaluer la possibilité d'utiliser les réactions physiologiques (activité électrodermale, cardiaque, musculaire et cérébrale) à la place d'une note donnée par l'auditeur, afin de caractériser la performance d'un codec. Ceci permettrait d'avoir une méthode d'évaluation plus proche de la perception réelle de qualité audio du sujet. Des tests d'écoute mettant en oeuvre des dégradations audio bien connues en concours avec la mesure des réactions physiologiques ont été réalisés pour 4 auditeurs. L'analyse des résultats montre que certaines caractéristiques physiologiques permettent d'avoir une information fiable sur la qualité audio perçue, et ce de manière répétable pour près de 70% des signaux audio testés chez un sujet, et près de 60% des séquences audio testées chez tous les sujets. Ceci permet de postuler sur la faisabilité d'une telle méthode d'évaluation subjective des codecs audio.

Imaging photoplethysmography (iPPG) has attracted much attention over the last years. The vast majority of works focuses on methods to reliably extract the heart rate from videos. Only a few works addressed iPPGs ability to exploit spatio-temporal perfusion pattern to derive further diagnostic statements. This work directs at the spatio-temporal analysis of blood perfusion from videos. We present a novel algorithm that bases on the two-dimensional representation of the blood pulsation (perfusion map). The basic idea behind the proposed algorithm consists of a pairwise estimation of time delays between photoplethysmographic signals of spatially separated regions. The probabilistic approach yields a parameter denoted as perfusion speed. We compare the perfusion speed versus two parameters, which assess the strength of blood pulsation (perfusion strength and signal to noise ratio). Preliminary results using video data with different physiological stimuli (cold pressure test, cold face test) show that all measures are in fluenced by those stimuli (some of them with statistical certainty). The perfusion speed turned out to be more sensitive than the other measures in some cases. However, our results also show that the intraindividual stability and interindividual comparability of all used measures remain critical points. This work proves the general feasibility of employing the perfusion speed as novel iPPG quantity. Future studies will address open points like the handling of ballistocardiographic effects and will try to deepen the understanding of the predominant physiological mechanisms and their relation to the algorithmic performance.

Camera-based photoplethysmography (cbPPG) is a novel measurement technique that allows the continuous monitoring of vital signs by using common video cameras. In the last decade, the technology has attracted a lot of attention as it is easy to set up, operates remotely, and offers new diagnostic opportunities. Despite the growing interest, cbPPG is not completely established yet and is still primarily the object of research. There are a variety of reasons for this lack of development including that reliable and autonomous hardware setups are missing, that robust processing algorithms are needed, that application fields are still limited, and that it is not completely understood which physiological factors impact the captured signal. In this thesis, these issues will be addressed. A new and innovative measuring system for cbPPG was developed. In the course of three large studies conducted in clinical and non-clinical environments, the system’s great flexibility, autonomy, user-friendliness, and integrability could be successfully proven. Furthermore, it was investigated what value optical polarization filtration adds to cbPPG. The results show that a perpendicular filter setting can significantly enhance the signal quality. In addition, the performed analyses were used to draw conclusions about the origin of cbPPG signals: Blood volume changes are most likely the defining element for the signal's modulation. Besides the hardware-related topics, the software topic was addressed. A new method for the selection of regions of interest (ROIs) in cbPPG videos was developed. Choosing valid ROIs is one of the most important steps in the processing chain of cbPPG software. The new method has the advantage of being fully automated, more independent, and universally applicable. Moreover, it suppresses ballistocardiographic artifacts by utilizing a level-set-based approach. The suitability of the ROI selection method was demonstrated on a large and challenging data set. In the last part of the work, a potentially new application field for cbPPG was explored. It was investigated how cbPPG can be used to assess autonomic reactions of the nervous system at the cutaneous vasculature. The results show that changes in the vasomotor tone, i.e. vasodilation and vasoconstriction, reflect in the pulsation strength of cbPPG signals. These characteristics also shed more light on the origin problem. Similar to the polarization analyses, they support the classic blood volume theory. In conclusion, this thesis tackles relevant issues regarding the application of cbPPG. The proposed solutions pave the way for cbPPG to become an established and widely accepted technology.

Objective. After skin flap transplants, perfusion strength monitoring is essential for the early detection of tissue perfusion disorders and thus to ensure the survival of skin flaps. Camera-based photoplethysmography (cbPPG) is a non-contact measurement method, using video cameras and ambient light, which provides spatially resolved information about tissue perfusion. It has not been researched yet whether the measurement depth of cbPPG, which is limited by the penetration depth of ambient light, is sufficient to reach pulsatile vessels and thus to measure the perfusion strength in regions that are relevant for skin flap transplants. Approach. We applied constant negative pressure (compared to ambient pressure) to the anterior thighs of 40 healthy subjects. Seven measurements (two before and five up to 90 min after the intervention) were acquired using an RGB video camera and photospectrometry simultaneously. We investigated the performance of different algorithmic approaches for perfusion strength assessment, including the signal-to-noise ratio (SNR), its logarithmic components logS and logN, amplitude maps, and the amplitude height of alternating and direct signal components. Main results. We found strong correlations of up to r = 0.694 (p < 0.001) between photospectrometric measurements and all cbPPG parameters except SNR when using the green color channel. The transfer of cbPPG signals to POS, CHROM, and O3C did not lead to systematic improvements. However, for direct signal components, the transformation to O3C led to correlations of up to r = 0.744 (p < 0.001) with photospectrometric measurements. Significance. Our results indicate that a camera-based perfusion strength assessment in tissue with deep-seated pulsatile vessels is possible.

Objective: The heart rate is an essential vital sign that can be measured remotely with camera-based photoplethysmography (cbPPG). Systems for cbPPG typically use cameras that deliver red, green, and blue (RGB) channels. The combination of these channels has been proven to increase signal-to-noise ratio (SNR) and heart rate measurement accuracy (ACC). However, many combinations remain untested, the comparison of proposed combinations on large datasets is insufficiently investigated, and the interplay with skin tone is rarely addressed. Methods: Eight regions of interest and eight color spaces with a total of 25 color channels were compared in terms of ACC and SNR based on the Binghamton-Pittsburgh-RPI Multimodal Spontaneous Emotion Database (BP4D+). Additionally, two systematic grid searches were performed to evaluate ACC in the space of linear combinations of the RGB channels. Results: Glabella and forehead regions of interest provided highest ACC (up to 74.1 %) and SNR (> -3 dB) with the hue channel H from HSV color space and the chrominance channel Q from NTSC color space. The grid searches revealed a global optimum of linear RGB combinations (ACC: 79.2 %). This optimum occurred for all skin tones, although ACC dropped for darker skin tones. Conclusion: Through systematic grid searches we were able to identify the skin tone independent optimal linear RGB color combination for measuring heart rate with cbPPG. Our results proved on a large dataset that the identified optimum outperformed conventionally used color channels. Significance: The presented findings provide useful evidence for future considerations of algorithmic approaches for cbPPG.

No description available.

Background Camera-based photoplethysmography (cbPPG) is a measurement technique which enables remote vital sign monitoring by using cameras. To obtain valid plethysmograms, proper regions of interest (ROIs) have to be selected in the video data. Most automated selection methods rely on specific spatial or temporal features limiting a broader application. In this work, we present a new method which overcomes those drawbacks and, therefore, allows cbPPG to be applied in an intraoperative environment. Methods We recorded 41 patients during surgery using an RGB and a near-infrared (NIR) camera. A Bayesian skin classifier was employed to detect suitable regions, and a level set segmentation approach to define and track ROIs based on spatial homogeneity. Results The results show stable and homogeneously illuminated ROIs. We further evaluated their quality with regards to extracted cbPPG signals. The green channel provided the best results where heart rates could be correctly estimated in 95.6% of cases. The NIR channel yielded the highest contribution in compensating false estimations. Conclusions The proposed method proved that cbPPG is applicable in intraoperative environments. It can be easily transferred to other settings regardless of which body site is considered.

Background Camera-based photoplethysmography (cbPPG) is a measurement technique which enables remote vital sign monitoring by using cameras. To obtain valid plethysmograms, proper regions of interest (ROIs) have to be selected in the video data. Most automated selection methods rely on specific spatial or temporal features limiting a broader application. In this work, we present a new method which overcomes those drawbacks and, therefore, allows cbPPG to be applied in an intraoperative environment. Methods We recorded 41 patients during surgery using an RGB and a near-infrared (NIR) camera. A Bayesian skin classifier was employed to detect suitable regions, and a level set segmentation approach to define and track ROIs based on spatial homogeneity. Results The results show stable and homogeneously illuminated ROIs. We further evaluated their quality with regards to extracted cbPPG signals. The green channel provided the best results where heart rates could be correctly estimated in 95.6% of cases. The NIR channel yielded the highest contribution in compensating false estimations. Conclusions The proposed method proved that cbPPG is applicable in intraoperative environments. It can be easily transferred to other settings regardless of which body site is considered.

A prospective, non-randomized observational study involving forty-nine patients undergoing coronary artery bypass surgery (CABG) with a unilateral harvesting of the internal thoracic artery (ITA) was carried out at the Department of Cardiac Surgery, Herzzentrum Dresden University hospital. Using a commercially available industrial-grade RGB camera and normal indoor lighting, the chest wall of the patients was scanned before surgery and in three follow-up measurements. The primary aim of this thesis was to show whether iPPG is sensitive enough to detect global signal changes after a major surgery – CABG in this case – and local signal changes due to the removal of the ITA, the main supply vessel of the chest wall. As a secondary aim, the thesis looked at subgroups of data to show if differences in signal existed between the colour channels of the RGB camera, subdivisions of the thorax and the surgical technique used as well as to show if demographic factors had an impact on signal strength. With mathematical programs developed by the Technical University Dresden, the scanned optical data was transformed into signal to noise ratios (SNR) used in imaging photoplethysmographic (iPPG) studies. The signal data was analysed in R and, based on a stepwise deletion, a multivariable mixed effects model was constructed. Adjusted versions of this model were used for the analysis of the subgroups of the data. Analysis of the data showed a significant decrease of iPPG signal strength after the CABG surgery with a steeper decrease and an attenuated recovery on the side of the ITA harvesting. Even though the signal variations were relatively small, using the models in this thesis, the differences were reliably detected by iPPG. The analysis of the data from the subdivisions of the chest and from patients’ groups determined by the surgical technique showed a caudo-cranial signal gradient on the ITA side twenty-four hours after the surgery and a stronger signal in the Pedicled group within twenty-four hours after the surgery. The latter calculations, however, were based on a possibly biased sample and should be verified using a controlled sample in prospective randomised study designs. Demographic factors showed no significant correlation with iPPG signal strength. iPPG was able to detect relatively small signal variations that could be associated with changes of cutaneous perfusion after major surgery. Future development could lead to non-invasive monitoring devices in the clinical practice of post-surgery care.:1. Introduction 1 1.1. Coronary Artery Bypass Grafting (CABG) 1 1.1.1. Historical Overview 1 1.1.2. Coronary Grafts 3 1.1.2.1. Pedicled vs. Skeletonised Grafts 4 1.2. Plethysmography 5 1.2.1. Air-Displacement Plethysmography (APG) 5 1.2.2. Strain Gauge Plethysmography (SGP) 6 1.2.3. Impedance Plethysmography (IPG) 6 1.2.4. Photoplethysmography (PPG) 7 1.2.5. Imaging Photoplethysmography (iPPG) 8 1.3. Hypothesis and Aim of the Thesis 11 2. Methods 13 2.1. Study Setting and Patients 13 2.2. Camera and Technical Setup 14 2.3. Recording Area and Regions of Interest 15 2.4. Signal Processing 16 2.5. Statistical Analysis 17 3. Results 19 3.1. Descriptive Properties of the Data 19 3.2. Signal Strength in the Three Colour Channels 20 3.3. Choosing a Multilevel Model 21 3.4. The Effect of the Major Surgery on the Signal Strength in the Three Colour Channels 22 3.5. The Effect of the Unilateral Resection of the Internal Thoracic Artery 25 3.6. Results from the Model Fitted to the Data 27 3.7. The Effect of Cofactors 28 3.8. Data from the Subdivisions of the Chest 29 3.9. The Effect of the Surgical Technique 31 4. Discussion 34 4.1. Signal Strength in the Red, Green and Blue Colour Channels 34 4.2. Signal from the Entire Chest Area 36 4.3. Signal from the Subdivisions of the Chest 37 4.4. The Influence of the Surgical Technique on Signal Strength 38 5. Conclusion 39 6. Abstract 41 7. Zusammenfassung 42 8. References 44 9. Appendix 60 10. Acknowledgements 82 11. Resume 83 Anlage 184 Anlage 2 85 / Eine prospektive, nicht randomisierte Studie mit neunundvierzig Patienten geplant für eine koronare Bypassoperation (CABG) mit einseitiger Präparation der Arteria thoracica interna (ITA) wurde im Herzzentrum Dresden, Universitätsklinikum durchgeführt. In einer präoperativen und in drei postoperativen Messungen wurde die Brustwand bei den untersuchten Patienten unter normaler Innenbeleuchtung mit Hilfe einer handelsüblichen, industriellen RGB Kamera untersucht. Das primäre Ziel der Arbeit war zu zeigen, ob iPPG als Messmethode genug Sensitivität besitzt um globale Signal-Veränderungen nach einem großen Eingriff – die CABG in diesem Fall – und lokale Signaländerung nach der Abnahme der ITA, die Hauptversorgungsarterie der Brustwand, zu erkennen. Als sekundäres Ziel der Arbeit war zu eruieren, ob iPPG Signaldifferenzen zwischen den Farbkanälen der RGB Kamera, den Brustwandaufteilungen und den Arten der ITA Präparation sowie nach den demographischen Faktoren detektieren konnte. Die gemessenen Daten wurden unter Verwendung von Eigentumsprogrammen der Technischen Universität Dresden in den, bei plethysmographischen Studien genutzten, Signal zu Geräusch Quotienten (SNR - signal to noise ratios) umgewandelt. Die gewonnenen Signaldaten wurden in R verarbeitet und durch Verwendung der Methode schrittweise Löschung wurde ein multivariables gemischte Effekte Modell erstellt. Angepasste Versionen dieses Modells wurden für die Analyse von Patientensubgruppen verwendet. Die Datenanalyse ergab eine signifikante Abschwächung des Signals nach der CABG, wobei die Thorax-Seite mit der ITA Präparation zeigte, im Vergleich mit der anderen Thorax-Seite, eine stärkere Abnahme und eine gedämpfte Rückbildung der Signalstärke. Obwohl die detektierte Signaländerungen relativ klein waren, sie konnten durch die entwickelten Modelle mittels iPPG zuverlässig detektiert werden. Die weitere Analyse der Daten aus den Brustwandaufteilungen und von Patientensubgruppen definiert nach Präparationsart der ITA zeigte auf der ITA Seite eine caudo-craniale Zunahme der Signalstärke ab vierundzwanzig Stunden und ein stärkeres Signal in der pedikulierten Präparationsgruppe bis vierundzwanzig Stunden nach der Operation. Allerdings, diese letztere Berechnungen wurden auf einem möglicherweise unausgewogenen Muster durchgeführt und sollten dementsprechend auf kontrollierten Mustern in prospektiven randomisierten Studien verifiziert werden. Die demographischen Faktoren hatten keiner signifikanten Korrelation mit der iPPG Signalstärke. Die iPPG war geeignet kleine Signaländerungen assoziiert mit den erwarteten Änderungen der dermalen Perfusion bei einem großen chirurgischen Eingriff zu detektieren. Weitere Entwicklung der Technologie kann die Anwendung dieses nicht-invasive Monitoringsverfahren in der klinischen postoperativen Patientenversorgung ermöglichen.:1. Introduction 1 1.1. Coronary Artery Bypass Grafting (CABG) 1 1.1.1. Historical Overview 1 1.1.2. Coronary Grafts 3 1.1.2.1. Pedicled vs. Skeletonised Grafts 4 1.2. Plethysmography 5 1.2.1. Air-Displacement Plethysmography (APG) 5 1.2.2. Strain Gauge Plethysmography (SGP) 6 1.2.3. Impedance Plethysmography (IPG) 6 1.2.4. Photoplethysmography (PPG) 7 1.2.5. Imaging Photoplethysmography (iPPG) 8 1.3. Hypothesis and Aim of the Thesis 11 2. Methods 13 2.1. Study Setting and Patients 13 2.2. Camera and Technical Setup 14 2.3. Recording Area and Regions of Interest 15 2.4. Signal Processing 16 2.5. Statistical Analysis 17 3. Results 19 3.1. Descriptive Properties of the Data 19 3.2. Signal Strength in the Three Colour Channels 20 3.3. Choosing a Multilevel Model 21 3.4. The Effect of the Major Surgery on the Signal Strength in the Three Colour Channels 22 3.5. The Effect of the Unilateral Resection of the Internal Thoracic Artery 25 3.6. Results from the Model Fitted to the Data 27 3.7. The Effect of Cofactors 28 3.8. Data from the Subdivisions of the Chest 29 3.9. The Effect of the Surgical Technique 31 4. Discussion 34 4.1. Signal Strength in the Red, Green and Blue Colour Channels 34 4.2. Signal from the Entire Chest Area 36 4.3. Signal from the Subdivisions of the Chest 37 4.4. The Influence of the Surgical Technique on Signal Strength 38 5. Conclusion 39 6. Abstract 41 7. Zusammenfassung 42 8. References 44 9. Appendix 60 10. Acknowledgements 82 11. Resume 83 Anlage 184 Anlage 2 85