Recording and monitoring vital signs is an essential aspect of home-based healthcare. Using contact sensors to record physiological signals can cause discomfort to patients, especially after prolonged use. Hence, remote physiological measurement approaches have attracted considerable attention as they do not require physical contact with the patient’s skin. Several studies proposed techniques to measure Heart Rate (HR) and Heart Rate Variability (HRV) by detecting the Blood Volume Pulse (BVP) from human facial video recordings while the subject is in a resting condition. In this thesis, we focus on the measurement of HR. We adopt an algorithm that uses the Independent Component Analysis (ICA) to separate the source (physiological) signal from noise in the RGB channels of a facial video. We generalize existing methods to support subject movement during video recording. When a subject is moving, the face may be turned away from the camera. We utilize multiple cameras to enable the algorithm to monitor the vital sign continuously, even if the subject leaves the frame or turns away from a subset of the system’s cameras. Furthermore, we improve the accuracy of existing methods by implementing a light equalization scheme to reduce the effect of shadows and unequal facial light on the HR estimation, a machine learning method to select the most accurate channel outputted by the ICA module, and a regression technique to adjust the initial HR estimate. We systematically test our method on eleven subjects using four cameras. The proposed method decreases the RMSE by 27% compared to the state of the art in the rest condition. When the subject is in motion, the proposed method achieves a RMSE of 1.12 bpm using a single camera and RMSE of 0.96 bpm using multiple camera.

Photoplethysmogram (PPG) signals detect blood volume variations during the heart cycle. They are useful to track physiological parameters of an individual, such as heart rate, heart rate variability or oxygen saturation. They are typically obtained using smart wearables and pulse oximeters, but our goal is to create remote PPG (rPPG) signals from video cameras. Since the signals obtained from a video camera are the RGB channels, we carried out an empirical study of the performance of each channel. RGB channels can be used to generate rPPG signals, but also as input to other processes that do so. As reference ground truth, we use contact PPG (cPPG) readings from pulse oximeters in the fingertip. In terms of several metrics, including dynamic time warping (DTW), Pearson’s correlation coefficient, root mean squared error (RMSE), and Beats-per-minute Difference (|∆BPM|), the green channel produced the best results, followed by the blue and red channels. Despite the green channel consistently outperforming the blue and red channels, the outcomes varied greatly depending on the dataset. We also applied different methods to obtain rPPG signals from the RGB channels, including CHROM-based rPPG, local group invariance (LGI), and plane-orthogonal-to-skin (POS). These techniques were contrasted with our novel technique based on a machine learning approach. For that, we made use of a variety of architectures, including convolutional neural networks and long short-term memory. The results were favourable for the ML approach in terms of DTW, r and |∆BPM|. / Fotopletysmogram (PPG)-signaler upptäcker variationer av blodvolym under hjärtcykeln. De är användbara för att spåra fysiologiska parametrar för en individ, såsom hjärtfrekvens, hjärtfrekvensvariabilitet eller syremättnad. De erhålls vanligtvis med smarta bärbara sensorer och pulsoximetrar, men vårt mål är att skapa fjärr-PPG (rPPG)-signaler från videokameror. Eftersom signalerna erhållna från en videokamera är RGB -kanalerna genomförde vi en empirisk studie av prestandan för varje kanal. RGB-kanaler kan användas för att generera rPPG-signaler, men också som input till andra processer som gör det. Som referens använder vi kontakt-PPG (cPPG) avläsningar från pulsoximetrar i fingertoppen. När det gäller flera mätvärden, inklusive Dynamic Time Warping (DTW), Pearsons korrelationskoefficient, Root Mean Squared Error (RMSE) och Beats-Per-minut-skillnaden (|∆BPM|). Uppnåddes bästa resultat med den gröna kanalen, följt av de blå och röda kanalerna. Trots att den gröna kanalen konsekvent överträffade de blå och röda kanalerna varierade resultaten mycket beroende på datasetet. Vi använde också olika metoder för att erhålla rPPG-signaler från RGB-kanalerna, inklusive CHROM-baserad rPPG, lokal gruppinvarians (LGI) och plan-ortogonal-till-hud (POS). Dessa tekniker kontrasterades med vår nya teknik baserat på en maskininlärningsstrategi. För det använde vi en mängd olika arkitekturer, inklusive konvolutionella neurala nätverk och LSTM-nätverk. Resultaten var gynnsamma för ML-metoden när det gäller DTW, R och |∆BPM|.

Durch die zunehmende Automatisierung der Tierhaltung gewinnt die Tier-Technik-Beziehung zunehmend an Bedeutung. Es besteht ein wachsendes Interesse am Tierverhalten, den Möglichkeiten des Lernens der Tiere und den Anpassungsstrategien. In der Dissertation untersuchte ich die Melk-, und Milchparameter und das Tierverhalten bei der automatischen Milchgewinnung. Die Messungen haben gezeigt, dass auf die Milchabgabe einzelner Individuen und insbesondere auf deren Euterviertel sehr differenziert einzugehen ist. Das Automatische Melksystem (AMS) vermag auf die speziellen Unterschiede der einzelnen Viertel, im Sinne einer verbesserten Tiergerechtheit, Rücksicht zu nehmen. Ein weiterer wichtiger Aspekt die Tiergerechtheit zu verbessern ist es, eine zuverlässige Prozesskontrolle durchzuführen. Dazu muss zunächst einmal festgestellt werden, wie sich die natürliche Variationsbreite der einzelnen Parameter darstellt, um pathologisch bedingte Abweichungen signifikant erkennen zu können. Die Gewinnung von verlässlichen Aussagen über den Gesundheitszustand von Kühen im AMS setzen voraus, dass verschiedene interdependente Parameter so verknüpft oder deren Messungen sooft wiederholt werden, bis ein intelligentes Entscheidungssystem seine Schlüsse ziehen kann. Dabei ist zu beachten, dass stark auffällige Werte ein hohes Maß an Information haben, aber eine entsprechend geringe Verlässlichkeit aufweisen. Sich wiederholende Werte haben hingegen einen geringen Informationsgehalt, dafür aber ein hohes Maß an Redundanz bzw. Verlässlichkeit, soweit sie nicht durch systematische Fehler entstehen. Für eine zuverlässige Prozesskontrolle kann es manchmal vorteilhafter sein, eine automatisierte Datengewinnung zu installieren, auch wenn der einzelne Parameter (z. B. die Leitfähigkeit) zwar wenig aussagekräftig, dafür aber die Durchführung deutlich zuverlässiger ist als bei Tests, die von Menschen manuell durchgeführt werden. Ähnliches gilt für die Haltungsumwelt von Tieren. Eine durch Automation dominierte Umwelt kann für Tiere deutlich berechenbarer und damit zuverlässiger gestaltet werden. Das bedeutet, dass für die Individuen weniger Stresssituationen mit den für sie unabsehbaren Folgen entstehen. Es sollte aber dabei beachtet werden, dass es zwingend ist, auf die Lerngeschwindigkeit der einzelnen Tiere, in Abhängigkeit von ihrer jeweiligen Lernsituation, einzugehen, um zuverlässige Umweltbedingungen für die Kühe mit einem AMS bereitzustellen. Es konnte weiterhin festgestellt werden, dass das Melken in einem AMS bei den Kühen nicht als belastender Stressfaktor identifiziert werden kann, wenn man die Herzfrequenz als Indikator heranzieht und diese über eine längere Zeit analysiert. Der zunehmende Einsatz von Technik in der Milchviehhaltung kann einen wichtigen Beitrag dazu leisten, die Haltungsumwelt der Kühe human und tiergerecht zu gestalten. / Relationship between animal and technology in automatic milk production: Due to the fact of the increasing automation in husbandry systems becomes the relationship between the animal and the technology more and more important. There is a growing interest to know more about animal behaviour, the ability of learning and the coping strategies in such systems. In the thesis I investigated the parameter of milking, of milk and of animal behaviour in an Automatic Milking System (AMS). The measurement has shown that the milk yield differs very much between the quarters of the udder. An AMS has the possibility to take such differences into consideration. This is one step towards more animal welfare. Another improvement of animal welfare is a better control of the process. For that, it is important to have a certain knowledge about the natural variation of different parameters, such as electrical conductivity of the milk, milk ingredients or milk yield. This makes the basis of the identification of anomalies depending on pathological problems. To get a reliable declaration about the state of health one has to connect different interdependent parameters and/or the measurement has to be repeated so often until an intelligent decision system can draw conclusions. Besides it is important to know, that a conspicuous value is highly informative, but it is less reliable, otherwise is an often repeatable value less informative but highly redundant, if there is no systematic failure. For a control of the process it is important to get reliable information, so it is sometimes better to automat the tests, instead of using human knowledge, which is often more informative, but less reliable (i.e. electrical conductivity). There is a similarity in husbandry systems because an automated system can be much more reliable and calculable for animals. That means less stressing situations because of incalculable reactions. For such a reliable environment in an AMS it is necessary to give every cow their individual time to learn the facts about the AMS. The milking in an AMS cannot be identified as a negative stress factor, if one uses the measurement of heart rate for identification. The increasing automation in the dairy husbandry can be an important contribution to create a humane environment for dairy cows and improve animal welfare.

Exploring surfaces is an essential ability for humans, allowing them to interact with a large variety of objects within their environment. This ability to explore surfaces is also of a major interest in the development of a new generation of humanoid robots, which requires the development of more efficient artificial tactile sensing techniques. The details perceived by statically touching different surfaces of objects not only improve robotic hand performance in force-controlled grasping tasks but also enables the feeling of vibrations on touched surfaces. This thesis presents an extensive experimental study of object surface exploration using biologically-Inspired tactile-enabled robotic fingers. A new multi-modal tactile sensor, embedded in both versions of the robotic fingertips (similar to the human distal phalanx) is capable of measuring the heart rate with a mean absolute error of 1.47 bpm through static explorations of the human skin. A two-phalanx articulated robotic finger with a new miniaturized tactile sensor embedded into the fingertip was developed in order to detect and classify surface textures. This classification is performed by the dynamic exploration of touched object surfaces. Two types of movements were studied: one-dimensional (1D) and two-dimensional (2D) movements. The machine learning techniques - Support Vector Machine (SVM), Multilayer Perceptron (MLP), Random Forest, Extra Trees, and k-Nearest Neighbors (kNN) - were tested in order to find the most efficient one for the classification of the recovered textured surfaces. A 95% precision was achieved when using the Extra Trees technique for the classification of the 1D recovered texture patterns. Experimental results confirmed that the 2D textured surface exploration using a hemispheric tactile-enabled finger was superior to the 1D exploration. Three exploratory velocities were used for the 2D exploration: 30 mm/s, 35 mm/s, and 40 mm/s. The best classification accuracy of the 2D recovered texture patterns was 99.1% and 99.3%, using the SVM classifier, for the two lower exploratory velocities (30 mm/s and 35mm/s), respectively. For the 40 mm/s velocity, the Extra Trees classifier provided a classification accuracy of 99.4%. The results of the experimental research presented in this thesis could be suitable candidates for future development.